Dockerizing a full .NET App for Windows Containers running in Kubernetes

Windows containers have been out there for a few years but I feel like it is now time to have our full .NET apps running in Kubernetes for production. The reason why I say this is because Kubernetes starts supporting Windows Containers running in AKS from version v1.14 and the supported Windows node’s OS must be Windows Server 1809/Windows Server 2019 or later. Of course, you don’t have to run a full .NET app in a Windows node but the question is why it even exists when you don’t have a plan to run them. That discussion is not the purpose of this post though.

Microsoft is working hard on Windows containers to keep up with Linux. This is good news. They came up with a lightweight version of Windows Server Core called Nano Server. However, these two operation systems are not the same. It’s kind of one way to compete with Linux. It’s lightweight but does not fully support the full .NET. This makes sense because a lot of .NET components are for GUI. Nano Server is headless, there is no local logon capability or graphical user interface. You can refer to this docs for more information on that topic. This will be one of the factors when choosing the underlying Windows OS for your container images.

Many of my projects were built in the full .NET, some are impossible to migrate to .NET Core due to their dependencies are not implemented in the new framework. For example, I have an integration system built for Dynamics CRM using CRM SDK and this SDK does not support .NET Core. So, nether Nano Server nor Linux OS can be used for my container images.

Since I already set up my Kubernetes cluster in Azure AKS, I’d love to deploy these projects to it.

The picture bellow highlights the differences.

Source: https://docs.microsoft.com/en-us/dotnet/architecture/microservices/net-core-net-framework-containers/net-container-os-targets

There are a couple things you need to pay attention to when dockerizing a full .NET app for Windows container.

• Choosing the correct Docker base image per .NET app type. You can read this documentation for more.
• Matching your build image with one of the Windows nodes in your AKS cluster

If you simply run your full .NET containers in your local Windows machine, you don’t need to worry about matching your container’s OS but here we are dockerizing a Windows container for Kubernetes hosted in AKS.

Dockerizing a Windows Container involves multi-stages build. Since we are building a full .NET container, we need a .NET SDK and its runtime as build stages.

I will take my project as an example.

I have an integration system built for CRM using CRM SDK. It reads messages from Azure Event Hub and CRUD CRM endpoint.  It’s a full .NET application so I have to use either: Windows Server Core image or Windows.

My first attempt was to use the 4.8 base images for both build SDK & runtime

FROM mcr.microsoft.com/dotnet/framework/runtime:4.8 AS base
...
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8 AS build

Since my application is targeting .NET 4.8, I thought that would do for it. It built and ran successfully on my local machine (Windows 10 enterprise) but when I deployed to Kubernetes, pods were failed to schedule. The error message was to do with the underlying container’s os does not match the host’s os (the Windows node).

I then changed my base image to the Windows Server image.

<code class="language-yaml">
FROM mcr.microsoft.com/dotnet/framework/runtime:4.8-windowsservercore-1909 AS base
...
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-1909 AS build

I thought 1909 was the latest version of the image and had more features added to it to support the latest Kubernetes version but I was wrong again.

I kept asking myself what does it mean by “your underlying container’s os must match your host version”.

The findings

First, check your node version using kubectl:

kubectl describe node akswinnp000000

As you can see my node’s OS image is Windows Server 2019 Datacenter, build version 17763

Capacity:
  attachable-volumes-azure-disk:  4
  cpu:                            2
  ephemeral-storage:              133703676Ki
  memory:                         8388148Ki
  pods:                           30
Allocatable:
  attachable-volumes-azure-disk:  4
  cpu:                            1900m
  ephemeral-storage:              133703676Ki
  memory:                         4403764Ki
  pods:                           30
System Info:
  Machine ID:                 akswinnp000000
  System UUID:                91CBA462-2958-4276-8DB9-94CDEFB72D54
  Boot ID:
  Kernel Version:             10.0.17763.1282
  OS Image:                   Windows Server 2019 Datacenter
  Operating System:           windows
  Architecture:               amd64
  Container Runtime Version:  docker://19.3.5
  Kubelet Version:            v1.17.7
  Kube-Proxy Version:         v1.17.7

Looking at this table, my base image must be 1809 (build 17763)

Source: https://en.wikipedia.org/wiki/Windows_10_version_history

Now taking the full build version: 10.0.17763.1282 and look it up on this table, mine was in the table highlighted in red.

I know my node does have long term support servicing channel so, my base images should be

FROM mcr.microsoft.com/dotnet/framework/runtime:4.8-windowsservercore-ltsc2019 AS base
...
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019 AS build

If your node version is different than mine, you can refer to this link to check yours.

Now the Dockerfile itself:

FROM mcr.microsoft.com/dotnet/framework/runtime:4.8-windowsservercore-ltsc2019 AS base
WORKDIR /app
ARG BUILD_ENV_ARG=development
ENV DOTNET_ENVIRONMENT=$BUILD_ENV_ARG
LABEL maintainer=""
#SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; Set-ExecutionPolicy Unrestricted -Force;"]
#SHELL ["powershell", "-Command", "tzutil.exe /s 'AUS Eastern Standard Time'; Set-ExecutionPolicy Unrestricted -Force;"]
#COPY registry.reg /
#RUN REG IMPORT C:\registry.reg
FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019 AS build
WORKDIR /src
COPY ["Integration.Jobs/Integration.Jobs.csproj", "Integration.Jobs/"]
COPY ["Integration.Domain/Integration.Domain.csproj", "Integration.Domain/"]
COPY ["Integration.Data/Integration.Data.csproj", "Integration.Data/"]
COPY ["Integration.Shared/Integration.Shared.csproj", "Integration.Shared/"]
RUN dotnet restore "Integration.Jobs/Integration.Jobs.csproj"
COPY . .
WORKDIR "/src/Integration.Jobs"
RUN powershell dir
RUN dotnet build "Integration.Jobs.csproj" -f net48 -c Release -o /app/build
FROM build AS publish
RUN dotnet publish "Integration.Jobs.csproj" -f net48 -c Release -o /app/publish
FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT Integration.Jobs.exe