Exciting new things for Docker with Windows Server 1709

What a difference a year makes… last September, Microsoft and Docker launched Docker Enterprise Edition (EE), a Containers-as-a-Service platform for IT that manages and secures diverse applications across disparate infrastructures, for Windows Server 2016. Since then we’ve continued to work together and Windows Server 1709 contains several enhancements for Docker customers.

Docker Enterprise Edition Preview

To experiment with the new Docker and Windows features, a preview build of Docker is required. Here’s how to install it on Windows Server 1709 (this will also work on Insider builds):

Install-Module DockerProvider
Install-Package Docker -ProviderName DockerProvider -RequiredVersion preview

To run Docker Windows containers in production on any Windows Server version, please stick to Docker EE 17.06.

Docker Linux Containers on Windows

A key focus of Windows Server version 1709 is support for Linux containers on Windows. We’ve already blogged about how we’re supporting Linux containers on Windows with the LinuxKit project.

To try Linux Containers on Windows Server 1709, install the preview Docker package and enable the feature. The preview Docker EE package includes a full LinuxKit system (all 13MB of it) for use when running Docker Linux containers.

[Environment]::SetEnvironmentVariable("LCOW_SUPPORTED", "1", "Machine")
Restart-Service Docker

To disable, just remove the environment variable:

[Environment]::SetEnvironmentVariable("LCOW_SUPPORTED", $null, "Machine")
Restart-Service Docker

Docker Linux containers on Windows is in preview, with ongoing joint development by Microsoft and Docker. Linux Containers is also available on Windows 10 version 1709 (“Creators Update 2”). To try it out, install the special Docker for Windows preview available here.

Docker ingress mode service publishing on Windows

Parity with Linux service publishing options has been highly requested by Windows customers. Adding support for service publishing using ingress mode in Windows Server 1709 enables use of Docker’s routing mesh, allowing external endpoints to access a service via any node in the swarm regardless of which nodes are running tasks for the service.

These networking improvements also unlock VIP-based service discovery when using overlay networks so that Windows users are not limited to DNS Round Robin.

Check out the corresponding post on the Microsoft Virtualization blog for details on the improvements.

Named pipes in Windows containers

A common and powerful Docker pattern is to run Docker containers that use the Docker API of the host that the container is running on, for example to start more Docker containers or to visualize the containers, networks and volumes on the Docker host. This pattern lets you ship, in a container, software that manages or visualizes what’s going on with Docker. This is great for building software like Docker Universal Control Plane.

Running Docker on Linux, the Docker API is usually hosted on Unix domain socket, and since these are in the filesystem namespace, sockets can be bind-mounted easily into containers. On Windows, the Docker API is available on a named pipe. Previously, named pipes where not bind-mountable into Docker Windows containers, but starting with Windows 10 and Windows Server 1709, named pipes can now bind-mounted.

Jenkins CI is a neat way to demonstrate this. With Docker and Windows Server 1709, you can now:

  1. Run Jenkins in a Docker Windows containers (no more hand-installing and maintaining Java, Git and Jenkins on CI machines)
  2. Have that Jenkins container build Docker images and run Docker CI/CD jobs on the same host

I’ve built a Jenkins sample image (Windows Server 1709 required) that uses the new named-pipe mounting feature. To run it, simple start a container, grab the initial password and visit port 8080. You don’t have to setup any Jenkins plugins or extra users:

> docker run -d -p 8080:8080 -v \\.\pipe\docker_engine:\\.\pipe\docker_engine friism/jenkins
3c90fdf4ff3f5b371de451862e02f2b7e16be4311903649b3fc8ec9e566774ed
> docker exec 3c cmd /c type c:\.jenkins\secrets\initialAdminPassword
<password>

Now create a simple freestyle project and use the “Windows Batch Command” build step. We’ll build my fork of the Jenkins Docker project itself:

git clone --depth 1 --single-branch --branch add-windows-dockerfile https://github.com/friism/docker-3 %BUILD_NUMBER%
cd %BUILD_NUMBER%
docker build -f Dockerfile-windows -t jenkins-%BUILD_NUMBER% .
cd ..
rd /s /q %BUILD_NUMBER%

Hit “Build Now” and see Jenkins (running in a container) start to build a CI job to build a container image on the very host it’s running on!

Smaller Windows base images

When Docker and Microsoft launched Windows containers last year, some people noticed that Windows container base images are not as small as typical Linux ones. Microsoft has worked very hard to winnow down the base images, and with 1709, the Nanoserver download is now about 70MB (200MB expanded on the filesystem).

One of the things that’s gone from the Nanoserver Docker image is PowerShell. This can present some challenges when authoring Dockerfiles, but multi-stage builds make it fairly easy to do all the build and component assembly in a Windows Server Core image, and then move just the results into a nanoserver image. Here’s an example showing how to build a minimal Docker image containing just the Docker CLI:

# escape=`
FROM microsoft/windowsservercore as builder
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
RUN Invoke-WebRequest -Uri https://download.docker.com/win/static/test/x86_64/docker-17.09.0-ce-rc1.zip -OutFile 'docker.zip'
RUN Expand-Archive -Path docker.zip -DestinationPath .

FROM microsoft/nanoserver
COPY --from=builder ["docker\\docker.exe", "C:\\Program Files\\docker\\docker.exe"]
RUN setx PATH "%PATH%;C:\Program Files\docker"
ENTRYPOINT ["docker"]

You now get the best of both worlds: Easy-to-use, full-featured build environment and ultra-small and minimal runtime images that deploy and start quickly, and have minimal exploit surface area. Another good example of this pattern in action are the .NET Core base images maintained by the Microsoft .NET team.

Summary

It’s hard to believe that Docker Windows containers GA’d on Windows Server 2016 and Windows 10 just one year ago. In those 12 months, we’ve seen lots of adoption by the Docker community and lots of uptake with customers and partners. The latest release only adds more functionality to smooth the user experience and brings Windows overlay networking up to par with Linux, with smaller container images and with support for bind-mounting named pipes into containers.

To learn more about Docker solutions for IT:

, , , ,

Exciting new things for Docker with Windows Server 1709


19 Responses to “Exciting new things for Docker with Windows Server 1709”

  1. Christian Sparre

    Exciting indeed, was trying out Linux container support and got the hello-world:linux image running. You might mention that Hyper-V should be enabled and Hyper-V nesting should be enabled for the VM. I tried running ubuntu but got the following error when pulling the image:

    failed to register layer: failed to start service utility VM
    and
    encountered an error during CreateContainer: failure in a Windows system call: Not enough storage is available to complete this operation.

    I'm not sure what it refers to, but there are plenty memory and storage on the VM

    Reply
    • Christian Sparre

      The above error seems to have something to do with available memory, tried increasing the host VM memory to 8GB, and I could start ubuntu. It failed when extracting the fs layers.

      Reply
      • Bart

        Hi Christian,
        I get a comparable error at build time, at a COPY statement. I have this running directly on my 16GB ram PC.

        "failed to copy files: failed to start service utility VM (lcowfs.startVM): container 85031661542e4b93e73cd3540200f06226de535145e6845b2924f3f9b4d5d909_svm encountered an error during CreateContainer: failure in a Windows system call: The file or directory is corrupted and unreadable."

        Files and everything are perfectly fine, because the same exact setup does build when targeting the stable linux Docker host.

        Reply
      • Michael Friis

        Yes, this suggests that you need more memory.

        Some operations are not super reliable, so you might just want to retry if you see other problems.

        Reply
  2. Bart

    Hi Michael, this is great news! I tried out the special Docker for Windows preview on an insider build, and running the alpine image seems to work fine. I do get some other issues with running other images, but I'm sure that's because this is all very beta still.

    The Docker for Windows preview does note that running Windows payloads next to Linux payloads isn't possible just yet, exactly what I'm trying to achieve. Will this be available in the final release of 1709 in october?

    Reply
  3. Rich

    Hi Michael,

    Does this new version allow Windows containers to establish a RDP session with a Windows host OS?

    The ability to display a containerized app's GUI was first demonstrated by Taylor Brown using Windows Server 2016 Technical Preview 3/4 but was removed in TP 5 right before MS released its initial version of Windows Server 2016. I was hoping that this feature would be reintroduced in a later (hopefully this) server version.

    Reply
  4. Ruben

    Hi Michael,

    Looks like there are some really interesting changes coming up.

    I am however a little confused about the removal of Powershell from Nanoserver. To me, having Powershell is a minimum requirement for a Windows Container. I need it when I create my images with Dockerfile and I also need it when I want to check some things on the container with docker exec.

    I saw that there is a workaround for the Dockerfile by using multi-stage builds but this feels a bit like hacking something that shouldn't need hacking.

    Will it be possible to create our own custom image that has nanoserver + Powershell? (basically installing Powershell from bash in Dockerfile)

    Reply
  5. Ken

    I have the Docker preview running on Windows Server 1709. I was able to run a microsoft/nanoserver container. I then set the LCOW_SUPPORTED to 1, restarted the Docker service and was able to run an alpine container. However, when I unset LCOW_SUPPORTED, restarted the Docker service and attempted to run a microsoft/nanoserver container again it failed with the following error.

    PS C:\programdata\docker> docker container run -it microsoft/nanoserver:1709
    C:\Program Files\Docker\docker.exe: Error response from daemon: container 374c1b271842ca7c9d5de4c632dd03287bdc77b8212cb7ce319871fbd493f3c7 encountered an error during CreateContainer: failure in a Windows system call: The operating system of the container does not match the operating system of the host. (0xc0370101) extra info: {"SystemType":"Container","Name":"374c1b271842ca7c9d5de4c632dd03287bdc77b8212cb7ce319871fbd493f3c7","Owner":"docker","VolumePath":"\\\\?\\Volume{4773913d-fd0f-45d4-a8e3-2902bbe20937}","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\\ProgramData\\docker\\windowsfilter\\374c1b271842ca7c9d5de4c632dd03287bdc77b8212cb7ce319871fbd493f3c7","Layers":[{"ID":"9d958f6a-b6cc-50f1-896f-b52470fe31df","Path":"C:\\ProgramData\\docker\\windowsfilter\\92392eff11be0d494efb3c0b9fae64b59464838a30f2645a9f95db48ca5a18bf"},{"ID":"55f5a56b-7211-5656-a3dd-0e3288c1b48e","Path":"C:\\ProgramData\\docker\\windowsfilter\\0e0c3c243b293c84487e0fe94c9b709f9567a11ded6e98dd4b433aa03ed4badf"}],"HostName":"374c1b271842","HvPartition":false,"EndpointList":["70a3df8b-ba7c-433d-b64a-7c23115f9749"],"AllowUnqualifiedDNSQuery":true}.

    We've tried a variety of ways to unset the environment variable, restart the service and rebooting the server. We've also gotten the same behavior on a second server.

    Any ideas?

    Reply
    • Igor

      I have the same issue with latest insider preview, but I never able to run any windows container there, with docker ee preview, docker ee stable, docker ce 17.09, latest docker binaries. Always get The operating system of the container does not match the operating system of the host.
      Containers: 0
      Running: 0
      Paused: 0
      Stopped: 0
      Images: 2
      Server Version: 17.10.0-ee-preview-3
      Storage Driver: windowsfilter
      Windows:
      Logging Driver: json-file
      Plugins:
      Volume: local
      Network: ics l2bridge l2tunnel nat null overlay transparent
      Log: awslogs etwlogs fluentd json-file logentries splunk syslog
      Swarm: inactive
      Default Isolation: process
      Kernel Version: 10.0 16278 (16278.1000.amd64fre.rs3_release.170825-1441)
      Operating System: Windows Server Standard
      OSType: windows
      Architecture: x86_64
      CPUs: 2
      Total Memory: 8GiB
      Name: WIN-447DPV82V22
      ID: CEFS:L4OD:AJOD:7HXW:AGAI:EVGX:LI74:HCDG:JQNY:MUZC:3EXK:DHJK
      Docker Root Dir: C:\ProgramData\docker
      Debug Mode (client): false
      Debug Mode (server): false
      Registry: https://index.docker.io/v1/
      Experimental: true
      Insecure Registries:
      127.0.0.0/8
      Live Restore Enabled: false

      Client:
      Version: 17.10.0-ee-preview-3
      API version: 1.33
      Go version: go1.8.4
      Git commit: 1649af8
      Built: Fri Oct 6 17:52:28 2017
      OS/Arch: windows/amd64

      Server:
      Version: 17.10.0-ee-preview-3
      API version: 1.34 (minimum version 1.24)
      Go version: go1.8.4
      Git commit: b8571fd
      Built: Fri Oct 6 18:01:48 2017
      OS/Arch: windows/amd64
      Experimental: true

      Reply
    • Imran Qureshi

      Having run into this too, you have to install the latest Preview version of docker: https://www.thomasmaurer.ch/2017/10/how-to-run-docker-linux-container-on-windows-server-1709/

      Reply
  6. Antonio Di Motta

    What is the way to force the update to 1709 on windows server 2016?

    Reply
  7. James Klimek

    We were able to get our Windows docker swarm(s) up and running today on 1709 and Docker EE-17.10-Preview – really fantastic technology – great job!!

    Is there any guidance on the timeline for EE17.10 to go to GA?

    Again – we were amazed at how easy it was to set up the swarms and scale!

    Reply

Leave a Reply to Michael Friis

Click here to cancel reply.

Get the Latest Docker News by Email

Docker Weekly is a newsletter with the latest content on Docker and the agenda for the upcoming weeks.