On the road to Overlay networking on Docker for Windows

The container networking stack has gone through many rapid improvements on Windows Server 2016, and it's nice to see that some new features are coming out on a regular basis: Docker's release pace is fast, and though they have had a few missteps, most of the discovered bug are promptly addressed.

In this post I want to talk you about the implementation of multi-host networking on Docker for Windows.

On Linux this is supported since Kernel version 3.16 but on Windows, Containers are a recent feature and Overlay networking is likely going to be released pretty soon.

So, let's have a look at what this is and how it works.

As you have learned from my previous posts, the Docker engine communicates with the underlying Host Network Service (HNS) through a Libnetwork plugin. This plugin implements the Docker Container Network Model (CNM) which is composed of three main components:

• A Sandbox, where the network configuration (IP address, mac address, routes and DNS entries) of the container is stored
• An Endpoint linking the container Sandbox to a Network: this is a vNIC in the case of a Windows Container or a vmNIC in case of a Hyper-V container
• A Network, which is a group of Endpoints belonging to different containers that can communicate directly

Behind each Network a built-in Driver performs the actual work of providing the required connectivity and isolation.

There are four possible driver packages inside Libnetwork:

• null
• bridge
• overlay
• remote

No network interface is attached to a container which is started with the Null driver:

docker run -it --network none microsoft/nanoserver powershell

Get-NetAdapter in this case returns nothing. And upon inspection this container will show no network:

In the second case, when you use the Bridge driver, the container won’t have a public IP but will be assigned a private address from the 20-bit private range defined by RFC 1918:

172.16.0.0 - 172.31.255.255 (172.16/12 prefix)

Get-Netadapter will show the virtual Ethernet adapter:

Get-NetAdapter

Name                      InterfaceDescription                    ifIndex
----                      --------------------                    -------
vEthernet (Container N... Hyper-V Virtual Ethernet Adapter #2          19

and Get-NetIpAddress will show the private IP address:

Get-NetIPAddress | Format-Table

ifIndex IPAddress                                       PrefixLength PrefixOrigin
------- ---------                                       ------------ ------------
19      fe80::29aa:cc8a:43f2:ae0f%19                              64 WellKnown   
18      ::1                                                      128 WellKnown   
19      172.31.2.5                                                20 Manual      
18      127.0.0.1                                                  8 WellKnown   

If I inspect this container, I can see the JSON describing the network specifications:

docker container inspect 4a44649f2b8d

Now just a couple of weeks ago (on version v1.13.0-rc1), Docker has implemented the third Driver (read Swarm-mode overlay networking support for windows), which basically means that your Windows running containers will be able to communicate even if they are residing on different hosts.

Actually this is a bit more complicated than that, because Overlay networking has been implemented in the Docker engine but not yet in the HNS service of Windows. So if you try to build a multi-host network you will get the following error message:

docker network create -d overlay --subnet 10.1.1.0/24 multihost
Error response from daemon: HNS failed with error : Catastrophic failure

Same output if you try the PowerShell version:

New-ContainerNet -Driver overlay -Name MultiHost
New-ContainerNet : Docker API responded with status code=InternalServerError, response={"message":"HNS failed witherror : Catastrophic failure "}
At line:1 char:1
+ New-ContainerNet -Driver overlay -Name MultiHost
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  + CategoryInfo          : NotSpecified: (:) [New-ContainerNet], DockerApiException
  + FullyQualifiedErrorId : Docker Client Exception,Docker.PowerShell.Cmdlets.NewContainerNet

Once the required Windows binaries to build a Overlay network will be released, it will be interesting to see if Microsoft is going to embed in a Nano Server the required Key-Value store which has to be accessible to all the Containers belonging to the same Overlay network to be discoverable.

For the moment the most used key-value store is the one provided by Consul, but it is only Linux based so you won’t be able to run it on Windows:

docker run -p 8500:8500 -d consul --name consul
Unable to find image 'consul:latest' locally
latest: Pulling from library/consul
C:\Program Files\Docker\docker.exe: image operating system "linux" cannot be used on this platform.
See 'C:\Program Files\Docker\docker.exe run --help'.

All the same, Overlay networking is going to be soon available for Docker containers on Windows. The first step has been done. Now it is up to Microsoft to do the next move. Stay tuned for more on the subject.