Bridge over Troubled Weavers
When we originally shipped Weave Net, there were a couple of things that people looked for extra help with: giving each container a unique IP address running containers using the Docker CLI or remote API, not via our weave script...
When we originally shipped Weave Net, there were a couple of things that people looked for extra help with:
- giving each container a unique IP address
- running containers using the Docker CLI or remote API, not via our
weavescript
Several people adopted the technique of telling Docker to use Weave’s network bridge, which has the side-effect that now Docker will allocate IP addresses on the weave network. We do not recommend doing this and we do not include this configuration in our testing.
Note that since weave Net release 0.11 we have included IP Address Management and a Docker API proxy which addresses the requirements cited above.
The pitfalls of running docker with --bridge=weave include:
- Linux’ TCP stack will sometimes attempt to send packets with an incorrect checksum, or that are far too large for the network link, with the result that the packet is rejected and TCP has to re-transmit. This slows down network throughput enormously. When using weave as designed, the virtual ethernet device assigned to each container is configured to avoid this, but with
--bridge=weaveDocker creates the virtual ethernet device and weave doesn’t get a chance to do its configuration. - Docker’s bridge network is inherently single-host, so the administrator has to give each host a different range of addresses to work with. Weave’s built-in allocator automatically shares the address space across hosts.
- Statically carving up the address space limits the flexibility – if you’ve already shared out all the space, then you need to go back and re-configure every host when you want to add some new ones, or expand the space given to one host.
- You cannot give containers addresses in several different subnets, to achieve isolation.
- Containers are not registered in weave’s Service Discovery subsystem, which lets you easily find their address from inside another container elsewhere on the network.
If you have already installed weave using the --bridge=weave technique, please try it without. Do get in touch if you find that the built-in features don’t solve your problem: we want to make weave the most productive way for developers to connect, observe and control Docker containers.
(Picture by Chris Combe, used under Creative Commons license)
