Remote Weaving with Weave 0.10
One of the highlights of Weave 0.10 is the ability to run Weave on remote Docker hosts, in a pure container-only environment where the only access to the hosts is through the Docker Remote API. The weave script, as we know it Weave has...
One of the highlights of Weave 0.10 is the ability to run Weave on remote Docker hosts, in a pure container-only environment where the only access to the hosts is through the Docker Remote API.
The weave script, as we know it
Weave has been containerised from its first release. But it has always been controlled via a script. That script needs to be run on each Docker host which is part of the Weave network, in order to launch the Weave containers, attach application containers to the Weave network, perform diagnostics, etc, etc. In short, everything you do with Weave you do through the script. Since the script needs to perform a number of network configuration tasks it needs to be run as ‘root’.
So in order to use Weave you need the ability to run shell scripts on the hosts, as root. That is not a problem in traditional configuration management and deployment systems like Puppet, Chef, Ansible or Salt, or even the new breed like Terraform and CoreOS CloudConfig. It does however pose a challenge for pure container environments. These have Docker deployed on a number of hosts, often via one of the aforementioned systems, and nothing else, i.e. all interactions with thee hosts have to go through Docker. Which does not sit well with running shell scripts, as root.
The 0.10 release of Weave changes all that…
Still a script, but not as we know it
Weave 0.10 still has a weave script. But this script works quite differently than before. Running it no longer requires ‘root’ privileges. Nor does the script need to execute directly on the Docker hosts; it respects the DOCKER_HOST environment variable, just like the Docker CLI, for interacting with remote Dockers.
Furthermore, the script now is just a very thin convenience wrapper that, under the covers, runs Docker containers. That is handy for humans, and easily automated. It also makes the script dispensable – by driving Docker directly through its Remote API. To see what is involved let’s launch Weave via the script and trace its execution…
$ sh -x weave launch
+ set -e
+ SCRIPT_VERSION=0.10.0
+ [ 0.10.0 = (unreleased version) ]
+ IMAGE_VERSION=0.10.0
+ IMAGE_VERSION=0.10.0
+ DOCKERHUB_USER=weaveworks
+ BASE_EXEC_IMAGE=weaveworks/weaveexec
+ EXEC_IMAGE=weaveworks/weaveexec:0.10.0
+ PROCFS=/proc
+ [ launch = run ]
+ [ launch != --local ]
+ exec_remote launch
+ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /proc:/hostproc -e VERSION -e WEAVE_DOCKER_ARGS -e WEAVEDNS_DOCKER_ARGS -e WEAVE_PASSWORD -e WEAVE_PORT -e WEAVE_CONTAINER_NAME -e DOCKER_BRIDGE -e PROCFS=/hostproc --privileged --net=host weaveworks/weaveexec:0.10.0 --local launch
dc7638905d92acd87cd75cb4398dd9e9597c68a6dced2f902b969583b5963ced
+ exit 0
All the lines leading up to EXEC_IMAGE=... are just determining the Docker image to run. The script then checks what weave command we are attempting to invoke, since some require special handling. It then runs a container, which looks quite complicated at first glance, but is in fact quite straightforward. Firstly, let’s strip out all the environment variables that get passed along, since typically few, if any, will be set. We are left with
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /proc:/hostproc -e PROCFS=/hostproc --privileged --net=host weaveworks/weaveexec:0.10.0 --local launch
As we can see, the container is based on the weavexec image, and is run with the arguments --local launch. It runs in privileged mode (--privileged), in the network namespace of the Docker host (--net=host), and is removed immediately after termination (--rm). There are two volume mounts; one for the Docker daemon unix domain socket (-v /var/run/docker.sock:/var/run/docker.sock), which is how the container interact with the remote Docker it is running on, and one for the Docker hosts’ /proc filesystem (-v /proc:/hostproc -e PROCFS=/hostproc), which is accessed for some network configuration tasks.
So what does this container do? It executes the weave script. Hang on, what? We have just run the script, haven’t we? Yes, but the script has two modes of operation. We have executed it in the (default) remote mode. When the script is run inside the weaveexec container, it executes in ‘local’ mode, selected with the --local flag. That makes it operate like the weave script of old, except it is running inside a container, on the remote docker host, with all its dependencies packaged in the image.
All weave commands get executed that way, e.g. weave connect host22 becomes
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /proc:/hostproc -e PROCFS=/hostproc --privileged --net=host weaveworks/weaveexec:0.10.0 --local connect host22
And weave run 10.2.0.1/24 -ti ubuntu becomes
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /proc:/hostproc -e PROCFS=/hostproc --privileged --net=host weaveworks/weaveexec:0.10.0 --local run 10.2.0.1/24 -ti ubuntu
Except it doesn’t, but that’s a story for another day…
Summary
As we have seen, Weave 0.10 can be run on remote Docker hosts, without requiring shell access and ‘root’ privileges on those hosts. This makes it possible to deploy Weave in ‘pure’ container-only environments.
Furthermore, all weave commands can be executed directly via the Docker Remote API – the Docker CLI commands we have seen are easily translated into calls on that API; after all, that is what the Docker CLI does. As a result Weave can be controlled programmatically through the Docker Remote API.
This new behaviour makes Weave 0.10 a well-behaved citizen in a container-centric world. And there is more to come. Projects like Powerstrip-Weave provide a glimpse of the future, allowing users to start application containers attached to the weave network by simply invoking docker run. Docker Network Plugins, which we are busily working on with the Docker team, will offer a route for seamless integration of Weave functionality with Docker, enabling key operations to be executed idiomatically via the Docker Remote API and CLI.
