Using Puppet, Weave and Docker in Production
Today on the Weave Blog, we welcome Hugh Esco back for another guest post. About six weeks ago, Hugh wrote about migrating his applications to a new data centre using Docker and Weave. As part of that work, Hugh wrote a Puppet...
Today on the Weave Blog, we welcome Hugh Esco back for another guest post. About six weeks ago, Hugh wrote about migrating his applications to a new data centre using Docker and Weave. As part of that work, Hugh wrote a Puppet module for Weave. Here he talks about new features in that module, which he is running in production.
— alexis
hesco/weave: v0.8.x released:
Using Puppet to wire an SDN across multiple Docker hosts
Five weeks since its initial inception, and a month since its last release, the new v0.8.x version of the hesco/weave puppet module has now been released to the puppet forge. Hopefully those 99 downloads of the last version will soon be upgraded and the new features described below will serve to generate a buzz driving new folks to try out this young, but already useful module.
This latest minor release brings three new features to the project.
(1) weave::simple::
The new version of hesco/weave introduces new weave::simple::(run|interface) defined types. These new types provide wrappers around the weave::(run|interface) base types made available in earlier versions of the module. As the namespace implies, they provide a more simple interface to the underlying weave SDN management tools. The same number of arguments are still required, but with the weave::simple::<types>, most of those arguments can now be hidden in your hiera data, keyed off a host name passed across the interface of the ::simple:: types.
(2) weave::firewall::
New types and classes have now been added to a new weave::firewall::namespace, meant to reproduce the firewall management offered by the docker and weave tools, but in a way which is compatible with the use of puppetlabs/firewall for the management of your firewall rules not specific to docker and weave, but nonetheless required to be run on your docker hosts. weave::firewall::docker reproduced the iptables rules created by daemonizing docker. weave::firewall::weave reproduces the firewall rules otherwise generated by `weave launch`.
In addition, weave::firewall::listen_to_peer opens up port 6783 for tcp and udp traffic on the docker host between the weave peers, with a sanity check to prevent the unnecessary rules for a peer listening to itself.
weave::firewall::dnat_published_port currently requires user intervention to invoke it for each port published by EXPOSE in a Dockerfile, or with the -p switch to `docker run` adding rules to the FORWARD chain and the DOCKER chain to provide network address translation through the firewall to the container. A future version should more tightly integrate this functionality with weave::simple::run to automate this functionality from hiera data.
(3) docker inspect output now exposed as Facter facts
Two new facts are exposed by this version as well.
weave_router_ip_on_docker_bridge retruns an IP as a string for finding the weaver router. docker_hosted_containers returns a complex data structure derived from running `docker inspect` on every container running on a docker host and exposing all of the metadata available to puppet as facts, accessible at the same keys used by docker.
Fifty-plus commits in the making, this new functionality has now been managing the configuration in the author’s production environment for the past two weeks. Combined with weave it has provided a stable software defined network bridging communication between docker containers on multiple docker hosts. And it has done so in a way which plays well with puppetlabs/firewall module, bringing the docker / weave SDN under configuration management; and most critically permitting the iptables chains to be anchored with rules to REJECT unwanted traffic — all in a reproducible and easy to manage way.
Check it out on the forge at:
https://forge.puppetlabs.com/hesco/weave
Make your contribution to its ongoing development with feature requests, bug reports, usage questions and pull requests at: https://github.com/hesco/hesco-weave/issues
weave remains the subject of active development, most recently adding DNS functionality which is not yet reflected in this module, but surely deserves to be addressed in an upcoming version.
docker is well on the way to its own 1.4.x release and has become the basis for an entire new ecosystem of development pushing systems administration and IT operations in new directions.
As a companion to the garethr/docker module, I anticipate that puppet shops dockerizing their services will find hesco/weave a valuable tool in their kit, filling an important niche in the docker+weave ecosystem. Still, outstanding issues remain to be addressed. Your input on the roadmap for future development of the hesco/weave module is both appreciated and necessary to ensure that it continues to serve the needs of a growing community of DevOps engineers hosting docker clusters under puppet management.
—–
Hugh Esco is General Manager of YMD Partners, LLC a consultancy offering systems administration, application hosting, telephony and custom development to small, home based and start-up businesses and more established firms applying a start-up approach to their ongoing growth. He also serves Green Parties and their candidates doing business as http://CampaignFoundations.com to serve a niche political clientelle. If YMD Partners can help you leverage Lean Business Development processes (making small bets and conducting controlled experiements in an iterative process to drive product, service and business development in a customer-centric way with an eye on metrics that inform future development choices), please write: hesco@yourmessagedelivered.com.
