Weave Increased Our Flux Capacity
A lot has been happening in Weave Flux. Catch up on Flux’s development, new features and its community!
Since its inception one of the primary motivations for Weaveworks has been to make the management and deployment of containers and workloads as easy as possible. Over time some of our own distilled learnings became GitOps, or as some refer to it “operations by pull request”. Among many other things, it’s what brings clarity and reproducibility to deployments.
Weave Flux has been powering GitOps and Weave Cloud for about two years. It’s what we use at scale internally; it’s what our customers rely on and what’s being worked on out in the open by a growing community.
If you are completely new to Flux and want to learn more, check out Flux’s hello-world tutorial to get your feet wet.
Flux is now 1.8.0
Last on this blog we announced Flux 1.1 (“release all the things”) and initial Helm support. The Flux team has been busy and didn’t blog much, so let’s catch up on what happened since the last blog posts.
More capable
Flux is more capable these days. It has grown support for signed manifests, Kube CronJobs, and it’s become a lot more configurable. Filtering is more straightforward, and supports glob, semver or regular expressions - it’s all there. It now supports Windows images, initContainers, multiple sync paths and a wider variety of Helm charts too. Fluxctl can now properly authenticate to GKE, AWS and other clouds. Image pull secrets are now attached to service accounts. Many more edge-cases are supported now too.
More Secure
Weave Flux is also more secure. Connections to Tiller now use TLS, fluxctl connects to fluxd via port forwarding. Flux works with RBAC now. Baked-in host keys (Github, Gitlab, Bitbucket, etc.) are verified. And more.
More Reliable
Weave Flux is more reliable too. The code regarding YAML operations and the image registry have been rewritten and are failsafe and support more cases today. Changes to the git repo are now verified, resources are now applied in dependency order (e.g. namespaces first). Image updates are more reliable, shutdowns are more graceful, the Helm operator better recovers from failed release installs and it has more validation too.
This rewrite excised so many bugs .. it was the finale of a series of improvements that started with Flux supporting just deployments, in a single file each, and ended with supporting all kinds of workload, and in multidoc files and list docs. Adding verification made it much more reliable too, since it now fails visibly if it hits an unanticipated corner-case, rather than writing a bogus commit to the git repo.
-- Michael Bridgen, Flux maintainer, on the YAML code rewrite
More responsive releases
Weave Flux is faster now too. Releases are more responsive, syncs are faster. The Flux and Helm operator images share more image layers. We reduced the amount of registry polling and respect throttling.
Improved usability
We also put more effort into making Flux more user friendly. Fluxctl now has a “sync” subcommand and it transparently port-forwards to your Flux instance. Things have become more configurable. We have better and clearer logging. The API provides more information, our examples and our docs are up to date and easier to read.
And a word about Helm: particularly the Helm integration has been enthusiastically picked up by many in our community, and steadily improved. While we still haven’t called it GA yet, many of us run it in production already. Check out our Helm hello-world tutorial to see how it all works.
Flux project stats
Flux has come a long way since only 1.1. 86% of its files were touched. The code size has grown by about 12.5%. While this might sound like somewhat dry statistics, it gives you an idea that almost every part of Flux was under scrutiny to bring you new features, support for more scenarios, ease of use, faster and more reliable operations and more security.
Flux has also become more popular. More than a thousand people have starred it on Github.
We have increased our Flux Capacity
(Credit:https://media.giphy.com/media/3o7aD46cTjbkp0gT5e/source.gif)
The community around Flux has grown in this last year as well. If you hang out in the #flux Slack channel you will quickly note that there’s a variety of people helping out and discussing changes and answering general questions.
Flux Community
Solutions around Flux have continued to increase. Many integrations and example deployments have appeared and folks who are using Flux have now formed a close-knit circle of contributors.
Michael Bridgen, a Flux maintainer since its inception, invited others to become maintainers as well. A big thanks to Hidde Beydals, Justin Barrick and Nick Cabatoff for stepping up! This will put Flux’s leadership on broader feet and help us move faster while making sure to support additional and more scenarios.
I am in my early twenties, father of a 2 year old toddler (yeah, _that_ early), software engineer (professionally PHP, but I would consider myself a self-taught polyglot) turned in to operator by chance. The company I work for has been running Kubernetes in production for about 2 years for a side project but we are in the progress of moving literally everything to K8S.
I am basically in charge of the whole operation described above, so my day to day work consists of rewriting the infrastructure of applications to Docker, manifests Kubernetes will understand, rewriting pipelines so they ship images and most importantly teaching developers what and why I changed stuff so I won't have to do it again in the future.
Started contributing to Flux after I started using it - which is a routine thing for me to do as I like to keep relationships as tight as possible for OS-tooling we use to speed things up when issues with an OS product surface (either because we have domain knowledge of the tool itself or because I can bug the right person).
What made you commit to become maintainer? -- World domination. I have a limited amount of free time and I like to use this time as wisely as possible. Flux matches some of the learning goals I have set for myself (Golang ✔, Kubernetes API ✔, Kubernetes operators ✔), has lots of room for improvements and instantly delivers back as a product to the company I work for.
How you use it -- we use Flux in a multi-env multi-branch setup to manage and maintain the manifests and Helm charts of several (micro-)services and fully fledged websites (although the latter is slowly being replaced by the former) of various products.
What you like about it or its community -- since the day I dropped in on Slack it has only give me a warm and welcome feelings. Daniel (you) is a strong advocate for being an open community - especially to newcomers to OS - and with #hacktoberfest almost behind us I think we do well.
-- Hidde Beydals, on himself and getting involved in Flux.
Thanks to all of the Flux Contributors
In general we’ve seen quite an influx of new contributors since 1.1.0: our Slack channel is buzzing and 52 folks authored patches, 113 generally contributed on Github since then. We saw many drive-by contributions which clarified docs or solved edge-cases people were running into. We are very happy that quite a few of you came back again and again to help the Flux project move forward. Here are a few examples of the excellent work:
- Thomas Kooi who made sure that the Helm operator can talk to Tiller using TLS
- Stephen Moloney verified known_host ssh keys, extended the docs and worked on other ssh config related bits.
- Among other things, Justin Barrick made sure that fluxctl automatically port-forwards to flux instances.
- Peter Vandenabeele fixed our docs
- Elena Morozova cleaned up the code in quite a few places and among other fixes made Flux provide more information when listing services.
- Stefan Prodan contributed the Helm charts for Flux and the Helm operator. Since then he worked tirelessly on making the Helm experience even better and paved the way for many of the integrations listed below.
- Roland Schilter added semver filtering, added the interactive release mode for fluxctl and worked on many other parts of Flux.
- Nick Cabatoff added quite a few tests to the code and improved logging of release diffs.
- Hidde Beydals added regex filtering for images, added a git timeout flag and improved the docs in many places.
One source of fixes was obviously Hacktoberfest - we, as the Flux community, had a lot of fun participating there! October alone saw 36 contributors in Github issues, 20 committed to Flux’s code - 14 of whom contributed to Flux for the very first time. ♥
#GitOps in the Community
Even if the whole point of this blog post was to get you up to speed on what happened in Flux, we are also very excited about how GitOps as a concept is growing and spreading. To have more general discussions about the subject, we created the #gitops channel on Kubernetes Slack. It’s a good place to learn about integrations, get in touch with the wider community and have more general discussions.
Integrations!
Many have worked on taking Flux out there and Flux does the bulk of the work in many scenarios already.
A big one is obviously Weave Cloud. If you haven’t seen it yet, check out the video and try it out for free. It’s GitOps workflows doesn’t just do your deployments, but also builds in observability. Behind the scenes it uses Flux but provides a more integrated experience with a beautiful and easy to use UI. Recently we added Promotions to Weave Cloud, allowing you to simply promote workloads from dev to prod.
There’s loads more! Because Flux is open-source, it makes integrations super-easy. Here are just a few of our favourites:
- Managing Helm releases the GitOps way
- OpenFaaS GitOps workflow with Flux
- GitOps for Istio Canary deployments
- Fluxcloud to receive events from Flux
Future!
We had a couple of discussions about where we want to take Flux. Making the Helm story rock-solid and call it “1.0” is a big one. We are looking at UX as well to make it super-easy to use.
But you can have your say and get involved too!
📞We’re having a meeting!
We are super happy with where Flux is going and we want to make sure we hear your feedback, gather your thoughts and help you get involved if you want to. After talking things through, we decided to have a meeting with everyone who’s interested. It’ll be a video call, where we can get to know each other, do introductions, see what our particular interest in Flux is and where we’d like the project to go. So if you have been participating, lurking, evaluating, or you’re just curious, join us and we’ll make plans together.
When: 2018-11-28 17:00 UTC
Where: Zoom call
We want YOU to be there, join us and define where Flux is going next!
