GitOps is about to get better with Flux v2

By bltc34b82b83b949c23

August 13, 2020

GitOps

Flux

We are proud to announce GitOps with Flux v2 and GitOps Toolkit. Flux v2 addresses many user requests such as support for synching multiple git repos and provides a smoother "Day 2"experience. The GitOps Toolkit is a set of core GitOps operators that work in concert, and Flux v2 is an off-the-shelf composition of those components that gives you equivalent capabilities to the original Flux.

Meet the Weaveworks Team at GitOpsCon Europe (Virtual Event)

KubeCon Chicago 2023 Recap: Cloud-native Scale & Growth with GitOps

Weave AI Controllers: GitOps Automation for Large Language Models (LLMs)

We’re paving a smoother path to GitOps with Flux v2 and the GitOps Toolkit.

The original Flux kicked off “GitOps”: an app dev and operations methodology that emphasizes

Describing your system declaratively;
Keeping configuration under source control; and
Using software agents to reconcile and ensure correctness (along with alerts, etc).

Flux version 2 is the successor to Flux, and will be more in tune with Kubernetes’ best practices for operations and observability, as well as more capable. The foundation for Flux v2 is the GitOps Toolkit, a set of components for building GitOps-based delivery and automation.

The GitOps Toolkit is available now as a pre-release. We’ve posted a roadmap to Flux v2 so you can see where it’s up to.

Flux v2 - How did we get here?

The premise of the original Flux is ensuring that the definitions in a Kubernetes cluster match a configuration given in git. Flux v2 delivers that premise, but with an up-to-date understanding of how Kubernetes applications are built and operated, meaning it is more amenable to today’s operating procedures and tooling.

For existing Flux users, Flux v2 addresses features that many of you have been requesting (such as support for syncing multiple git repositories) as well as a more agreeable “Day 2” experience.

The new architecture, with more accessible moving parts, makes contributing to and extending Flux v2 and the GitOps Toolkit a lot easier.

What is this GitOps Toolkit and where did it come from?

When we started work on Flux, our main goal was to automate deploying newly-built container images into our Kubernetes clusters (that powered our SaaS Weave Cloud). It ran a rolling update of each ReplicationController (remember those?), then committed the change to git. Someone - I think it was Paul, or Adam - suggested that we drive everything by committing to git, then applying what’s in git. That’s when the idea of “GitOps” arose. In hindsight, it’s an obvious mechanism, but it took a generous measure of courage to run it in our cluster the first time.

It turned out we had wandered into a pretty compelling idea. Flux made the central equation of Kubernetes -

cluster reality ≍ declared state

- amenable to day-to-day engineering practice, and it clicked.

The Helm Operator wasn’t far behind. By then, Kubernetes’ API extensions (CRDs) had gone mainstream, and operators were all the rage. There were some parts of the Kubernetes libraries that could be used to write your own controllers, but they were esoteric enough to support a cottage industry in how-to blog posts.

By the time newer and more accessible tooling arrived in the form of controller-runtime and kubebuilder, Flux was starting to sag under its own weight.

While it would have been possible to retool incrementally, a fresh start with a major version bump gave us the room both to realize some long-held ambitions, and to deprecate some features that weren’t paying their way.

As an example of the former: Flux (v1) runs git-mirroring, git-syncing, and image-updating loops, which are logically disjointed but closely entwined in the implementation. It’s been clear for a while that these deserve to be disentwined. In the GitOps Toolkit, these are distinct controllers that can run independently, but work in sympathy; e.g., the kustomize controller watches the git source controller, and reacts when there’s a new revision available.

Along with better factoring, building the new controllers around Kubernetes API extensions means that the status of a particular resource - say, a git repository being mirrored - is transparently there to be examined. With Flux v1, the state was largely internal, and only glimpsed through logs and events. With GitOps Toolkit controllers, the workings are open. That means it’s easier to operate and build systems upon them.

Are there breaking changes for Flux v1 users?

There are many differences that stem from using custom resources, which are improvements but also breaking changes. There’s a list of these (and other differences) in our FAQ.

A significant incompatibility is that .flux.yaml files will not be supported. These were introduced to support generating configuration by arbitrary commands; and especially to make it possible to use kustomize with Flux for people who wanted it. The intention was good, and the implementation solid; but in practice, having the escape hatch to arbitrary commands made everything too hard to troubleshoot, and it compromised the simpler goal of supporting kustomize.

We understand that our dedicated community of Flux v1 users will have concerns about these breaking changes. Once Flux v2 reaches full parity with Flux v1, we’ll begin a six-month window for people to move their systems over to Flux v2. We’ll be supporting that effort with migration guides and recommendations.