You aren't Doing GitOps without Drift Detection

May 10, 2022

GitOps unifies ​​Kubernetes YAML files and application code in Git repositories to prevent configuration drift. Our latest blog describes how this works in detail.

Related posts

How GitOps Eases Multicloud Migration

Continuous AWS Cloud Security with Trusted Delivery

​6 Key Ideas Shaping GitOps Today

‘Drift’ is a given in a dynamic software system where code gets outdated the minute it is released. This is true of application code, and in systems where infrastructure is managed as code as well. Drift is a problem for builders and maintainers of these systems as it causes incompatibility issues, brings unpredictability, and makes systems unreliable. Reducing configuration drift is a top priority for system architects and SREs alike. In this post, we look at why drift detection is key to modern operations, and how GitOps enables it using a new breed of tooling and approaches.

What is Configuration Drift?

‘Configuration drift’ is a common term to describe this change that takes place in production environments. It is a gradual, unnoticed change from the desired state of the system that was originally deployed. Drift is an issue because it causes systems and parts of a system that are supposed to be consistent, to become inconsistent and unpredictable.

Configuration drift happens when manual changes are allowed in production environments. These manual changes gradually change the state of the system to something completely different from its original state.

Infrastructure as code (IaC) tooling like Chef and Puppet were the first real effort at stopping drift in production about a decade ago. They were focused on defining infrastructure as code and discouraging manual changes to be made by administrators. These solutions were a great improvement from previous infrastructure tools, but their drawback was that they were predominantly server-centric and were disconnected from the application code, data layer, and networking layer.

As cloud-native practices have begun to unify all parts of the technology stack, drift should not be prevented at every level - application, networking, data, and infrastructure. The older generation of DevOps tooling is unable to deliver such expansive coverage. This is where a modern approach like GitOps is uniquely positioned to be the answer to fighting drift in cloud-native systems that are powered by ​​Kubernetes.

Kubernetes Makes Infrastructure Declarative

One of the key tenets of Kubernetes is that it takes a declarative approach to infrastructure. That means you don't have to tell Kubernetes how to deploy your application - you simply tell it how you want your infrastructure to look like and Kubernetes takes care of the rest. Kubernetes objects are created, updated, and deleted based on their config as defined in YAML files.

Git Makes the Application Stack Declarative

Git, with its repository-based system, has existed since before ​​​​Kubernetes and has enabled developers to define the state of their applications using repositories. This allows complex applications to work as intended in an ever-changing environment.

GitOps Calls to Declare Everything as Code

Git is most commonly thought of as a mechanism to control the development of software, but it also has a powerful impact on the rest of the application stack. With GitOps, a modern evolution of the DevOps model, the key focus is on declaring every part of the system as code. This includes the application, data, networking, and infrastructure. Once this is done, the powerful version control features of Git can be leveraged to track and even prevent changes from happening anywhere in the system.

It is this aspect of GitOps that makes it ideal to prevent drift. Git is able to track changes in the content of files and version them for greater visibility and control. Further, GitOps also enables you to prevent any change from a declared state of the system.

Git is able to bring together application code and dependencies stored in repositories, alongside YAML files that define the state of production Kubernetes clusters. In this way, GitOps unifies operations across application development and platform engineering teams.

For application developers, this delivers a self-service approach where they can gain immediate access to the resources they need to take their code to production. For platform engineers, creating ready-made deployable cloud resources reduces their manual workload, and does not compromise on security as every deployable package is pre-approved and production-ready.

How GitOps Prevents Drift

GitOps leverages an agent like Flux that is connected to all Git repositories in a system and owns the deployment of changes from these systems into production. Along with that, Flux also integrates with Helm to manage the YAML files that define the state of the production Kubernetes system.

Flux has built-in drift detection capabilities that make it a powerful management layer for applications and infrastructure. Flux monitors the state of production Kubernetes clusters and reconciles them to their original declared state. Whenever Flux notices a deviation from the original state, it fires off an alert. Flux can also be set to automatically rollback changes so that the production system stays faithful to the declared state.

Trusted Delivery Builds on Drift Detection

Trusted delivery is the use of policies and guardrails to be alerted when any new change would cause a system to deviate from its declared state. Magalix (now part of Weaveworks) is a great example of a solution that enables trusted delivery, by detecting potential problems with security, resilience and coding standards in the configuration before it gets automatically deployed to your clusters. Magalix ensures your systems stay compliant.

As the technology stack evolves and becomes more complex and automated, preventing drift will become more important. By leveraging Kubernetes along with GitOps, you can unify operations end-to-end and prevent the possibility of drift. A solution like Weave GitOps has built-in defaults to enable trusted delivery and advanced features like rollback of non-compliant changes. All this combines to bring granular control and advanced capabilities. GitOps makes systems reliable and predictable and is the way forward to prevent drift in cloud-native systems.


Try it out for yourself and download our free and open source Weave GitOps Core. The getting started guide will walk you through a few short steps and you will be up and running.

Download Core

Related posts

How GitOps Eases Multicloud Migration

Continuous AWS Cloud Security with Trusted Delivery

​6 Key Ideas Shaping GitOps Today

Whitepaper: Trusted Delivery with GitOps and Policy as Code

Learn more about what is Trusted Delivery and how you can get started!

Download Whitepaper