The History of GitOps
The history of GitOps closely follows that of the container and Kubernetes revolution of the past few years. In this post, we look at all the key milestones in the journey of GitOps as it went from a fledgling idea to the global technology movement it has become today.
2014 - The pre-GitOps era
About a decade ago, cloud vendors, more specifically PaaS vendors like Heroku, Cloud Foundry, and OpenShift were trying to offer application development capability that would run anywhere in the cloud with great ease of use. However, none of these solutions really provided much more than the 12-factor application model. While this worked for some types of applications, it faced challenges with setup, scaling, and load balancing for heavily data-oriented enterprise applications.
Docker: With the launch of Docker containers we now had a very simple operating model for a single node. A container is a package that is also a runtime, and is portable. Containers are ephemeral – something that you can start and stop quickly - this follows the cattle vs pets concept. They are developer-friendly, not requiring developers to understand infrastructure. All this led to containers being adopted by pretty much the entire industry, something that PaaS never enjoyed.
2015 - The container orchestration rush
Weaveworks, the company, started out initially with a goal of making it easy to build applications like RabbitMQ as a service, or Redis as a service. To fill in the missing pieces, they had to build a networking product called Weave Net.
Seeing the need for developers to manage and monitor containers in the cloud, the team at Weaveworks built a tool to do just this and called it Weave Cloud. Weave Cloud minimizes the complexity of operating Kubernetes clusters with automated continuous delivery pipelines, observability and Prometheus monitoring.
Weave Cloud itself was run on containers, which needed a container orchestrator to be managed. The team set out to build an orchestrator, but soon realized it's not the best use of time and resources. They instead opted to leverage one of the available container orchestration tools - Mesos, Docker Swarm, Nomad from Hashicorp, and Kubernetes. The first three had issues such as being buggy, complex to manage, or falling somewhere between open source and commercial. For these reasons, the team went with Kubernetes as their container orchestrator of choice.
Kubernetes was not without its issues though. It was relatively well-thought-out, but it was an architectural mess, and was far too complicated. Still, this was the best choice back in 2015, and the team had to become experts at installing, deploying and running Kubernetes. The team built their SaaS offering - Weave Cloud - operating Kubernetes on EC2.
2016 - Betting the farm
Soon, the team at Weaveworks was able to streamline and improve their Kubernetes operations with a couple of different staging areas, multiple zones for reliability, and monitoring and security baked in. The team had become bolder to deploy risky changes into production.
One day, one of the engineers declared that he was about to push a change to the system, which if it didn’t work as intended, could wipe out the entire system. It didn’t work and the system was wiped out, but the Weaveworks team was able to bring the system back up in about forty minutes because it was fully described in various Git config files. The system included not just the cluster, but also the app, monitoring, and other pieces. Whenever a change is made to the system it is first committed and then allowed to propagate automatically through into production.
The Weaveworks team was practicing something like a version 0.1 of GitOps before the term even existed. They had simple rules such as not allowing a change to happen, if it was a permanent change. Every change had to be committed to the system of record for the desired state which was stored in a resilient, distributed, authenticated, non-repudiable, secure GitHub instance.
Many of these practices were introduced to the team at Weaveworks by employees who brought with them these learnings of how to build resilient systems using declarative principles.
2017 - The birth of GitOps
After the incident, pleased with their progress, the team sat together and made a list of these principles by which they operated their Kubernetes system. They distilled down the principles into the few most important ones. It was during one of these discussions that Alexis Richardson, CEO of Weaveworks, realized that one word that describes all of this was 'GitOps.' The fundamental idea behind GitOps was to make operations automatic for the whole system based on a model of the system which was living outside the system - and Git was where they had chosen to put that model.
One afternoon while discussing this pattern of managing the entire system using Git, Alexis came up with the word 'GitOps.' Richardson ran the phrase by a friend, James Governor, Founder of RedMonk, who said it was possibly the ugliest word he'd heard in a long time, and now he couldn't un-hear it. Wanting a term that was easy to pronounce and would stick in someone's memory once they heard it, Richardson knew he found what he was looking for.
Alexis first describes GitOps
Richardson soon presented the four principles of GitOps in talks. These are the four principles:
1. The entire system is described declaratively
2. The canonical desired system state is versioned in Git
3. Approved changes that can be automatically applied to the system
4. Software agents to ensure correctness and alert on divergence
You can read more about these principles in the Guide to GitOps.
2018 - Industry-wide adoption
It wasn't long before many in the cloud-native ecosystem started to take notice of the uniqueness and simplicity of the GitOps approach. Kelsey Hightower tweeted "GitOps: versioned CI/CD on top of declarative infrastructure. Stop scripting and start shipping."
GitOps soon saw adoption by the major cloud providers AWS, Azure and Google Cloud.
As interest around GitOps grew, and new tools were being introduced within the cloud-native ecosystem, there were still questions around GitOps. Alexis published a follow-up blog, “What is GitOps Really?” which aimed to provide the answers. Weaveworks also published a Guide to GitOps and GitOps FAQ to further assist teams in understanding the GitOps methodology.
2019 - Progressive delivery with Flux & Flagger
As GitOps' benefits were becoming more evident, Kelsey Hightower tweeted "GitOps is the best thing since configuration as code. Git changed how we collaborate, but declarative configuration is the key to dealing with infrastructure at scale, and sets the stage for the next generation of management tools."
Weaveworks then released Flux and Flagger, both open source tools that enable progressive delivery using the GitOps model. These tools made GitOps more operational for organizations. Both Flux and Flagger became CNCF sandbox projects. This was the beginning of the GitOps movement growing from a small group of early adopters to a community of collaborators from every corner of the cloud-native ecosystem.
2020 - Building an ecosystem around GitOps
The year 2020, though marked by the pandemic, saw GitOps events bloom virtually. Weaveworks along with other industry experts launched the first GitOps Days virtual event, soon followed by GitOps Days Europe.
In their first-ever technology radar, CNCF recommended to adopt Flux stating "We can clearly recommend this technology. We have used it for long periods of time in many teams, and it has proven to be stable and useful."
Flagger became a CNCF project, part of the Flux family of GitOps tools. Furthermore, the CNCF began to be more involved with the development of Flux and more overt of their support for GitOps. Under the CNCF oversight, the GitOps Working Group was founded with support from many companies such as Amazon, Red Hat, Codefresh, GitHub, Microsoft, and Weaveworks.
Weaveworks has been innovating to make GitOps easier to adopt by companies of all sizes and in all industries. To this end, Weaveworks launched the Weave Kubernetes Platform (WKP) which is based on GitOps, and enables organizations to manage their Kubernetes clusters from on-prem to cloud to edge with a single management interface.
2021 & beyond - The GitOps journey continues
Weaveworks started 2021 on a high by receiving funding from leading cloud and telco companies, who see great potential in the concept of GitOps. The growth of Flux and Flagger continues at CNCF as they were recently promoted to incubating status.
Continuing the momentum, Weaveworks recently launched a tiered Weave GitOps product that helps organizations adopt GitOps to standardize on codified best practices, accelerating continuous application delivery and continuous operations in multi-cloud environments.
Conclusion
Today, GitOps is a household name in software delivery teams all over the world, at companies large and small. From looking at its history, it's easy to see why. GitOps has grown organically from the grassroots level and made its way to the largest enterprises of the world because it simply works. It simplifies the complex operations of Kubernetes and containers, and gives organizations the confidence they long for when deploying code from a Git repository to a production Kubernetes cluster. Customers enjoying the benefits of GitOps include DataScan, Axel Springer, National Australia Bank among others.
Find out more about Weave GitOps or request a demo.
The History of GitOps Infographic
Download noweBook: A Beginners Guide to GitOps
Take the first steps in getting started with GitOps today!
