Components of a GitOps Software Delivery Pipeline - an Infographic
We outline the components needed to get started with GitOps in a handy infographic, and offer some suggestions as to what you can add as you start to scale up.
What you need to do GitOps
Download a PDF copy of this infographic
GitOps: Starting out and scaling up
With so much information out there about GitOps, it can be a little confusing for newcomers trying to work out exactly what constitutes a GitOps pipeline – and, crucially, what doesn’t. In part, this is due to the fact that GitOps was designed to run on a familiar toolset, and one of the tools in that set is Kubernetes – a platform that comes with a giant ecosystem of its own. That giant ecosystem and the vast number of tools in it is one reason why it can be hard to see the wood from the trees, to work out what, at a basic level, you need to do GitOps.
So in the interest of simplicity and clarity, let’s look at what the main components of a GitOps pipeline are – both the essentials without which you’re simply not doing GitOps, and then the types of tools that are most frequently deployed alongside them, when you’re ready to get serious.
Starting out
In essence, there are five components you really need to do GitOps: container platform, Git repository, container image registry, GitOps agent, and build server.
So let’s take a quick look at each one.
Container platform
A declarative system for managing the containers in which your app will run. An example of this is Kubernetes. But when we say ‘example,’ we mean it’s really the only choice. Kubernetes is now ubiquitous, and its ecosystem, as we’ll see, is enormous.
Container image registry
A system for building, organizing, and deploying the images of the containers in which your app will run. You need it to be compatible with your choice of Git hosting service (we’ll come to that next). An example here would be Quay.io.
Git hosting service
Git may be available to anyone who wants to download it and run it on their servers, but in reality, you’re almost certainly going to make use of a Git hosting service. You’ll need this for your application code, of course. But because this is GitOps we’re talking about, you’ll also need it for all the configuration information that goes with it. Probably the most well-known examples of a Git hosting service are GitHub and GitLab, but there are many more.
Reconciliation (GitOps) agent
This is the software that makes sure the application running on Kubernetes matches precisely what’s described in your Git repositories (also known in the GitOps world as your “single source of truth”). The reconciliation agent forms a bridge between Git and Kubernetes. It watches what’s going on in production and continuously compares what it sees to the description of what should be happening that’s stored in Git. As long as there are no discrepancies, then everything’s fine. If anything in production diverges from the description in Git, however, the reconciliation can trigger various reparatory sequences. Depending on how your pipeline is configured, these may include some degree of automated fixing, or a series of alerts fired off in the direction of your platform team. Either way, it enables equilibrium to be restored.
These reconciliation agents are few and far between, due to the relative youth of GitOps as an operating model. The most well-established is the CNCF’s Flux, open source software originally created by Weaveworks that sits at the heart of Weave GitOps.
Build server
The bigger your operation gets, the more developers will be contributing to it - often submitting updates (called commits) on a daily basis or faster. That’s a lot of updates to make to a single codebase, with the obvious potential for error. To manage this continuous integration of code updates (often shortened to just CI), it helps if you can automate some aspects – and that’s where your build server comes in. This is a piece of software that automates the build of every new update to be deployed, shunting it further along the pipeline towards production. Popular examples include Jenkins X and CircleCI.
Scaling up
One way of looking at GitOps is to see it as a way of automating Kubernetes. That’s a significant understatement, but it’s not exactly wrong. In fact, it’s not at all wrong. The problem this gives us when listing the next level of components is that practically any piece of software in the Kubernetes ecosystem could warrant a place on that list. As we’ve noted above, it is a very large ecosystem.
Nonetheless, it’s worth considering a few types of components that can be found in many GitOps-controlled software delivery pipelines, because they complement GitOps very well. So let’s start our selective list on that basis.
Container configuration automation
The ability to automate the configuration of containers in your Kubernetes cluster is a very desirable feature, especially for large, complex applications that see the frequent deployment of updates and new features. And this is not just for labor-saving purposes. By automating this configuration, you can ensure that all deployments follow pre-set rules, avoid the errors that can arise when all configuration takes place manually, and even enable your developers to deploy their own updates to production.
In the Kubernetes world, the most popular solution for achieving this is a templating system called Helm. Helm enables you to create automation templates known as charts, which can be used as part of your pipeline to introduce rules and standardization to the deployment process.
Policy engine
Helm will help you impose certain rules on the way updates are deployed, by providing templates in the form of Helm charts. But Helm can only do so much – and its use is optional. To ensure that all updates comply with configuration rules that will improve the security and compliance of everything that passes through the pipeline, you can implement policy as code, built on the functionality of Open Policy Agent (OPA). By codifying policies and enforcing them automatically, it means you can rely on your security, coding, and compliance standards as you scale.
At this level, you’re venturing beyond the confines of the open-source Kubernetes ecosystem, and into the world of commercial GitOps software. Weave GitOps Enterprise, the version of Weave GitOps designed for deployment at scale, includes a sophisticated policy engine you can use to set rules for all deployments, making Trusted Application Delivery a reality.
Service mesh
Cloud-native applications are made up of services – and the bigger your application becomes, the more services will be involved. A service mesh, for example the open source project Istio, is a layer of your application’s infrastructure that dictates how each of the individual services that make up your app will interact with one another, keeping them all running without any one service becoming overloaded. It works by creating a proxy for every microservice in the app. These proxies are sometimes called ‘sidecars.’ But each one lives in the service mesh layer, rather than in the corresponding microservice itself. The service mesh can reroute requests if a particular service fails, for example, and also provide valuable data about the way the application is performing ‘on the inside’.
Progressive delivery manager
To minimize the disruption that can be caused to your user base by failed software updates, many applications now use progressive delivery. In essence, this involves making a new update available to a small subset of users (e.g. 10% of the whole user base), checking the update has been successful, then continuing to roll out the deployment. To manage and automate this process, you can use progressive delivery management software such as Flagger.
Event monitoring system
In any distributed software application, there’s one thing you can be sure of: from time to time, things will go wrong. To give your platform team the earliest possible indication that things might not be working as they should, you need an event monitoring system, like Prometheus or Grafana. By constantly watching for events you tell it to look out for, it effectively shows you what’s happening inside your pipeline, with any problems triggering alerts that are sent to the teams responsible.
Browser-based interface
In GitOps, everything described above can be configured and managed the old-fashioned way, by using a command line interface. But there are many cases when even the most experienced team members will find it easier to visualize issues when viewing things in a graphical interface. Commercial packages such as Weave GitOps Enterprise provide browser-based graphical interfaces that can make management at scale significantly easier, while reducing the likelihood of human error when managing the platform.
Weave GitOps is a full-stack GitOps platform brought to you by the creators of GitOps. You can automate continuous application delivery and Kubernetes operations at scale. Try the forever free open source version today if you’re starting your GitOps journey or talk to one of our experts to learn more about Weave GitOps Enterprise.
Infographic: Components of a GitOps Software Delivery Pipeline
Understand the fundamentals required for getting started with GitOps. Download our latest infographic.
