GitOps Everything: What it Means in Practice
GitOps is a modern approach to managing ‘everything’ as code - from infrastructure to networking & more - using a policy-based approach. Read about how it can be used to define and automate entire IT workflows.
While DevOps played a pivotal role in shaping how we deliver technology, there’s a growing need for greater release velocity - this was clear in the recent DORA report. With the rapid pace of innovation, organizations are expected to transform their product development to meet the market's demands. We have help, of course. Modern technologies allow us to move at a pace faster than ever before but the problem we’re facing is the disconnect between the pace at which the application can be developed and what an Infrastructure and Operations team can deliver. While we’d all like to go faster, the processes, tools, and techniques followed and used by most organizations aren’t adequate to sustain the pace.
GitOps paves the way to automate Kubernetes cluster management and application delivery. It uses Git as a single source of truth for declarative applications and infrastructure to build on DevOps. GitOps uses Git and takes it a step further to facilitate operations in terms of infrastructure provisioning. But today, the applications of Git have gone far beyond simple developer collaboration, and now include infrastructure, networking, and security management as well. Moreover, GitOps is being used to define and manage entire IT infrastructures declaratively to the point that the idea of “GitOps Everywhere” is very real for modern cloud-native systems.
GitOps Everything
The goal of all IT strategies eventually boils down to automating processes, helping organizations become more effective, thus allowing them to deliver new features to their customers faster. But there are a few main reasons why most organizations still struggle to achieve this goal:
- Outdated development practices and legacy systems hamper the pace of software development and reduce release frequency. These outdated practices include manual processes, unclear handoffs across the pipeline, and hidden dependencies between components that make up the supply chain. What’s needed is clarity between teams, and transparency into the exact components that are used at every step of the supply chain. To get there, it takes a radical effort to define everything consistently, using something like Git. This is what GitOps brings to the table.
- The increased cost of application maintenance as older applications are seldom retired and maintenance costs are often overlooked if the development costs are cheaper.
- Hefty regulatory costs given the high risk of non-compliance and manual compliance checks aren’t scalable and automating these compliance checks can be viewed as prohibitively expensive.
<p>Ideally, all of us would want our organizational processes to be automated and well-integrated without the need for human intervention. However, it can be challenging for organizations to prioritize long-term sustainability in the face of pressing day-to-day issues, and aggressive business goals. </p>
The day-to-day operations of developers in most organizations are highly fragmented. For a developer to implement a feature, they have to go through a CI system, a test system, a service for the deployment, and again a service for monitoring the application in production. It’s clear that development teams often lack clarity about what’s going on in their services. By following GitOps principles, we can describe the entire IT infrastructure of an organization declaratively and automate operations in a manner that minimizes the need for human intervention.
Managing Infrastructure as Code (IaC) With GitOps
Infrastructure that’s managed manually is prone to configuration drift. This challenge was initially tackled by earlier IaC tools such as Chef and Puppet. However, this problem has only worsened with the advent of Kubernetes. With the thousands of containers in production today, configuration drift is a given in most enterprise systems. However, this need not be the case.
GitOps, with the help of agents like Flux, helps in drift detection especially when there are hundreds of services to monitor, allowing developers to look for changes inside their code and detect drift from their desired state. For instance, if an EC2 instance on AWS gets accidentally deleted from the desired state, the development team gets alerted of its absence. It includes a set of controllers for keeping Kubernetes clusters in sync with configuration sources and uses containers for immutability and various cloud-native tools like Ansible and Terraform for automating and managing the configurations. It looks at all the repositories in Gitlab, for instance, and if any changes are detected, it clones the artifacts from the repositories and saves them in the declared state.
Enforcing Network Policies
Networking used to be simpler in the days of client-server communication. Today, networking is many-to-many across both internal and external applications. Service meshes have risen to tame the complexity around networking. However, it takes a conscious decision to apply declarative principles across the system for declarative networking to actually work. That’s what GitOps enables - declarative everything - including networking.
GitOps integrates with service meshes to enforce network policies and is designed to give developers confidence in automating production releases. It can integrate multiple service meshes like Istio, Linkerd, and Kuma to split the traffic to flow into different versions of an application. Taking advantage of this powerful networking model, GitOps goes further to enable progressive delivery - implementing canary releases to reduce the impact of bad code in production. GitOps enables this using Flagger - a tool purpose-built for managing progressive delivery.
GitOps also delivers critical monitoring metrics such as HTTP/gRPC success rates, latency, etc, and can show live updates on the traffic. It can even be integrated with messaging tools like Slack to receive real-time notifications on the status of the product release.
Enforcing Security Policies
Managing security and compliance has mostly been a manual process. Security and compliance checks are tacked on to a finished product, and changes are hastily made to meet minimum requirements before release. This is an anti-pattern that inevitably leads to firefighting in production. What’s required is a proactive approach to security and compliance, and one that is automated with the help of tools rather than rely on human review.
GitOps makes this possible with the introduction of policy-as-code through which we can now manage policies and automate pipeline operations to safeguard infrastructure. Trusted application delivery helps fully integrate policy-as-code into the GitOps pipeline and makes the application resilient from source to production.
With GitOps, a user can now define anything they want as a service, but they need to be mindful of who can access the service and who has permission. To that effect, users can enforce policies in microservices, Kubernetes, CI/CD pipelines, API gateways, and more. For instance, deleting or updating resources can be considered risky operations, so a policy can be created to flag such resources for human review. Again, policies can be written to limit the size of PRs to reduce the chances of missing parts of code during reviews.
Weave GitOps
Weave GitOps is a revolutionary solution that takes the key ideas around GitOps and delivers it in a single, unified product. Weave GitOps builds on the capabilities of Flux, Flagger, Helm, and other Kubernetes ecosystem tools and makes them more usable at the enterprise level. Taking a policy-based approach to software delivery, Weave GitOps enables you to apply policies that are defined by Ops teams, are automatically followed by development teams, and are enforced in production on a continuous basis. If you’re looking for a way to unify how you manage your cloud-native system end-to-end, Weave GitOps is an elegant and comprehensive solution that manages everything from code to infrastructure to networking, and everything in between. It applies the security and compliance policies you define at every step of the pipeline, and makes an otherwise complex process much simpler with the power of Git.
Conclusion
For organizations using Kubernetes to run their applications, Weave GitOps is a convenient, production-ready solution. Adopting GitOps practices at every layer allows us to take the next step toward automating various IT strategy requirements.
Cloud-native toolchains have allowed for fast product delivery to customers, and it’s important for traditional IT organizations to adopt these toolchains to modernize their delivery methods. GitOps is steadily becoming the centerpiece of every modern IT strategy, automating it, and making it more scalable. Such a strategy uses standard GitOps tools to navigate from DevOps loopholes toward more hands-free yet controlled operations where both Dev and Ops use the same tooling and automation processes.
Whitepaper: The GitOps Guide to Building & Managing Internal Platforms
We look at how GitOps is the solution for building internal platforms at scale, enabling software delivery teams to release software continuously, and reliably.
