GitOps for Istio - Manage Istio Config like Code
Read how GitOps workflows can be implemented to rollout and manage non-atomic canary releases to an Istio service mesh. Stefan Prodan then takes you through the steps of setting up a canary with GitOps workflows to Istio.
At this year’s Kubecon conference held in Copenhagen, Alexis Richardson, CEO of Weaveworks, and Varun Talwar of a stealth startup spoke about GitOps workflows and Istio. The talks were followed up with a demo and tutorial by Weaveworks’ Stefan Prodan on how to rollout and manage canary deployments to Istio using GitOps principles.
The talks and the demo explain:
- What GitOps is and why you need it?
- How Istio and GitOps best practices can manage applications running on it.
- How to do a Canary deployment on Istio using GitOps workflows.
What is GitOps?
GitOps is a way to do Continuous Delivery. “It works by using Git as a source of truth for declarative infrastructure and applications,” says Alexis Richardson.
Automated delivery pipelines roll out changes to your infrastructure when changes are made to Git. But the idea goes further than that – it uses tools to compare the actual production state with what’s under source control and then tells you when your cluster doesn’t match the real world.
Git enables declarative tools
By using declarative tools, the entire set of configuration files can be version controlled in Git. By using Git as the single source of truth, your entire infrastructure can be reproduced, reducing your mean time to recovery from hours to minutes.
GitOps empowers developers to embrace operations
The GitOps core machinery in Weave Cloud is in the CI/CD tooling and the critical piece is continuous deployment (CD) and release management that supports Git-cluster synchronization. Weave Cloud deploy is specifically designed for version controlled systems and declarative application stacks. Every developer uses Git and makes pull requests and now they can use Git to accelerate and simplify operational tasks for Kubernetes and other declarative technologies like Istio as well.
Three core principles of GitOps
The core principles described below, according to Alexis, are why GitOps is both Kubernetes and Cloud Native centric:
#1. Declarative configuration is at the core of GitOps
By using Git as the source of truth, and Kubernetes rolling updates, it is possible to observe your cluster and compare it with the desired state. By treating declarative configuration like code, it allows you to enforce convergence by reapplying changes if they didn’t succeed.
#2. Kubectl should not be used directly
As a general rule, it's not a good idea to push directly from CI to production. Many people let their CI tool drive deployment, but when you're doing that you’re potentially giving a notoriously hackable thing access to production.
#3. Use an operator pattern
With the operator pattern, your cluster always stays in sync with what’s been checked into Git. Weave Flux, which is open source and is the basis of the canary deployments Istio demo below, you can manage changes in your cluster with an operator.
With both a developer flow and a production flow, merging from staging to production, the operator pulls changes into the cluster and deploys them atomically even when you have multiple changes.
GitOps Workflows for Istio
Next up Varun Talwar spoke about what Istio is and how GitOps workflows are made for managing apps on it.
Istio is a service mesh that launched about a year ago. It’s a dedicated infrastructure layer for all your service to service interactions in a microservices architecture. All actions in Istio are driven through declarative configuration files. This means that service meshes like Istio enable developers to manage service behavior completely through files that are kept in Git with your code.
With Git workflows developers can model anything in Istio including service behavior and their interactions like timeouts, circuit breakers, traffic routing, load balancing, as well as A/B testing, and canary releases.
Many sets of configuration across teams
Istio has four broad areas that are all declaratively configuration driven:
1. Traffic management - everything to do with managing your ingress as well as service to service traffic.
2. Observability - monitoring, latency QPS error rates of all of the traffic.
3. Security - authentication all of the service to service calls as well as authorization.
4. Performance - including failure scenarios retries timeouts, fault injection, and circuit braking.
Because all of these areas can span different teams within your organization, it makes managing applications on Istio particularly challenging.
Many of these configuration driven settings are cross-team. One team for example, may decide to use Zipkin for tracing, but another may want to use Jaeger instead. These decisions can be made for one service or they may be made across services. When decisions span teams, the approval workflows become more complex and are not always atomic. Canary releases for example are not a one-time atomic thing.
Canary deployments to Istio with GitOps workflows
Stefan Prodan showed us how to do a canary release to Istio using GitOps workflows with Weave Flux and Prometheus -- all of which you can use in Weave Cloud as well for canary deployments and observability.
Briefly, canary deployments or releases are used when you want to test some new functionality with a subset of users. Traditionally you may have two almost identical servers: one that goes to all users and another with the new features that gets rolled out to only a set of users.
But by using GitOps workflows, instead of setting up two separate servers, your canary can be controlled through Git. When something goes wrong, the older version can be rolled out, and you can iterate on the canary deployment branch and keep rolling that out until it meets expectations.
Git controlled canary release with full observability in Weave Cloud
A change is pushed through your pipeline where you can send a percentage of your users a select subset of traffic. Using Weave Cloud you can observe in the dashboard whether the canary is working as expected. And if there are issues you can keep making changes, promote the next version, tag it, and deploy it through the same pipeline. This is an iterative process that GitOps workflows can help you manage.
Final Thoughts
Alexis Richardson gave us an overview of GitOps, and why you need to consider this approach when managing applications running on Kubernetes and Istio. Varun Talwar then spoke about what Istio is and how GitOps workflows can be used to manage your apps. Finally Stefan Prodan showed us a particular use case where non-atomic workflows like canary releases can also be managed with GitOps on a service mesh like Istio.
To view the talk in its entirety, watch below.
Need help?
Weaveworks now offers Production Grade Kubernetes Support for enterprises. For the past 3 years, Kubernetes has been powering Weave Cloud, our operations as a service offering, so we couldn’t be more excited to share our knowledge and help teams embrace the benefits of cloud native tooling.
Kubernetes enables working as a high velocity team, which means you can accelerate feature development and reduce operational complexity. But, the gap between theory and practise can be very wide - which is why we've focused on creating GitOps workflows, building from our own experiences of running Kubernetes in production. Our approach uses developer-centric tooling (e.g. git) and a tested approach to help you install, set-up, operate and upgrade Kubernetes. Contact us for more details.
