Multi-cluster Application Deployment Made Easy with GitOpsSets

By Heba Eid

May 23, 2023

Weave GitOps

Product Features

Weave GitOps Enterprise’s new feature, GitOpsSets, groups application resources into sets, thus simplifying the process of applying configuration updates and tracking changes across environments. Read all about it here.

The Problem: Cloud-Native Infrastructure Complexity

With the majority of organizations running numerous clusters hosting multiple applications, managing the configuration of these different environments is a nontrivial challenge faced by most DevOps teams. Let’s just take a moment to consider the complexity involved:

A typical microservices application deployed on a Kubernetes cluster has numerous Pods, and each of those Pods will be at different stages of their delivery pipeline being deployed across development, staging, and production environments. That’s already a lot of moving parts for just one application, and organizations have more than just one application.

The 2021 Datadog Container Use report shows that on average, an organization is running 1000 Pods, and the trend is rapidly rising. Therefore an average Kubernetes estate is running 20 clusters hosting 1000 Pods, with each Pod running at different versions as they are promoted through development, staging, and production. That’s a lot of configuration complexity to keep up with.

Git is the ideal tool to manage the complexity of all these different configurations. A previous blog post provided an overview of the many different ways to structure Git repositories for micro-service deployments across multiple Kubernetes environments. A real-world example of how Zscaler manages its multiple tenant deployments - here is the recorded talk from Zscaler.

Automating Complexity with Weave GitOpsSets

The pure GitOps approach does make it considerably easier to manage the complexity of deploying multiple applications across multiple clusters, however, there’s still quite a lot of manual work involved. Weave GitOps has automated most of the manual steps with numerous features - such as GitOpsSets - and wrapped it all up with a nice GUI. Let’s take a look.

What are GitOpsSets?

GitOpsSets enable Platform Operators to have a single definition for an application for multiple environments and a fleet of clusters. With a single resource definition, the Ops team can generate environment and cluster-specific configurations. GitOpsSets will create all the objects and Flux primitives that are required to deploy this application successfully. Now Operators no longer need to edit 100’s of files - all of that can now be done with a single command.

GitOpsSets combine Generators and Templates:

Generators: these are dynamic functions that can fulfill a variety of tasks, they generate key/value maps of data that can be injected into templates.
Templates: These are templated YAML files describing Kubernetes resources which are combined with the generated data and applied directly to the cluster.

GitOpsSets supports multiple generators including:

List Generator: The simplest generator which provides a list of Key/Value pairs that you want to feed the template with.
Git Generator: Enable to extract a set of files (environment-specific configurations) from a Flux GitRepository, and make the contents of these available to the templates, this would let you have config in app-dev.json, app-staging.json and app-production.json for example, and the contents of these would be available to the templates.
Matrix Generator: Combine generators into the desired compounded input.
Pull request Generator: Automatically discover open pull requests within a repository to generate a new deployment.
API Client Generator: Poll an HTTP endpoint and parse the result as the generated values.
Cluster Generator: Query for a set of Weave GitOps Clusters matching a set of labels.

Supercharge your Kubernetes adoption with the Assured Accelerator

The Weave GitOps Assured Accelerator program delivers a GitOps enabled Kubernetes platform in 4 weeks on any environment.

Learn more

How do GitOpsSets work?

Templated resources can be regular Kubernetes entities such as: Deployment, Statefulset, Service, etc. But to really get value with complete automation the Flux entities should be used: HelmRelease, Kustomization, etc.

The example environment consists of a management cluster running Weave GitOps and leaf clusters: Tom, Dick, and Harry. If these clusters were provisioned by Weave GitOps or provisioned manually then joined to the management cluster, the Flux engine will be automatically deployed to them.

Tom stage=staging

Dick stage=production

Harry stage=production

As an example, a HelmRepository and HelmRelease are created and committed to Git, reconciliation ensures they are deployed to the management cluster as templates. Next, using the GitOpsSets the distribution of these resources can be defined. The GitOpsSets includes templates for HelmReleases which can reference the cluster secrets to target remote clusters. Following GitOps principles, the definition for the GitOpsSet is merged into Git and the reconciliation deploys it to the cluster. The scheduler picks up the definition, deploying the resources to the target clusters. The Weave GitOps reconciliation engine on each of the target clusters then picks up those resources, and ultimately the Helm Chart is deployed on the target clusters.

The whole process is driven by GitOps principles. Following on from the example above, if the target cluster selector label is changed via a Git workflow from staging to production, the Helm Chart will be removed from staging and deployed to production; promoting the release. The initial staging deployment would be on the cluster Tom by the label selection. Changing the label selection to production would move the Helm Chart to Dick and Harry.

The target cluster definition supports multiple cluster failover with weighting but that’s beyond the scope of this introductory article.

How can Organizations Benefit from GitOpsSets?

Weaveworks GitOpsSets can significantly benefit organizations focused on managing complex cloud-native infrastructure and applications. By enabling the grouping of resources and applications into sets, it simplifies the process of applying configuration updates and tracking changes across environments.

Consequently, developer teams reap many benefits, including:

Reduces errors and speeds up deployment frequency and lead time by giving developers the ability to quickly review code changes in a short-lived preview environment.
Streamlines and shortens the process of getting changes into environments triggered through a natural pull request-based developer flow.
See the resources created from GitOpsSets and the status for each of those resources centrally in the Weave GitOps GUI.
Simplifies the arduous and complex process of defining repetitive, individual parameters and variables by abstracting them to a single definition, making it fast and easy to configure and deploy multiple environments.
Automates deployments to many sets of targets from a single abstraction, saving teams time and removing errors.
Reduces the number of files in the Git repository, no more wondering “is this file up-to-date with the template that it was rendered from?”

See GitOpsSets in Action

In our latest product launch webinar,Joe Dahlquist, VP of Product Marketing, and James Wilson, Director of Product Management, demonstrated the Pipelines features along with GitOps Sets and GitOps Templates.

Watch the webinar in its entirety or skip to some of the key highlights:

Speaker Introductions [min 2.00]
About Weaveworks [min 2:39]
Weave GitOps [min 3:34]
Product Announcements: [min 5:04]
Weaveworks Solutions [min 5:43]
New in Weave GitOps 2023.04 [min 9:12]

Pipelines [min 10:36]
GitOps Templates [min 11:32]
GitOps Sets [min 12:56]>

Weave GitOps Enterprise Demo [min 14.40 - 31.40]
Q&A [min 34.33]

Streamline and Automate Deployment with GitOpsSets

Brought to you by the GitOps and Flux creators, Weave GitOps Enterprise makes it easy to deploy and manage Kubernetes clusters and applications at scale, in any environment. The single management console automates trusted application delivery and secure infrastructure operations on-premise, in the cloud, and at the edge. In addition to GitOpsSets, Weave GitOps Enterprise boasts a number of other distinct features such as GitOps Run, Pipelines, Multi-tenancy, VS Code Extension, Terraform Controller, with many more to come.