GitOps Thrives at the World’s Leading Cloud Vendor
AWS provides tools like CodeCommit, ECR, and EKS to build an end-to-end GitOps pipeline to manage application deployments in Kubernetes at scale and speed.
Businesses want to release applications faster without compromising on key priorities like security and compliance. This need has paved the way for GitOps - a fast, reliable, and simplified way to do fully-automated continuous delivery. The global leader of cloud services, Amazon Web Services (AWS), has been a longtime investor and technology partner (e.g. we jointly developed eksctl, the official CLI for Amazon EKS) of Weaveworks. Apart from this visible show of support, AWS has taken many steps to endorse GitOps practices in many of its services. The focus of this post is to show how AWS champions the GitOps methodology and the various AWS services that can be leveraged for a GitOps practice.
A Typical GitOps Delivery Pipeline
Before we explore the various AWS services, we will briefly outline what a typical GitOps delivery pipeline looks like. It consists of the following steps or phases:
- Source Code Management: Source code and infrastructure code is stored and managed in a Git-based repository. This is where your team can push code changes to be implemented. The AWS tool that you can use for source code management is CodeCommit.
- Build & Test: Continuous integration tools enable you to build containers and integrate them further with automated test tools. CodeBuild is the AWS resource that facilitates this stage of the GitOps pipeline.
- Publish: Container registries are an artifact storage service that enables easy deployment of containers. Amazon ECR allows you to publish container images.
- Deployment: This is the stage where infrastructure changes are applied and containers are deployed to the EKS cluster. Weave GitOps handles the deployment and continuous reconciliation of configuration and applications in the environment.
AWS Services that Support GitOps
GitOps is an ideal approach for managing Kubernetes on AWS. GitOps can be made operational using many AWS services like ECR, CodePipeline, and CodeCommit. Let’s take a look at these services.
CodeCommit to Facilitate Git Actions
To implement GitOps, your code needs to be stored in a version control system like Git that facilitates branching and change history tracking. CodeCommit is AWS’ git-based code hosting platform that offers these functionalities.
As a fully-managed source control platform, CodeCommit allows you to host and manage private Git repositories to store your source code for both applications as well as the infrastructure required to operate them. With CodeCommit, you can outsource the operational parts of managing source control systems like scaling up or down, and streamline collaboration with your team. As the service supports standard Git functionality, you can easily integrate your repositories with Git-based tools.
CodeBuild Streamlines Artifact Building
Once you have the codebase ready in CodeCommit, it is pushed ahead to build containers. The tool from the AWS family that enables this stage is CodeBuild.
AWS CodeBuild is a fully managed build service that produces deployment-ready software packages after compiling and testing source code. It is a continuous integration service that eliminates the need to manage your build services while scaling and processing multiple builds concurrently.
With CodeBuild, you can either create your build environments using your build tools or with pre-packaged ones. Regardless of any approach, CodeBuild ensures that your builds don’t wait in queues for deployment.
Publishing Container Images with Amazon ECR
After the containers are built using CodeBuild, the next GitOps step includes pushing the images into the container repository. Once the image is uploaded, it can be pulled automatically from the repository and run in EKS.
Amazon ECR is a managed container image repository service used to store, manage and deploy Docker images, and Open Container Initiative (OCI) images and artifacts. It integrates with IAM to provide your private repositories with greater security by allowing resource-based permissions. You can push, pull and manage container images through any CLI tool.
CodePipeline or Weave GitOps to Configure GitOps Workflow
AWS CodePipeline is a workflow management tool that helps you to automate the GitOps pipeline by releasing the process of your software by configuring all the stages of the process. It allows you to create and manage a process to build, test and deploy code in either a testing or production environment providing a continuous delivery service. It ensures that no bugs or errors are moved to the deployment stage by stopping the entire process whenever an issue is detected.
If you want to make your automated delivery pipeline even more reliable consider Weave GitOps to facilitate your continuous deployments. Weave GitOps continually reconciles the desired state of your applications, stored in Git, with the actual state, running in Kubernetes. As soon as a change is merged in Git, it will be applied to Kubernetes. Any change directly applied to Kubernetes will be automatically reverted back to the desired state, stored in Git; configuration drift is eliminated. Continuous deployment automation with an integrated feedback and control loop like Weave GitOos has, speeds up your mean time to deployment (MTTD) by supporting more frequent releases.
Provisioning Kubernetes Cluster with Elastic Kubernetes Services (EKS)
You can use Amazon EKS, a fully managed and secure Kubernetes service, to provision and manage your clusters and orchestrate the deployment of your containerized workloads. It eliminates the hassle of doing it all on your own. EKS offers a flexible, highly available, and secure ecosystem for simplified deployment, scaling, and management of your containerized applications.
By using Weave GitOps together with EKS, you can operate the Kubernetes cluster by declaring the desired state in Git as the single source of truth. It seamlessly enables changes made in the repository through pull requests. So, you can focus on building your application instead of worrying about the infrastructure.
eksctl
You can effectively manage clusters on EKS using a simple CLI tool eksctl jointly created by Weaveworks and AWS. It automates cluster creation with just one command eksctl create cluster and allows you to manage the entire lifecycle with GitOps.
Here is a step by step tutorial that walks you through the setup.
Creating Reliable Distributed Clusters with Elastic Kubernetes Service Distro (EKS-D)
Amazon EKS-D is a Kubernetes distribution based on and used by Amazon EKS to ensure the reliability and security of Kubernetes clusters. It helps you create clusters using the same versions of Kubernetes, its dependencies, and security patching support deployed by EKS. It enables you to standardize the EKS Kubernetes distribution running across different systems, on-premises or in the cloud.
Unlike EKS, you need to manually run EKS-D and deploy and manage clusters yourself.
Multicluster GitOps on EKS-D
Watch a demo on a multi-cluster and hybrid scenario with Weave GitOps (formerly WKP), EKS and EKS-D.
Learn MoreWhy is Weave GitOps an Ideal Kubernetes Reconciler?
Flux CD is a Cloud Native Computing Foundation (CNCF) project that facilitates continuous delivery with Kubernetes. Flux CD uses Git as the single source of truth to define the desired state of a cluster. Weave GitOps our free and open source continuous delivery project, is powered by Flux and available on the AWS Marketplace today.
When setting up a GitOps pipeline with AWS, it is recommended that you use Weave GitOps not just because of the self-healing capabilities it brings to the table, but also because it supports multi-tenancy. It uses multiple controllers to ensure that your Kubernetes clusters are in sync with deployments. It primarily helps you with,
- Automating complex tasks like updating YAML manifests
- Securing your GitOps pipeline
- Sending alerts in case of any deviations from the desired state
- Syncing with multiple Git repositories
Fidelity Practices GitOps on AWS
When Fidelity Investments, managing $8.3 trillion in assets and $3.3 trillion in total discretionary assets, wanted to digitize 500+ applications, the firm chose GitOps. Fidelity partnered with Weaveworks and AWS to build highly efficient Kubernetes applications. The key objectives were building a reliable platform for a heavily regulated industry with innovative business requirements on a replicative cluster configuration.
Fidelity built the FIDEKS platform with GitOps principles and Weaveworks tools on multiple clouds like EKS and Azure managing its 500 applications on Kubernetes. With GitOps automating platform configuration, the time to market was drastically reduced allowing Fidelity to introduce innovations at speed.
Fidelity realized the below-listed benefits by adopting Kubernetes via Weaveworks and AWS collaboration:
- Improved deployment speed: With an automated methodology where platform configuration stored in Git can be triggered through GitOps, resources can deploy applications faster without compromising on innovation.
- Reduced time to market: Fidelity could release newer applications and features quickly by managing processes directly from Git which drastically reduces development time.
- Faster disaster recovery: GitOps significantly improves recovery time if a cluster fails since GitOps uses Git as the single source of truth reducing recovery time from hours to minutes.
Conclusion
Weaveworks has pioneered the GitOps approach to accelerate application development by better management of application deployment and infrastructure changes. Today, GitOps accelerates your AWS and EKS operations in an ideal approach to managing application development at scale. Connect with your AWS rep, find us on the AWS marketplace or schedule a demo with one of our architects.
