Instrumenting Real-World Pre-Deployment Checks for GitOps
GitOps is a revolutionary practice that adopts philosophies of CI/CD and Infrastructure-as-a-code, but it needs a systematic approach. In this article, we’ve curated a list of pre-deployment checks that’ll ensure seamless deployment.
GitOps ensures the agility and scalability that containerization promises via efficient container management using orchestrators like Kubernetes. One key factor that accelerated the adoption of GitOps is its comprehensive operating model. It allows you to make changes to the code, identify errors, review code and roll back changes through a simple process. Despite these strengths, there are certain pre-deployment checks that would benefit any GitOps practice. That is the focus of this post.
Before we understand how GitOps does that, a quick refresher on what GitOps is.
GitOps is a set of principles that powers the complete automation of continuous delivery pipelines. It uses Git as the single source of truth to store and manage infrastructure and application configuration. With GitOps, you declaratively define the desired state of the system in Git. You can make changes to the desired state in Git repositories using pull requests. Once approved, these changes are automatically applied across the live pipeline. All through the process, GitOps operators ensure that the current state and the desired state remain the same.
Now, to look at a list of pre-deployment checks that will help you build a streamlined GitOps practice.
GitOps Pre-Deployment Checks
Building a resilient and robust GitOps pipeline requires a systematic approach featuring security checks, storage provisioning, and monitoring activities.
1. Configuring Kubernetes clusters
Kubernetes is the most ideal orchestrator to manage containerized applications because of its ability to maintain execution consistency across different environments. To leverage Kubernetes benefits such as autoscaling and self-healing, you must optimize its performance. By setting proper usage limits, namespaces, and resource limits depending on the workloads, you can streamline the usage across the infrastructure.
When it comes to GitOps, it is essential to define the state of the Kubernetes cluster using Git and YAML files. Kubernetes was built to be a declarative system, and GitOps is the way to make this a reality.
2. Implementing a GitOps Pipeline
Implementing GitOps is very different from a traditional CI/CD pipeline. It puts Git at the heart of the process, driving the entire workflow. Also, the reliability of the entire codebase can be guaranteed by testing, building, and managing container instances within a registry. Below are the elements crucial to implementing an effective GitOps pipeline:
- Git Management: You will need a Git hosting platform like GitHub or BitBucket to ensure streamlined storage and processing of your code. It also facilitates collaboration, code reviews, issue tracking, and managing changes through the versioning control system.
- Container Registry: GitOps leverages containers and requires you store all application components as container images. You can also use a cloud-based registry service like Docker Hub for storage. This registry integrates with GitOps agents like Flux to automate the deployment of updated container images to a production cluster.
GitOps also allows you to leverage tools like Flagger for progressive delivery and to automate rollbacks in case of errors. Flagger can enable you to implement complex deployments like canary, blue-green, and A/B testing.
3. Observability to Maintain Health
People often get confused between observability and monitoring or use them interchangeably. However, there is a sharp distinction between the two. But to give you a basic idea, monitoring informs you when something goes wrong, while observability tells you why something went wrong. In a way, monitoring is a subset of observability.
With observability, you supervise the entire workflow from system design and coding to testing. You can take in factors like system metrics, log management, and implementation time. Below are the key parameters you need to observe:
- The difference between the desired state declared in Git and the current, existing state of the system. This will tell you if your commit was successful or if you should change anything.
- Tracking the time it takes for you to bring a new feature to market, and optimizing the deployment pipeline to achieve greater release velocity.
- Assessing your system’s readiness to deal with outages and building in resilience.
- Monitoring system health by identifying bugs, latency, and security vulnerabilities.
Monitoring tools like Prometheus help you gain visibility into your system performance in real-time. You need to implement an efficient alert system that triggers rule-based notifications.
4. Secrets Management
When it comes to secrets in a cloud-native system, things get a little tricky. Accessing secrets in Kubernetes clusters is possible if one has API access as these secrets are stored in an unencrypted format. Therefore, it is important to encrypt them at REST in addition to configuring role-based access control (RBAC) rules. In any case, there are multiple tools available to store and manage secrets within a GitOps workflow in a secure manner. Some of them are listed below:
- Bitnami Sealed Secrets: With Sealed Secrets, you encrypt the secrets with a CLI ‘Kubeseal’, and it can only be decrypted by the cluster-side controller.
- External Secrets Operator: This tool delivers sensitive information from third-party platforms like AWS Secrets Manager as Kubernetes secrets automatically.
- Hashicorp Vault: The encryption-as-a-service platform stores your secrets at rest and allows identity-based authorization. More on the blog.
5. Security Against Vulnerabilities
GitOps supports continuous security to mitigate vulnerabilities, security threats, and errors. You can implement a range of measures like data management, embedding security protocols, permission management, and regular progress auditing to safeguard your system.
Weave GitOps Enterprise is a trusted partner for many enterprises that need to enforce GitOps practices and security protocols, and standardize governance policies. It ensures that security guardrails are in place, which often gets overlooked in an automated ecosystem. Below are the two most critical pre-deployment security threats that GitOps faces:
- Securing cloud-native infrastructure: Modern distributed systems make container-wise detection of vulnerabilities hard. Weave GitOps Enterprise tackles this challenge by implementing security policies that block the application of any configuration that doesn’t comply with predefined security policies.
- Unsecure image registries: Businesses often fail to implement governance policies when it comes to image registries. This opens up their registries to attack. With Weave GitOps Enterprise, you can set policies to ensure that containers are built from secure and verified base images, and store them in verified registries. It also allows you to build an authorized list of container images.
Conclusion
GitOps has been in the spotlight in recent years and all for the right reasons. It helps organizations build a seamless chain of tools and frameworks to implement continuous application delivery, version control, and infrastructure-as-a-code. As you build and enhance your GitOps pipeline, ensure the steps mentioned here are in place as a part of your pre-deployment process.
Whitepaper: Accelerate Software Lifecycles Through DevOps Automation
We survey the entire landscape of automation from a cloud-native perspective and offer an adoption path for your organization.
