How Policies Enable Safer And More Predictable Progressive Delivery
GitOps is the most operationally sound solution to automate the otherwise manual and complex process of progressive delivery. Weave GitOps, along with Flagger and Open Policy Agent (OPA) greatly automates, stabilizes and secures deployment processes for application teams.
Progressive delivery is a long-standing practice that builds on the capabilities of Continuous Integration and Continuous Delivery (CI/CD) to help organizations get more control over software rollout. With it, we are able to limit the potential negative impact on user engagement when releasing new product features. While the idea makes sense, progressive delivery is a complex mechanism if done manually. It requires various minor tweaking of things like traffic routing, security checks, change of deployment targets, and more. This can become tedious and error prone.
In this post, we discuss a practical way of implementing progressive delivery so that it is automated, and importantly, policy-based.
The Current State of Progressive Delivery
First introduced by James Governor of RedMonk, Progressive delivery has an important role to play in defining the future of DevOps. As a set of modern practices in application deployment, canary releasing, blue/green deployments, A/B testing, and feature flagging are core to its principles. While all these practices maintain subtle differences, fundamentally, they all are driven by the idea of releasing software updates and features to a progressively growing subset of users for bug verification and performance testing. Progressive delivery provides early feedback on a given application version and reduces the risk of deploying new features if the changes are negative.
While the idea itself is groundbreaking, there are certain pitfalls when it comes to implementing, managing, and monitoring all the phased releases. With lower traffic, it can take a long time to even gather a statistically relevant dataset, and even longer to exercise all code paths if the application is complex. Moreover, we need to manually enable and disable new features and shift traffic routing options. The entire process usually takes a few days for a progressive rollout and complicates the entire release pipeline. Even if we consider single-change deployments (which are easier to debug and rollback), multiple changes need to be batched together for a timely release.
Automating Progressive Delivery Through Weave GitOps
Given the challenges with progressive delivery, the process could do with some simplifying — and that’s where GitOps comes into the picture. With Weave GitOps, organizations can implement complex progressive delivery patterns that would otherwise be difficult to achieve.
Based on the idea of defining the infrastructure, configuration, and applications declaratively, GitOps requires the entire system to be defined in Git repositories. These repositories serve as the single source of truth and production environments are closely aligned with the source. If there is any drift or deviation from Git, Weave GitOps notices the drift and reverts the changes back to the desired state. This is the declarative approach in action and is what modern cloud-native systems are best suited for.
A key component in the GitOps toolkit, Flagger (part of Weave GitOps) is responsible for managing the step-by-step implementation of progressive delivery. Flagger integrates a GitOps agent with a service mesh like Istio, or Linkerd to split the flow of traffic (by percentage) between different versions of an application. It delivers core monitoring metrics such as latency, HTTP/gRPC success rates, etc. and displays live updates on the traffic split between the canary and the original versions of the application. Furthermore, to enable DevOps workflows, organizations can integrate Flagger with various collaboration tools like Slack. Doing this ensures developers receive timely notifications on the status of the release as it is happening.
MediaMarktSaturn achieved stable rollouts at high deployment speed
“With Flagger, we merge a pull request to deploy a new version, and basically forget about it because we feel so safe and sure about it not breaking anything in production.” - Bernd Stübinger, MediaMarktSaturn”
Read Case StudyLeveraging Kubernetes Network Policies For Traffic Restrictions
A network policy in Kubernetes enables us to enforce restrictions on inter-pod communication. For instance, we can set it so that when a pod connects to database pods, it has an app=web label. Practices such as these reduce the attack vector in the cluster but for the policy to be efficient, it must be implemented well. To that end, we must ensure that there is a network policy in place before pod creation and that no critical aspects are missed while creating the policy — a risk that can be alleviated by Open Policy Agent (OPA) (also part of Weave GitOps).
How OPA ensures a network policy exists prior to creating pods
Say, you have application pods that contain proprietary code and need extra protection. You need to ensure that none of the other pods except the frontend ones have access to the application. Here’s what you can do in such a situation:
- Develop a network policy that enforces the required restriction.
- Set up a quick lab to ensure the network policy is working as expected and create a deployment for protected pods.
- After deployment, ensure that you have two pods running which will now be connected using a permitted pod.
- Open a shell session to the container and establish an HTTP connection to the pod to verify that it is accessible via protected pods.
- Create another deployment with different labels for the client and don’t allow it access to the protected pods. Open another shell session to ensure the connection is not established.
Now, an OPA policy (written in Rego) can be deployed that denies creating the app=prop-labelled pods if the network security policy is not in place.
Validating and enforcing network policy rules
The OPA policy defined above only ensures the existence of a network policy. We need another means to allow connections to the proprietary-app pods only from pods labeled app=frontend. To that end, we can create and deploy another OPA policy to ensure that the policy object serves the final requirement. The point of all this is that you can have peace of mind amidst all the complexity of progressive delivery. This is only possible when you leverage policies and not wing it the manual way.
If you want to learn more about trusted delivery and how Weave Gitops automatically enforce security and compliance throughout the software lifecycle, schedule a demo with us today.
Final Thoughts
For any organization trying to automate feature flagging and progressive delivery, GitOps is the way to go. With various open source tools like Flux, Flagger, and Open Policy Agent, Weave GitOps is the most operationally sound solution for taking progressive delivery to the next step.
Whitepaper: Trusted Delivery with GitOps and Policy as Code
Download our latest whitepaper and learn how automated security and compliance checks, in the form of policy as code, make automated continuous deployments safe and secure.
