Lunar Bank Uses GitOps to Standardize Workflows and Enhance the Developer Experience
Lunar Bank needed to remove obstacles inhibiting their developer experience and autonomy while ensuring security and compliance is baked-into the deployment process. To enable this, they turned to GitOps and policies.
About Lunar
Lunar is a 100% digital bank and a FinTech company that strives to democratize the power of money and the way we spend and invest it. Headquartered in Aarhus, Denmark, Lunar operates in Denmark, Sweden, and Norway serving its 500,000 active customers with a team of 650+ employees. Having received over €345 in funding, Lunar’s valuation is currently at $2.25B.
In the latest episode of our podcast the Art of Modern Ops, Bjørn Hald Sørensen, Engineering Manager of the Platform Team, and Kasper Nissen, Lead Platform Architect at Lunar Bank, are describing how Lunar Bank streamlined workflows and enhanced their developer experience with GitOps
Challenges
Beginning in 2016 with a monolithic system that was containerized, Lunar faced teething pains as their distributed systems grew quickly. Here are the key challenges:
1. Deliver a seamless developer experience
Lunar has been an early adopter of Kubernetes. Since then, they’ve been evolving their systems to become more streamlined and efficient. Today, a key priority for Lunar is to enable higher developer productivity and velocity, so product enhancement and new features are landing faster. The goal is to reduce any manual overhead for developers especially when it comes to managing Kubernetes and remove any blockers in the release process:
“Developers should be able to make any changes they need without being blocked by anything, or having to do the tedious work of configuring YAML files.” Kasper Nissen, Lead Platform Architect at Lunar Bank
2. Security and compliance cannot be an afterthought
Being a FinTech company, Lunar operates in a highly regulated sector and they cannot afford to compromise on security and compliance. Here again, the key is to automatically embed security and compliance in the deployment workflow, so developers don’t have to waste time and effort on manually creating configurations for every deployment.
“It's all about building in the (security & compliance) stuff that needs to be there for our developers. So they don't have to spend time doing this and can actually create features that benefit our customers.” Kasper Nissen, Lead Platform Architect at Lunar Bank
The goal is to bake security in from the start, instead of it being an additional hurdle for developers to overcome before rolling out new code that serves and improves the customer experience.
3. Monolithic architecture prohibits scale
Lunar’s monolithic architecture, that they built the developer platform on, made it difficult to scale, especially as the number of services and applications grew exponentially.
“We started with a monolith, and as it grew, we needed to figure out how to serve several hundred applications in a scalable way without the platform getting in the way of developers.” Bjørn Hald Sørensen, Engineering Manager of the Platform Team
Lunar’s monolith platform relied on Jenkins as an orchestrator and additional tools like Docker, Terraform, and Ansible to manage infrastructure. This turned out to be trickier than they had imagined as all services had to be deployed at the same time. As the teams kept growing, so did the complexity of orchestrating all release details which ended up with a lot of coordination, overhead and many avenues for failure.
Whitepaper: GitOps Accelerates Self-service for Developers and Operators
GitOps enables organizations to adopt a platform model along with a self-service developer experience and automation for operators. Create and manage development environments and deliver a modern developer experience.
Download NowSolutions
Rather than a single solution, this complex set of challenges required Lunar to come up with a multi-pronged approach:
1. Switch from monolithic to microservices architecture
Lunar already started building microservices for application but now the infrastructure needed to be transformed in order to support the rapid scale of services and speed of deployments.
“Much of the microservices value proposition is about how we could scale the platform to serve several hundred applications. It's been a journey, but it has proven really valuable to us at least.”
By adopting a cloud native infrastructure approach with an architecture that supports microservices, the platform team could provide more scale and velocity. However, through an imperative style of deploying yaml files, every deployment was manual, not repeatable and not observable. Hence the team had to take it a step further.
2. Adopt the declarative GitOps approach
Along with the microservices model, they also needed to change from an imperative to a declarative model. While it is possible to manage microservices the imperative way, once the system scales to hundreds of services and numerous deployments, it’s hard to manage this using the imperative model. The declarative approach would enable the team to do things like recovery of state, and inventory management at scale.
The move to declarative is something GitOps helped them accomplish. In order to introduce a more consistent, repeatable and observable deployment process, the team started to operate the platform in a declarative way. As a first step the team needed to be specific about which functionalities and layers to manage declaratively.
You need to have a goal in mind about what you want to achieve with GitOps. You really want to know what layer you are trying to make declarative, and to what degree it needs to be declarative. - Bjørn Hald Sørensen, Engineering Manager of the Platform Team
Lunar started moving artifacts to a Git repository, and the natural progression was to go all the way and adopt the GitOps approach. Kubernetes prefers a declarative rather than imperative way of building and managing systems, but it’s GitOps that operationalizes this declarative approach. With Git as the source of truth, all they need to do is declare the desired state of the system, and have a GitOps agent like Flux implement this state in production automatically. Once this happens, they can simply check the state of the system in Git, receive notifications in case of drift and be certain that the same state is running in production - such automated and reliable deployment patterns give teams tremendous confidence.
3. Leverage policies with Flux & Crossplane
Being a financial institution and undergoing frequent and stringent auditing processes, Lunar needs to monitor very carefully which changes go into production and by whom and when. To that end, they created an open-source tool called Release Manager that has pre-defined policies that can restrict Git branches to certain environments - basically security guardrails that prevent any potential conflict. With this in place, all teams can now directly deploy all reviewed changes into production, while some teams can also choose to release only into a dev environment if needed. This enforces compliance requirements by default.
Using a policy engine with Flux makes GitOps processes far more convenient. If the service is running, it is implied by default that the environment is compliant because all necessary compliance policies have been applied on the fly. If you want to learn more about this workflow - read up on trusted application delivery, a method that combines policy as code with GitOps workflows.
Results
Let’s take a closer look at what results the aforementioned changes brought to the teams.
1. Secure, reliable and autonomous deployments
The control and visibility of release has greatly improved for the development and platform team. For developers the deployment process is simpler, requires less cognitive overhead and saves time. The deployment process can be straight to production or as nuanced as the developer team decides it needs to be.
“If they need to go into a specific environment, they can configure the rules (or policies) for the release by themselves. For some teams, they’ll release code to production every time because they have a high confidence during tests. For other teams, they may require a more manual test approach. They send code into different test environments before it goes to production.” Bjørn Hald Sørensen, Engineering Manager of the Platform Team
At Lunar bank these rules and policies are automatically implemented by Release Manager. The flexibility and autonomy of development teams has been preserved, productivity even increased all while the platform teams are confident that all releases are protected by default through predefined security guardrails.
2. MTTR for failed clusters in under an hour
Apart from speed, another benefit is the ability to recover quickly from cluster failures. Lunar’s mean time to recovery for failed clusters is within the hour thanks to GitOps as providing them with resilience amidst changes.
“We use GitOps to recover or bootstrap a cluster, and get the entire system back up and running using all the data that is stored in the Git repository. It takes about an hour in total from zero to a fully functioning cluster.” Kasper Nissen, Lead Platform Architect at Lunar Bank
Using Git to define the state of the system is most valuable in times like this when an entire cluster needs to be spun up according to the desired state and then gradually shift traffic. GitOps gives Lunar the confidence to do this in record time.
3. Enabling Kubernetes clusters to autonomously bootstrap themselves
The final benefit is an ongoing effort at Lunar, the building of autonomous clusters.
“What we are trying to achieve is to make clusters autonomous and to have them report back their configuration and state. So, Crossplane and ClusterAPi are two projects that we are keeping a close eye on this year.” Kasper Nissen, Lead Platform Architect at Lunar Bank
The main benefit of autonomous cluster is the fact that the failover process (bootstrapping) is already defined. For example core components get automatically started in the right order and the control agent knows which environment it is running in and can report that information back to the rest of the services that are running on the same cluster. These pieces of intelligence can reduce the cognitive load of platform engineers as less services are needed to be handled manually. They are also looking to leverage cloud resources and bring GitOps to other layers in their system.
Based on their experience over the past years, Lunar advises organizations to establish clear goals for GitOps, meaning what layers to define declaratively and to what degree. Their story is one of transformation at every level, and the impact of this is clearly visible in the accelerated lifecycle through streamlined and secure process and reliability - all thanks to the declarative GitOps approach, and leveraging policies.
Your organization can reap these same benefits by leveraging a GitOps solution like Weave GitOps. It brings together Flux, Flagger, Helm, and other open source cloud-native tools to form a seamless GitOps pipeline that makes deployments fast, automated, and effortless. For an additional layer of security, compliance and audits, Weave GitOps added Policy as Code to ensure compliance from code to production. Contact us for a demo.
Listen below to the full podcast with Lunar Bank to know how they successfully implemented GitOps, and their plans to scale it in future.
The Art of Modern Ops · Lunar Uses GitOps to Standardize Workflows and Enhance the Developer Experience
