GitOps Days 2021: infrastructure as code and boosting developer productivity
At GitOps Days 2021, one of our popular panel discussions featured Kief Morris of Thoughtworks and Sneha Rao of Spotify, to discuss infrastructure as code in their respective organizations. Tune in to hear about the similarities and differences between IaC and GitOps and how GitOps can enhance IaC practices.
On the second day of GitOps Days 2021, Cornelia Davis chaired a discussion on the benefits of infrastructure of code, in the context of GitOps and developer productivity. You can watch the session here.
To look into the concept of infrastructure as code – and whether GitOps complements, duplicates or rivals it – Weaveworks CTO enlisted the help of two experts.
Kief Morris is the author of Infrastructure as Code, published by O’Reilly and now in its second edition. As Global Lead for Infrastructure Engineering at Thoughtworks, he has spent the last ten years working with clients on solving their infrastructure problems.
Having worked with IaaS for her entire career, Sneha Rao is Principal Product Manager in the Core Infrastructure team at Spotify. With thousands of features, millions of users and 7,500 employees to support, it is her team’s job to manage and optimise the company’s microservice architecture, boosting developer velocity without compromising on reliability.
How infrastructure increases velocity
With the introductions over, Cornelia posed her first question: does infrastructure as code enhance developer productivity?
Kief began by recapping on how the idea of infrastructure as code emerged. It began with the desire to make environments more consistent. After all, things can get messy when you build environments by hand, as was often the case before infrastructure as code gained popularity. This invariably led to differences between development, testing, staging and production environments – which in turn would lead to deployment failures and wasted time. By defining infrastructure as code, ops teams were able to solve these problems using tools from the software development world – agile engineering practices and pipelines, for example.
Sneha went on to explain how this idea is brought to bear at Spotify. Essentially, the company treats its infrastructure as a product with a target audience. That audience happens to be a community of software developers – some more experienced at the company than others. So they divide them into two sub-audiences. First is feature developers. Often, they are at an early stage in their careers. They lack the tribal knowledge that comes with experience at the company and as a consequence, they face an uphill battle, in which they have to learn both a new environment and a new ecosystem. To address this audience, Sneha’s team takes inspiration from software development, as Kief described. They build architectural design patterns they call blueprints that enable even new hires to focus on the work for which they were hired – building new features that will improve the customer experience. For more experienced engineers who, in some cases, have been with Spotify since the beginning, 15 years ago, it is an issue of focus. These engineers are used to having control over their environments. They are happy to work with the infrastructure – but therein lies the problem. All that access and all that control can be counter-productive. So Sneha’s team has a programme for these engineers they call fleet management. Everything is declarative and automated accordingly, to prevent them from getting bogged down in the infrastructure – which again, enables them to focus on software development. Crucially, all tools and workloads are managed through one seamless interface, giving her team a foolproof way to replicate what works, and deprecate what doesn’t.
Where does GitOps fit in?
It seems there can be no doubt that infrastructure as code can boost developer productivity. Yet that is a central promise of GitOps. So Cornelia asked Sneha and Kief about the relationship between the two.
Kief suggested we see GitOps as one particular school of thought – one way to do infrastructure as code. He used the analogy of agile software development. You can do it via SCRUM or extreme programming. There is no single right way.
Sneha agreed, explaining that she certainly didn’t see a scenario in which GitOps competes with infrastructure as code. In her view, the two are part of a single ecosystem. At Spotify, where they operate at very high scale, consistency is critical. GitOps is used widely, as is the practice of writing infrastructure as code – that means managing configurations, setting up declarative infrastructure pipelines and managing it all through a CI/CD framework, all with the aim of delivering new features to market faster. Feedback loops are important, as they need to be able to experiment, to pull back what doesn’t work and rescope when required. But this is all dependent on the infrastructure. If the foundation is not resilient to change and cannot react to this feedback loop, it is too brittle.
This is where a model like GitOps can really come into its own. The panel went on to discuss the importance of automation in those feedback loops and the need to make continuous use of it, rather than simply switching it on periodically – or giving developers the opportunity to avoid it. They discussed the shifting division between infrastructure and application development – a line that is blurring further as cloud providers start to offer functions as a service, and serverless architecture. And ultimately, they shared their thoughts on the future of GitOps and where it could go.
You can watch the full session here (it’s only 25 minutes long, believe it or not!)
