Separating CI from CD with GitOps and Other Insights from Kubecon EU
Originally scheduled to be held in Amsterdam in March of this year, Kubecon EU was postponed to August and eventually switched from an in-person event to a virtual one. In this post, Cornelia Davis provides the key takeaways from AWS Container Days held at Kubecon EU this year.
Last week was the first ever KubeCon Virtual conference. Originally scheduled to be held in Amsterdam in March of this year, the conference was initially postponed to August and then eventually went from in-person to virtual. I’ve attended quite a few virtual conferences in the last few months, but this one was a greater production than any of the others I’ve seen. I am grateful to the organizers and the team behind them that I’m sure worked tirelessly in the last few months to bring us this experience. Thank you!
Despite the fact that much of the program ran while I was sleeping out here on the west coast of the US, I did manage enough participation where it definitely felt like a conference week. Over the course of the coming weeks I’ll take in as many additional sessions as I can and will try to catch up on even just a small fraction of what transpired in all of the Kubecon slack channels . I do want to share my initial reactions with you now.
Priyanka Sharma, CNCF General Manager welcomes the user community
The official event kicked off with a keynote from the new general manager of the CNCF, Priyanka Sharma. I had, of course, read a good deal about Priyanka when her new appointment was announced and her session, while prerecorded, did not disappoint. She delivered great messages and announcements with enthusiasm and grace. One such announcement was the addition of SPD Bank to the CNCF user community.
At almost 150 members strong, Priyanka reported that this is the largest user community in an open source foundation. For my entire time working in OSS I’ve always represented a vendor, and indeed, Kubecon events have always felt a bit vendor-heavy to me, so I am very happy to see such emphasis placed on the user community at the CNCF.
Immediately following Priyanka’s keynote was one from Cheryl Hung, the Vice President of Ecosystem at the CNCF, where she talked about the efforts to support and leverage that user community. One specific example is the Tech Radar that Cheryl spearheaded with that community, identifying the strongest and most popular tools in the continuous delivery space - spoiler: Flux, which Weaveworks donated to the CNCF , and Helm were the strongest technologies in that report (I’ll say more about both below).
All of this effort around the user community is well summed up by Priyanka when she says that the CNCF are “champions of end-user-driven open source.”
But Kubecon isn’t just about the main program but is also about the pre-conference events, one of which was scheduled in the Pacific time zone so I was able to fully participate - AWS Container Day. The day was chock full of great content, much of which I’d like to give my perspective on, but rather than go too broadly, I’ll share over a series of posts. Be sure to sign up for our newsletter, so you don’t miss any of them.
Kubernetes for AWS at Container Days
Bob Wise (@countspongebob), who leads all things Kubernetes for AWS, kicked things off with an overview of all of the places that Amazon is investing in the Kubernetes space. For the most part, you can see those themes reflected in the agenda for the day. EKS is central, of course, and many sessions touched upon it in the areas of networking, node group management, security and more. But Amazon is also investing in things that are relevant to any Kubernetes system - things like CDK8s, which provides an alternative to YAML engineering, and the AWS Controllers for Kubernetes (ACK), which “Kubeifies the AWS universe of services”. Personally, I am really excited by what my friend Jay has done on ACK, but that’s something I’ll explore more fully in a future post..
Bob also significantly highlighted something near and dear to our hearts here at Weaveworks: GitOps. Not everyone on the agenda talked GitOps quite as directly as Nate Taber (@nctaber), the Product Manager for both EKS and CDK8s, who in his session on CDK8s very noticeably said the words “deploy to Kubernetes” while his slide showed the commands:
git commit … git push
AWS embraces GitOps as an operational model
Bob’s emphasis as well as numerous mentions of GitOps throughout the day made it clear how central this new operating model is to their strategy. And he certainly made yours truly smile when he referred to “CICD” as a “dangerous” term - hallelujah! CI is a development process. CD is an operational process. The two are, of course, related but must be decoupled - it’s GitOps that provides the protocol that draws them together in a powerful way.
Speaking of GitOps, I want to draw your attention to something really exciting that is happening in EKS. As you know, Weaveworks are the primary caretakers for the preferred CLI for EKS, EKSCTL, so I can provide a bit of an insider’s view on this topic - if you will, I’ll help you read the tea leaves of what Nate said.
Simpler workflows for EKS add-ons
Under the heading of “simpler workflows”, Nate listed two things: EKS Add Ons and Integrated GitOps. They are two different things yes, but when they are brought together, that’s when the magic happens.
We all know that a base Kubernetes cluster leaves a lot of work to be done before the cluster can be easily used to operate workloads on it. You’ll need additional configurations for networking and storage, as well as various security and compliance configs. You’ll need to install and configure tools like Prometheus and Grafana to monitor those workloads. Having a standardized way of packaging these additional capabilities and managing the lifecycle of them across your entire Kubernetes landscape is critical and that is what Nate is referring to as an add-on.
I already hear you asking the question - “Isn’t that helm?” The answer is “yes and no”.
Yes, helm is about packaging, it parameterizes the deployments these charts encapsulate, and charts are versioned. While it’s not the only way of packaging components for Kubernetes deployment, it is, as noted above in the Tech Radar on CD published by the CNCF user community, arguably the most popular.
Cluster add-ons need more than Helm
But “No”, Helm alone does not deliver everything needed for what Amazon is envisioning for add-ons. What GitOps adds to Helm are the operational flows for delivering the right versions of Helm charts to the right clusters, upgrading them when necessary, and more. Git repositories as well as delivery controllers like Flux are the enablers for these additional capabilities.
Remember the comment I made earlier about loosely coupling CI and CD? That is showing itself here. Amazon will be using a CI process to develop and release those EKS add-ons. And the EKS user will use GitOps to continuously deliver up to date, security-patched versions of those add ons to their clusters. GitOps codifies the protocol between CI and CD. You’ve got to have both, add-ons and integrated GitOps, to enable the modern operational practices that are required in this cloud-native world.
Final Thoughts
I confess, I am missing in-person Kubecon a great deal, but I’m thrilled to have had the virtual experience this past week. Kubecon “Boston” is also going virtual and with the time difference being only three hours for me, I will fully participate with a great deal of joy and gratitude in my heart.
