WKSctl: GitOps Management of Kubernetes Clusters with Jerry Jackson
Recently, Jerry Jackson, Software Engineer at Weaveworks, spoke about WKSctl and demonstrated how to create a cluster with this open source project at a Weave Online User Group (WOUG).
In a recent Weave Online User Group (WOUG), Jerry Jackson, Software Engineer at Weaveworks, spoke about WKSctl and demonstrated how to create a cluster with this open source project.
What is WKSctl?
WKSctl is an OSS tool that easily builds Kubernetes clusters on a given set of machines and then manages them with GitOps. It is currently based on v1 of the Cluster API. To start creating clusters, there are very few requirements:
- Cluster description:
- Subnet definitions for services and pods
- Path to the ssh key with access to all machines
- Username of ssh user
- Boilerplate configuration of the yum repositories and Docker
- Machine descriptions (IP addresses, ports, roles (master/worker))
- Git repository
What can you do with WKSctl?
Fundamentally, you can construct Kubernetes clusters using configurations that are stored in Git. Once you’ve created clusters you can also manage them via Git commits and perform upgrades, add/remove nodes, and other node changes like turning masters into workers and vice-versa.
Why manage clusters with GitOps?
Well there are a lot of reasons, but the most important are:
- Git acts as your Single Source of Truth: Whenever you look at the contents of the Git repository you know what the state your cluster is in or what the state your cluster is trying to achieve. It also means that the definition of workloads is always accurate.
- Changes are recorded: You can use all of the standard GitHub tools to review and audit changes. You can ask people to submit pull requests and they get reviewed, approved (or denied). Then once they get applied there’s a complete audit trail of exactly how you got to where you want to be.
- Previous states can be easily restored: if something goes wrong with an applied change you can revert your Git repository and the cluster will work its way back to the previous state.
Want to know more about GitOps? Check out the What is GitOps, Really? blog post.
Components of Running WKSctl System:
Cluster API Project
The Cluster API project underlies WKSctl and it supports managing k8s clusters from within. The Cluster API defines a set of Customer Resource Definitions (or CRDs) to represent machines, clusters, and a few other ancillary things. It specifies that there should be a goal-seeking controller in the cluster that watches the machines and the cluster manifests. When changes or updates are made to the manifests, updates to the cluster will rollout to match those changes.
This works really well with GitOps since the cluster machine manifests can be managed just like service manifests (via flux for example).
Steps: Set up a cluster with wksctl
How to set up and manage a cluster with WKSctl and manage it with GitOps:
- Set up ssh connectivity on a set of machines.
- Define clusters with simple manifests in Git.
- Run the `wksctl apply` command to start processing.
- Run `wksctl kubeconfig` to get a kubeconfig file that provides cluster access.
That’s it! -- The cluster is created which can then be managed by Git updates.
Set up SSH connectivity
- Need a single private ssh key that can access all cluster machines
- Can use any user with `sudo` permissions
- User is specified in `cluster.yaml` (defaults to “root”)
- Key in cluster.yaml (release 0.8.1)
- Command argument (release 0.8.2+)
Define clusters with simple manifests
- cluster.yaml
- Ancillary files
- Docker-config.yaml
- repo-config.yaml
- Ancillary files
- machines.yaml
- cluster.yaml and machines.yaml specific to installation
- docker-config.yaml and repo-config.yaml are boilerplate
- All committed and pushed to GitHub
Note: At timestamp 13:17 in the video below Jerry talks about these sample.yaml files.
Once you have your cluster defined in GitHub, you only need to run wksctl apply, which is the workhorse command of wksctl for creating a cluster. In this demo it looks like this:
wksctl apply --git-url= \ --git-deploy-key
To see all of the functionality of wksctl, type wskctl --help or view the docs.
Footloose for machine creation
You can also use Footloose to create the machines that are underlying your cluster. Footloose is another Weaveworks open source tool for creating containers that look like VMs that you can run locally on your machine. It works with both Docker containers and ignite/firecracker microVMs. Some have described it like “Vagrant, but with containers.” It’s an extremely fast startup. Jerry’s demo runs on footloose machines.
Learn more about Footloose on GitHub or watch the Weave Online User Group session: An Introduction to Footloose.
Demo
Steps:
- Create GitHub repo and clone locally.
- Create and install a deploy key.
- Run wksctl apply.
- Run wksctl kubeconfig.
Here's a link to the slide deck as well.
After running wksctl apply, you’ll see lots of “Undone” and “Undoing” as the cluster is being created. This is because wksctl removes everything previously installed and get down to a known state. It then builds the cluster from this known state. This is also how wksctl updates the nodes. Node upgrades however are handled differently, and for those, wksctl uses kubeadm. Kubeadm assumes that the cluster is already running and that it’s in a decent state to apply a kubeadm upgrade.
Get a cluster up and running in no time with the WKS quickstart
If you’d like to experiment with wksctl or GitOps, and perhaps, create a local cluster for testing, the fastest and easiest way to do that is to fork and clone the wks-quickstart-firekube Weaveworks repository.
After you’ve forked and cloned the repo, you can simply cd to the cloned directory and run ./setup.sh which sets up a cluster on top of Footloose. The setup script specifically downloads all the required tools, creates your clustering machine configurations from the Footloose information and also sets up GitHub.
You can get a cluster up and running in no time with this quickstart. It’s a particular configuration and you won’t have all of the flexibility, but it’s easy to use if you want to get started quickly.
WKSctl Under the Hood
How does WKSctl actually do the installation?
- Initial Master Node installed by wksctl via commands over SSH
- wks-controller running on initial master node installs other nodes
- All Installation performed via “Plans” and “Resources”
- Resources represent individual tasks
- Execute a command or script
- Install a package
- Install a file
- Etc.
- Plans are resources that group other resources
WKSctl creates clusters on a given set of machines. If you’re looking for more features, as well as commercial support the Weave Kubernetes Platform (WKP) is also available.
For more talks and discussions like this and to be notified of new events, sign up for the Weave Online User Group.
