Deploying And Maintaining Kubernetes Using Ansible
Learn how an Ansible Operator can manage your Kubernetes clusters and associated applications that a generic operator can’t.
Writing an operator can be difficult because of the amount of Kubernetes component knowledge required to do so. The Operator SDK is a framework that uses the controller-runtime library to help make writing operators simpler - it enables the development of an Operator in Helm, Go, or Ansible.
What Can An Ansible Operator Do That A Generic Operator Can’t?
An Ansible Operator offers the same things that an Ansible does, but with a lower entry barrier and faster iterations. It essentially delivers the power of Ansible and its ecosystem. Around the period CoreOS was being acquired by Red Hat, the release of a framework and development kit for a concept known as Operators was developed. Operators have now evolved and become one of the most effective ways of managing Kubernetes clusters and associated applications. However, it can sometimes be difficult to define what an operator is, or explain how a cluster can be improved. While they do a good job of abstracting out Kubernetes objects, it’s a little difficult to explain the interaction between objects and operators.
We’re going to attempt to solve a totally selfish problem here - we’ll write a Kubernetes Operator that’s intended to deploy customized customer workshops. This way, we’ll spend less time deploying them manually when a workshop attendee books their environment. Before we proceed with highlighting the solution, let’s give a quick overview of the problem we intend to solve.
The first task will be to create a workshop resource within Kubernetes which deloys and maintains the shared resources for the workshop. This includes things such as container registries, shared lab content, etc. For each deployed workshop, we should be able to specify each of these shared resources.
Next, the workshop object must be designed in such a way that it accepts a given number of students. It will then use that number of students to deploy resources that create content customized for each student like lab guides databases and so on.
How Operators Work?
Although components of Kubernetes operators are not new in themselves, they are built and implemented in a revolutionary way. In its most fundamental form, an operator is a CRD (Custom Resource Definition) as well as a Custom Controller.
The purpose of a controller is to integrate a CRD into the database of the Kubernetes API. They also control how customer resources and users interact. The beautiful thing about Operators is that they can abstract away the need to manage these interactions on your own. The CRDs do all the heavy lifting required for your custom work.
CRDs and Custom Controllers make it possible to extend the Kubernetes database arbitrarily. This allows it to handle all sorts of data as it relates to different aspects of your application and its platform lifecycles. Thanks to operators, any IT professional or team now has this capability, instead of its use being restricted to elites.
Ansible Operator
The Ansible Operator uses an ansible-runner within its CRD. This operator is able to execute specified roles and playbooks infinitely. Although the Ansible Operator is not as flexible as the Golang Operator for many tasks, it still proves to be useful for most projects.
Step 1: Creating An Operator
After installing the operator SDK, a binary named operator-sdk will be installed on the system. The following are the steps to follow to create a new operator.
- name: You give a name to your operator after the new parameter.
- api-version (API Group): You specify the API group in a REST path as well as the apiVersion field of the serialized object.
- kind: This is a string value which represents the REST resource that the object represents. The Server may infer the kind from the endpoint that the clients submit their request to. This cannot be updated.
- type: This specifies the type of operator you intend to build i.e., the Ansible operator.
- tcluster-scoped: Since the operator will be creating new projects and resources within them, we’ll need it to gain access to the entire cluster, and not just one namespace.
$ operator-sdk new Worksnop-operator \
--api-version=workshops .operator.redhatgov. io/v1 \
--kind=workshop \
--type=ansible \
--cluster-scoped
The output should be as following:
INFO [0000] Creating new Ansible operator 'workshop-operator'.
INFO [0000] Created deploy/service_account.yaml
INFO [0000] Created deploy/role.yaml
INFO [0O00] Created deploy/role_binding.yaml
INFO [O000] Created deploy/crds/workshops_v1_workshop_crd.yaml
TNFO [O000] Created deploy/crds/workshops_vl_workshop_cr.yaml
INFO [O000] Created build/Dockerfile
INEO [OO00] Created roles/workshop/README.md
INFO [0O00] Created roles/workshop /meta/main.yml
INFO [OO00] Created roles/workshop/files/.placeholder
INFO [O000] Created roles/workshop/templates/.placeholder
INFO [O000] Created roles/workshop/vars/main.yml
INFO [O000] Created molecule/test-local/playbook.yml
INFO [0O00] Created roles/workshop/defaults/main.yml
INFO [O000] Created roles/workshop/tasks/main.ym1
INFO [O000] Created molecule/default/molecule.yml
INFO [0O00] Created build/test-framework/Dockerfile
INFO [0O00] Created molecule/test-cluster/molecule.yml
INFO [OO00] Created molecule/default/prepare.yml
INFO [OO00] Created molecule/default/playbcok.yml
INFO [0000] Created build/test-framework/ansible-test.sh
INFO [OO00] Created molecule/defaults/asserts.yml
INFO [0000] Created molecule/test-cluster/playbook.yml
INFO [O000] Created roles/workshop/handlers/main.yml
INFO [0000] Created watches.yaml
INFO [0000] Created deploy/operator.yaml
INFO [0000] Created.travis.yam1
INFO [0O00] Created molecule/test-local/molecule.yml
INFO [O0O0] Created molecule/test-1ocal/prepare.yml
INFO [O000] Run git init...
Initialized empty Git repository in /Users/jduncan/Code/workshop-
operator2/workshop-operator/.git/
INFO [OO00] Run git init done
INFO [OO00] project creation complete
$ tree
-build
- Dockerfile
- test-framework
- Dockerfile
- ansible-test.sh
-deploy
- crds
- workshops_vl_workshop_cr. Yaml
- workshops_vl_workshop_crd.yaml
- operator. yaml
- role.yaml
- role_binding.yaml
- service_account. yaml
- molecule
- default
- asserts.yml
- molecule.yml
- playbook.yml
- prepare.yml
- test-cluster
- molecule.yml
- playbook.yml
- test-local
- molecule.yml
- playbook.yml
- prepare.yml
- roles
- workshop
- README.md
- defaults
- main.yml
- files
- handlers
- main.yml
- meta
- main.yml
- tasks
- main.yml
- templates
- vars
- main.yml
- watches.yaml
What we have above is the base model for the Operator. Although it doesn’t do much, it contains everything we need. Let’s look at the structure of this directory:
- build: The build directory holds a Dockerfile as well as some other tests that run when a build is triggered through the SDK.
- molecule: This is a test framework for Ansible roles.
- roles: When the SDK creates a work environment, it also creates an empty Ansible role.
- watches.yaml: The purpose of this file is to tell the SDK the roles and playbooks that are to be inserted into the CRD container when it is built. This is where you instruct the Operator SDK about the Ansible that you intend to run as the operator.
Step 2: Making Our Operator Do Something
The first task is to make the operator create a namespace that will hold the shared resources for the workshop. This is a simple role and something we can use to confirm that the operator works when we deploy it.
To start, add the task below to roles/workshop/tasks/main.yml. The code that configures the ansible-runner within the container is expected to set the meta name variable inside the container at run time.
name: Create project for global workshop content
k8S:
api_version: v1
kind: Namespace
name: ‘{{ meta.name }}”
When this initial task has been added to the role, we can proceed to build an operator to see how it works.
Step 3: Building The Workshop Operator
With the Ansible Operator built, we can now incorporate the playbooks and the roles that are referenced in watches.yaml. This is done by using a specialized base image that contains the ansible-runner. This process is controlled by the build/Dockerfile.
$ cat build/Dockerfile
FROM quay.io/operator-framework/ansible-operator: v0.6.0
COPY roles/ ${HOME}/roles/
COPY watches.yaml ${HOME}/watches.yaml
Step 4: Deploying The Workshop Operator
Once the new operator was created, it also created deploy/operator.yaml. You’ll need to change the default values to specify the uploaded image, as well as a restart policy. It’s important to note that the opererator.yaml creates a Kubernetes Deployment Object.
After making these changes, it’s time to deploy the initial version of the operator. This can be done by running many of the yaml files within deploy. Because this is being deployed into a Kubernetes cluster, the oc command will be used.
First, we need to create a project to house the new operator, then create a service account for it to use. Next we give the operator a role, and then bind the role to the service account. Once this is done, we can begin to deploy the CRD that the operator will use. Finally, the operator itself can be deployed with references to the role binding earlier created.
These steps complete the deployment process. You can now check to see if the deployment is up and running.
Step 5: Extending The Workshop Operator
Watching Playbooks Instead Of Roles:
The Ansible Operator checks for specific roles by default. However, the best practice when dealing with Ansible is to use smaller roles then pull them all together into one playbook. The following steps highlights how to do this:
- Create a playbook that references the role (or roles).
- Instruct the watches.yaml to look for playbooks instead of watching for the default Ansible role.
- Add the playbooks directory to build/Dockerfile.
In order to create a playbook, you’ll have to create a directory for it within the top level directory of your operator. A file named workshop.yml has to be created within that directory.
Once this is done, the playbook will look within the roles directly for the workshop role we created. Next, we can edit the watches.yaml so that it now references the playbook, rather than the role. For both the playbook and the roles, the operator container image that you build with the SDK will be located within /opt/ansible as its parent directory. Ensure that this is the path that you use in place of the default path on your development system.
Finally, ensure that the build/Dockerfile includes the playbooks directory when it builds our operator image.
TL;DR
- Ansible Operator gives its users a lower barrier to entry, faster iterations, and the power of Ansible and its ecosystem.
- The Operator SDK is a framework that uses the controller-runtime library to help make writing operations in Kubernetes easier.
- CRDs are the actual resources that do the heavy lifting for custom work.
Whitepaper: Production Ready Checklists for Kubernetes
Download these comprehensive checklists to help determine your internal readiness and gain an understanding of the areas you should update
