The Configuration Template Pattern
There are many ways you can inject outside configuration to your application containers. ConfigMaps and environment variables were always the obvious way.
Injecting external configuration data to a containerized applications is a crucial requirement. You can seldom find an application that does not accept (or require) customization that alters its behavior. There are many examples to demonstrate:
- Tomcat uses server.xml
- PHP depends on php.ini
- Node.js parses package.json
- Python Django depends on settings.py
The list goes on. Hence, there’s been more than one way to decouple the application from its configuration, and this saves you from having to alter the application code in enabling a feature or changing how the application behaves. When working with Docker, for example, you can use environment variables to define which database instance the application should connect to; prod or test. If the configuration data is stored in files, you can mount them through volumes so they can be modified separately. There are similar techniques that we can use in Kubernetes clusters e.g ConfigMaps, or configuration containers.
However, sometimes the configuration file is lengthy, and multiple versions need to be maintained. For example, you may have a settings file that has hundreds of lines. Adding this file to a ConfigMap is the obvious option. However, you may need to create several other ConfigMap resources for the same file with only slight modifications to serve different environments. For example, you may need to maintain two ConfigMaps for your php.ini, one for the production environment with error display turned off, and another for testing purposes that allows the script to display any runtime errors for debugging purposes. To avoid content duplication, we can only save volatile information (like DB name, log level, error display, etc.) in the configuration resource (ConfigMap or environment variables). The main configuration file is a skeleton (template) that gets changed at runtime according to the intended scenario.
The Configuration-Template Container Concept
In a non-clustered environment that uses Docker as the container runtime, you can follow this pattern by configuring the ENTRYPOINT script to modify the configuration template prior to starting the application. The template itself can be stored on the same application image or in a dedicated image that is used only for holding configuration data (for a deeper discussion on this, please refer to our unchangeable configuration pattern article). The docker scenario works as follows:
-
The container starts launching and fetches the configuration template.
-
The ENTRYPOINT script makes the necessary changes to the template according to environment variables.
-
The application starts and reads the processed template file as its configuration.
In Kubernetes, we can use the init containers for this purpose. Init containers always start before any other container in the Pod. The typical workflow may look as follows:
-
The init container fetches the configuration template from its image.
-
It uses the template processor to modify the template according to values grabbed from the ConfigMap.
-
Once it finishes processing the template, the init container moves the file to an emptyDir volume.
-
The emptyDir volume is shared with the application container. Thus, the application can access the configuration file.
LAB: Using Init Containers and Configuration Templates
In this lab, we demonstrate how we can use templates with init containers to create one skeleton configuration file. We’ll create a very simple Python Flask API that does nothing but display a message when it receives an HTTP GET request. This message is stored in a configuration file that gets loaded each time a request is received. Instead of creating multiple files for different messages, we’ll write a template and change the message according to a ConfigMap variable. The following illustration depicts what we are about to do:
The Application Files
Let’s start by displaying the application files:
app/main.py
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
app.config.from_pyfile('/config/config.cfg')
return app.config['MSG']
if __name__ == "__main__":
# Only for debugging while developing
app.run(host='0.0.0.0', debug=True, port=80)
config.cfg
MSG="Welcome to Flask"
Dockerfile
FROM tiangolo/uwsgi-nginx-flask:python3.7
COPY ./app /app
The Template (Gomplate)
There are many templating engines existing, for example, handlebars, Jinja, Tiller, and Gomplates. In this lab, we’ll use Gomplates. Our config.cfg should look like this:
MSG="{{ (datasource "config").message }}"
Let’s spend a few moment with this template. Anything between {{ and }} is parsed by the Gomplate processor. The datasource is one of the ways you can obtain external data (more details in the documentation). Through datasource, you can reference the name of a YAML file, config in our example, followed by a hierarchical path to the required value. Thus, our config.yml file should look as follows:
message: "Welcome to Kubernetes"
The Init Container
Our init container is tasked with the processing template file and producing a ready-to-use configuration file. Since we are using Gomplates, we’ll use the gomplate Docker image. Let’s create the necessary Dockerfile for this image:
FROM hairyhenderson/gomplate
COPY ./config.cfg /src/config.cfg
We build this image using a command like the following
docker build -t magalixcorp/goconfig -f Dockerfile-init .
Before we go on, let’s pause for a moment and see how a container that uses this image should work:
Assuming that our parameters file is located in ./parameters/config.yml, let’s create a directory for holding our processed file:
mkdir dest
Our docker command should look as follows:
docker run -v $(pwd)/parameters:/parameters -v $(pwd)/dest:/dest goconfig -f /src/config.cfg
-d config=file:///parameters/config.yml -o /dest/config.out
In the above command, we are mounting the parameters and the dest directory so that we can dynamically change the message or where the should container drop the processed file. In a Kubernetes cluster, the parameters file would come from a ConfigMap while the output directory would be mounted on an emptyDir volume shared with the application container.
For completeness, you should find a dest/config.out file that contains:
MSG=Welcome to Kubernetes
Applying On Kubernetes
Now we make use of the resources Kubernetes provides us, so we can have a dynamically configured application.
The ConfigMap
Create the ConfigMap using our config.yml file as the source:
kubectl create configmap config.yml --from-file=parameters/config.yml
The Pod
Our Pod definition contains two containers; the init container and the application container:
kind: Pod
apiVersion: v1
metadata:
name: myflaskapp
spec:
initContainers:
- image: magalixcorp/goconfig
name: appconfig
args: ["-f", "/src/config.cfg", "-d", "config=file:///parameters/config.yml",
"-o", "/config/config.cfg"]
volumeMounts:
- mountPath: "/parameters"
name: params-vol
- mountPath: "/config"
name: config-vol
containers:
- image: magalixcorp/app
name: app
volumeMounts:
- mountPath: "/config"
name: config-vol
volumes:
- name: config-vol
emptyDir: {}
- name: params-vol
configMap:
name: flaskconfig
Let’s have a quick look at the important points in this definition:
- The init container is running the goconfig image that we built earlier. The container just runs the gomplate binary, which requires the template file (embedded in the image); the parameters that it will inject into the template (mounted through the configmap), and the path where it will save the destination file. This destination path is mounted through an emptyDir volume.
- Hence, the process starts with the init container grabbing the template that it has, uses the configmap to process its values, outputs the final file to the shared volume for the application to use.
- Init containers always run before the application containers. Thus, the configuration file is guaranteed to be processed and ready when the application attempts to use it.
Now, let’s apply this definition and ensure that the application works as expected:
$ kubectl apply -f pod.yml
pod/myflaskapp created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
myflaskapp 0/1 Init:0/1 0 4s
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
myflaskapp 1/1 Running 0 13s
$ kubectl exec -it myflaskapp bash
root@myflaskapp:/app# curl localhost && echo
Welcome to Kubernetes
root@myflaskapp:/app# cat /config/config.cfg && echo
MSG="Welcome to Kubernetes"
root@myflaskapp:/app#
By applying the definition, the init container started first, processed the file and availed it to the application container. Once the process starts, we could log in to the container and issue a simple curl request to examine the response. The response matches the message that we specified in the configmap.
TL;DR
There are many ways you can inject outside configuration to your application containers. ConfigMaps and environment variables were always the obvious way. However, depending on your scenario, you may need to tweak things a little. For example:
- Environment variables are more suited to a small subset of variables. If you need to use long configuration files with many values, ConfigMaps are a better choice.
- ConfigMaps also have their drawbacks. For example, due to etcd inherent limitations, you cannot save a single configuration that is larger than 1 MB.
- For very large configuration files with many variables, we can use a template. Only the configuration settings that are likely to change should have placeholders.
- The template file (no matter how large) is embedded in a configuration image that is later used by an init container.
- The values that need to be injected are added to a configmap.
- The init container is responsible for grabbing the values from the configmap, processing the template, and saving the result of emptydir volumes.
- Because emptydir volumes are shared with all containers on the same Pod, the application container can access the processed configuration file.
- If you need to deploy the application to a different environment where you need to make a configuration change, you only need to modify the configmap. The difference here is that the configmap does not contain a lengthy configuration file, only the values that need to be changed.
Whitepaper: Production Ready Checklists for Kubernetes
Download these comprehensive checklists to help determine your internal readiness and gain an understanding of the areas you should update
