Kubernetes Patterns: The ConfigMap Pattern
Wanting to learn about various Kubernetes patterns? Check out our blog post regarding the config map pattern.
In one of our articles, we discussed the environment variable pattern; how to define them, where to use them, and their potential drawbacks. We briefly touched the configMap and Secrets resources as a means of injecting external configuration settings to the Pod (and its running containers). In this article, we have a deeper demonstration of the configMap and Secret usage patterns and best practices.
What’s The Problem with Using Environment Variables?
There are several considerations that you must take into account when deciding to use environment variables for all your configuration needs:
- They are honored in multiple layers. For example, an environment variable that’s set in bash_profile of an image will be overridden if the same variable name was set in the Dockderfile. Even further, the same variable can be overridden in the Pod definition. Such behavior can cause hard-to-detect bugs when you’re not certain that your environment variable won’t get overridden by mistake.
- Environment variables cannot be changed once the application is launched. Modifying an environment variable requires restarting the container to apply the new data but depending on your deployment pattern, this may or may not be desired.
- More often than not, the external configuration is not limited to a bunch of variables, it spans throughout the whole configuration file. Think of php.ini, config.json, or package.json files. Those files need to be externally availed to the running container. The application expects to find its configuration file in a specific location rather than a set of environment variables.
- Using environment variables for storing sensitive data is a security risk.
Lab: Python Flask With a ConfigMap
Let’s have a practical lab. In this lab, we launch a Python Flask application that uses an external configuration file to define which message the API should respond with. The image expects to find a file called main.py in a directory called app. The main.py file looks as follows:
from flask import Flask
app = Flask(__name__)
@app.route("/")
def hello():
app.config.from_pyfile('/config/config.cfg')
return app.config['MSG']
if __name__ == "__main__":
# Only for debugging while developing
app.run(host='0.0.0.0', debug=True, port=80)Line 3 loads a configuration file called /config.cfg and makes use of it on line 6, where it returns the message to the client. The configuration file, config.cfg contents are as follows:
MSG="Welcome to Flask!"
Finally, the Dockerfile looks as follows:
FROM tiangolo/uwsgi-nginx-flask:python3.7
COPY ./app /app
COPY config.cfg /config/config.cfg
Let’s build the image, run the container and verify that it works as expected:
$ docker build -t magalixcorp/myflask .
Sending build context to Docker daemon 5.632kB
Step 1/3 : FROM tiangolo/uwsgi-nginx-flask:python3.7
---> 53353fb7df32
Step 2/3 : COPY ./app /app
---> 398f39d565cb
Step 3/3 : COPY config.cfg /config/config.cfg
---> ebf4ac67f220
Successfully built ebf4ac67f220
Successfully tagged magalixcorp/myflask:latest
$ docker run -p 8080:80 -d magalixcorp/myflask
9f4d904231ead30b38d0f858f0af816a22bf9878c9342d0712ecf90b6b0fc440
$ curl localhost:8080
Welcome to Flask!
Now, we need to run this application on Kubernetes. First, we need to create a configMap to hold our configuration file:
kind: ConfigMap
apiVersion: v1
metadata:
name: appconfig
data:
config.cfg: |
MSG="Welcome to Flask!"
Then, comes our Pod definition:
kind: Pod
apiVersion: v1
metadata:
name: app
spec:
containers:
- name: app
image: magalixcorp/myflask
volumeMounts:
- name: config-vol
mountPath: /config
volumes:
- name: config-vol
configMap:
name: appconfig
As you can see, we are mounting the /config directory to a volume that is backed up by our configMap. Each item in the configMap data part is transformed into a file on the volume. The key becomes the file name and the value is the file contents.
Now, let’s apply the definitions and test our configuration:
$ kubectl apply -f configmap.yml
configmap/appconfig configured
$ kubectl apply -f pod.yml
pod/app created
$ kubectl exec -it app -- bash
root@app:/app# curl localhost
Welcome to Flask!root@app:/app#
The container has a volume mounted under /config containing the config.cfg file. The Python script is designed so that it reads the configuration file each time an HTTP request arrives. Let’s change the message that the API outputs by changing the configMap:
kind: ConfigMap
apiVersion: v1
metadata:
name: appconfig
data:
config.cfg: |
MSG="Welcome to Kubernetes!"
And apply this modified definition:
$ kubectl apply -f configmap.yml
configmap/appconfig configured
You may have to wait for up to one minute until the changes are reflected in the mounted volume. You can check that we can receive the updated message by logging to the Pod and sending the HTTP request:
$ kubectl exec -it app -- bash
root@app:/app# curl localhost
Welcome to Kubernetes!root@app:/app#
Of course in a real environment, you should create a Service interfacing with this Pod and send the HTTP request to it. We needed to keep the example as concise as possible.
ConfigMaps As Volumes
In the preceding lab, we demonstrated how we can use a ConfigMap to load “hot” configuration from a mounted volume. The advantages of this approach can be summarized as:
- You can dynamically update the configuration and have the changes reflected in the application while it is running (as long as the application supports this).
- You can load lengthy configuration files by any number to the application.
- Making use of the separation of concerns principle by segregating the Pod definition from its configuration. Each file can be authored separately, thus allowing the same configuration to be used with multiple Pods.
However, mounting ConfigMaps as volumes is not a silver bullet to all your configuration needs. There are times when an environment variable is more suitable for your scenario. For example, setting the log verbosity level (debug, info, notice, warn, etc.) in a mounted file is not the best practice. Fortunately, ConfigMaps allow you to mix environment variables with mounted volumes. This is demonstrated in the following lab.
LAB: Mixing ConfigMap Volumes With Environment Variables
In this lab, we show you how we can use the same ConfigMap for exposing configuration data to the containers both as mounted-volume files and environment variables. Let’s create a ConfigMap definition that mounts a hypothetical application configuration file called application.ini and also an environment variable for the log verbosity level called LOG_LEVEL:
kind: ConfigMap
apiVersion: v1
metadata:
name: appconfig
data:
LOG_LEVEL: info
application.ini: |
foo=bar
var=abc
var2=xyz
The Pod that makes use of this ConfigMap may look like this:
kind: Pod
apiVersion: v1
metadata:
name: app
spec:
containers:
- name: app
image: bash
command: ["bash","-c","sleep 10000"]
env:
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: appconfig
key: LOG_LEVEL
volumeMounts:
- name: config-vol
mountPath: /config
volumes:
- name: config-vol
configMap:
name: appconfig
Let’s have a quick look at the key points in this definition before we apply it:
- Starting the 10th line, we define the environment variables that the container should have.
- Line 11: we specify the variable name (LOG_LEVEL) by which the container can reference the environment variable.
- Line 12 to 15: we instruct the Pod how to find the environment variable information: it should lookup a key called LOG_LEVEL in a ConfigMap called appconfig.
The rest of the definition mounts the /config directory on the container. This directory contains all the data specified in the ConfigMap, including the items that we used as environment variables. Let’s apply our definitions and test the results:
$ kubectl apply -f configmap.yml
configmap/appconfig created
$ kubectl apply -f pod.yml
pod/app created
$ kubectl exec -it app -- bash
bash-5.0# cat /config/application.ini
foo=bar
var=abc
var2=xyz
bash-5.0# echo $LOG_LEVEL
info
bash-5.0#
As you can see, we had the environment variable available as well as the configuration file. We didn’t include it in the demonstration, but if you list the contents of /config you’d find a file named LOG_LEVEL with “info” as its only content.
So far we used ConfigMaps for exposing external configuration data to containers. However, if this data is critical, it shouldn’t be injected through ConfigMaps for security reasons. Instead, you should use Secrets. Let’s have a small lab where we inject an imaginary API authentication file to our Pod through a Secret.
LAB: Using Secrets For Confidential Data
Assuming that we have our Google API authentication token: AIzaSyCNwsOvVIIzmJQ0fgp6s8zJJeWAdmrDF7M, we need to add it to a Secret so that we can use it securely in our container.
Secrets require that your data gets base64 encoded first:
$ echo -n AIzaSyCNwsOvVIIzmJQ0fgp6s8zJJeWAdmrDF7M | base64
QUl6YVN5Q053c092VklJem1KUTBmZ3A2czh6SkplV0FkbXJERjdN
Now, let’s create a Secret definition for this key:
apiVersion: v1
kind: Secret
metadata:
name: apiauth
type: Opaque
data:
auth: QUl6YVN5Q053c092VklJem1KUTBmZ3A2czh6SkplV0FkbXJERjdN
Notice that we could equally create the Secret using the command line as such:
kubectl create secret generic apiauth --from-file=./auth
Where auth is a text file containing the base64-encoded string.
Our Pod definition should look as follows:
kind: Pod
apiVersion: v1
metadata:
name: app
spec:
containers:
- name: app
image: bash
command: ["bash","-c","sleep 10000"]
volumeMounts:
- name: apiauth-vol
mountPath: /config
readOnly: true
volumes:
- name: apiauth-vol
secret:
secretName: apiauth
As you can see, the definition is very similar to the one that used a ConfigMap in our previous examples with some minor differences. For completeness, let’s login to our container and ensure that the API key is where it should be:
$ kubectl exec -it app -- bash
bash-5.0# cat /config/auth && echo
AIzaSyCNwsOvVIIzmJQ0fgp6s8zJJeWAdmrDF7M
Base64 is NOT an Encryption Algorithm: Securing Your Secrets
Base64 is a method that is commonly used to encode binary data into text format. This allows data to be easily transferred through different network mediums. Base64-encoded data can be easily decoded using the base64 command. There are also many online tools that offer base64 encoding/decoding. Hence, nothing prevents a hacker from decoding your base64-encoded password if she gained access to it. So, does that mean that Secrets are no different than ConfigMaps security-wise? Well, not exactly. Kubernetes implements Secrets in a special way that makes them more secure:
- A Secret is availed to a node only if this node has Pods that use this Secret.
- When a Secret is put on a node, it is not stored on physical storage. Instead, it gets saved to memory using tmpfs.
- When storing Secrets in Kubernetes configuration (etcd), it is stored in an encrypted form.
Hence, it’s safe to say that your Secrets are secured when stored and handled by Kubernetes. However, it is your responsibility to handle their security when used by the application. One possible way is using your own encryption mechanism when storing and retrieving data from your Secrets. That is, encrypting the data before base64-encoding it then unencrypting it through the application.
Needless to say, you should never commit your Secret definition to version control.
TL;DR
- ConfigMaps and Secrets allow you to store your application configuration settings on a different resource than the Pod definition. Thus, establishing a loosely-typed relation between the definition and the configuration.
- Using separate resources for your configuration allows you to use it with different Pods. If you need to make a configuration change, you only need to make it in one place.
- You should use Secrets when storing sensitive data because Secrets are handled more securely by Kubernetes. However, Secrets have the limitation of allowing no more than 1 MB of data per Secret. A reasonably adequate limit if you used Secrets for just passwords and keys. Hence, do not use Secrets unless the data is confidential.
Whitepaper: Production Ready Checklists for Kubernetes
Download these comprehensive checklists to help determine your internal readiness and gain an understanding of the areas you should update
