Using Cloud-Native BuildPacks for Operational Efficiency
CNB offers a rich capability for building OCI images. Learn about layer caching practices, image inspection, and runtime rebasing.
Containers have changed the way of application development. Developers are not only building applications, but they often perform the task of application containerization by using Docker files. Building containers from the start often helps the team package the application and its dependencies, resulting in a more reliable product and improved development velocity.
As the project reaches its rollout milestone, enterprise operation teams start collaborating with development teams. Operation teams have a different perspective on the application container images. The ops team aims for security and standardization of the application’s various facets.
Their evaluations are around the following elements:
- What is the base image for the application container? Which version of that image should be used?
- What are the different versions of dependencies for the application?
- What are the ports exposed by the applications?
- Does the application have a prescribed logging format?
All the above decisions are aimed to optimize operational efficiency and enterprise security. Development teams need to update Docker files to attain the above goals. Often the operations team is also responsible for maintaining the deployed application containers. If there are CVE fixes in the base image, then ops need to update the Docker files with the newer version of the base images. To summarize, Docker files are owned by the development team, but there are continuous demands from the operations team. The development team must be well versed with Docker practices and syntax to support these needs.
Cloud Native Buildpacks (CNB) is an alternative to Docker files. CNB provides a reliable, safe, modular, and fast way to build Open Container Initiative (OCI) spec-based images. These images can be deployed to any open container runtime like Docker, RKT, etc. CNB tool enables decoupling between the development and operations teams. Its focus is to provide container maintenance and security without compromising developer flexibility and productivity.
Getting Started
This post demonstrates CNB’s focus on operational governance while building application containers. It starts by building an example project, followed by enforcing different operational practices.
If you'd like to follow along with this tutorial on your machine, you'll need a few things first.
- The pack command-line tool is provided by CNB. It will be used to work with various examples. The documentation offers various ways to install the CLI. Validate the CLI by using the following command:
pack --version
- Locally running Docker. CNB uses Docker for its purpose. You can also use a remote Docker daemon.
- The
fakeuserapplication provides an API to generate random usernames. It is a web application created using Java and Spring. The application is built using maven. All code and configuration accompanying the article are available on GitHub.
./mvnw clean install
You can run the application and generate a username by using /usernameAPI
curl <a href="http://localhost:8080/username">http://localhost:8080/username</a>
The fakeuser is a simple application, but it provides the test-bed for working with CNB. You can build the container by using the following command :
pack build fakeapp
If you look at the generated output, you can see pack initiates a Docker build in the following steps.
- It started the application build by selecting a base image. CNB calls this base image a build image.
- It executed a lifecycle of Detect and Build phases.
- The detect phase selects different applicable
buildpacks - The build phase is responsible for executing selected
buildpacks. Eachbuildpackis applied as an abstract layer. - A buildpack is executed in either build or launch mode. Build mode means the execution results are available only during the application build process. On the other hand, the launch mode will mean the pack will be applied during the build, and the results are preserved for the container image.
- Lastly, the layers generated by
buildpacks, executing launch mode, are copied to a different base image. CNB calls this base image a runtime image.
The image is named fakeapp. You can execute it using the docker runcommand. CNB pushes the image to your local registry. Alternatively, there is --publishoption to send the generated image to a remote registry.
CNB provides good support for major languages like Java, .NET Core, NodeJS, Go, Ruby, Python. There are buildpacksfor diverse needs like SDK setup, Application packaging, configuring the environment, etc. These buildpackshave been published by enterprises like Heroku and Pivotal.
Configuring Policies and Enforcing Best Practices with CNB
Operations teams strive for standardization across their tools and techniques. These standardizations help in delivering reliability and security. They are often termed as a set of best practices followed across the organization. CNB can be used to enforce these practices for container builds by using the principle of policy-as-code.
CNB also helps in delivering application security. Some of the policies enforced by CNB are motivated by Docker CIS principles. Let's look at how to build policies for the most common use case.
All Applications Must Run on the Same OS Version
All application containers are built using a base image. CNB calls this base image the run imagefor the application. Additionally, CNB compiles the application code in a container, instead of compiling it locally on the workstation. This container is known as the build imagefor the application. These two images, the run imageand the build image, in a pair are known as CNB stack. A CNB stackis the foundation for all operations.
The above component diagram shows the components of the CNB container. It starts a Build Image and then applies the related buildpackslike Bellsoft Bulidpack, Maven Buildpack and Springboot Buildpack. Each of these buildpacks performs their operation, which may or may not be copied to the run image. CNB stack, build and run image pair, along with the associated buildpacksis also known as CNB builder.
A stackcan be enforced for the project by using the --buildercommand-line option. This would select the corresponding run image for generating the application container. CNB registry publishes builders which can be applied to your application.
pack build fakeapp --builder paketobuildpacks/builder:full
The above command will select Ubuntu Bionic for application build and ≈runtime. CNB registry contains Ubuntu and Alpine stacks, so organizations may not find a suitable CNB builder for their needs. In such cases, you can also create a custom stack that best fits.
All Applications Must Start as a Process User
The CNB stack specification enforces that all operations are performed as a configured user. All default builders use the cnb user and cnb group for performing operations. Custom CNB stack creators are enforced to configure the CNB_USER_IDand CNB_GROUP_IDvariables for their images.
Applications Must Run on Approved Runtime Versions
The pack command generated the application container automatically. It discovered that you have a Java application. All related buildpacks were configured automatically. CNB buildpack is responsible for executing a task. The task can compile your code, install dependencies, configure environment variables, etc.
A CNB builder has a default set of buildpacks. But every buildpack has a concept of auto-detection followed by execution. Thus only a subset of the default buildpacks are executed for a particular build.
CNB allows you to provide your own set of buildpacks. In such cases, it does not pick the superset of buildpacks available with the CNB builder. Thus there is fine control on the process of image creation. The buildpack override is accomplished by using the --buildpackcommand-line option. Alternatively, you can create a descriptor file in toml format, as shown below.
[[build.buildpacks]]
uri = "java-maven"
The above file contains which buildpacks apply to your project. The uriidentifies any of the existing build packs, or it can also define a custom build pack. The java-mavenis a custom buildpack that enforces Azul open JDK version.
As per CNB specification, a buildpack consists of two scripts. Each of these scripts is executed for the following two phases of the buildpack :
- detect: The buildpack auto-discovery script executed in Detect Phase
- bin: The buildpack execution script executed in Build Phase
The detect script for java-maven buildpack validated if the project contains pom.xml. On the other hand, the build script downloads Azul JDK 11 build and triggers a maven build. It also creates the launcher process configuration. You can build the project by using the above-created descriptor.
pack build fakeapp -d ../buildpacks/descriptor.toml --builder paketobuildpacks/builder:full
Applications Must Run on a Specific Port
As discussed previously, CNB can also alter the runtime environment by setting up environment variables. This behavior can be used to enforce various practices like application ports. Spring boot applications like fakeuser expose the application on 8080 port. But you can change the port by configuring the SERVER_PORTenvironment variable. The build variable BPE_OVERRIDE_SERVER_PORTaccomplished the same. It is picked by paketo-buildpacks/environment-variablesto configure the SERVER_PORT environment variable in the runtime container image.
[[build.env]]
name = "BPE_OVERRIDE_SERVER_PORT"
value = "8000"
[[build.buildpacks]]
uri = "java-maven"
[[build.buildpacks]]
uri = "paketo-buildpacks/environment-variables"
It is important to note that the buildpacks are executed in the order specified in the descriptor file.
Application Must Publish Logs in a Specified Format
Logging is another practice that is standardized by operation teams. Often application logs are ingested in a monitoring tool. The monitoring tool can build dashboards and alerts based on this information. Thus these logs must adhere to a particular format.
Application logging is often accomplished by various language-specific logging libraries. These libraries require a log configuration file. Logging for the fakeuser application can be controlled by adding a logback.xmlto the application classpath. You can create a custom buildpack to generate the required logback.xml. The buildpack would also set the required environment variables.
[[build.env]]
name = "BPE_OVERRIDE_SERVER_PORT"
value = "8000"
[[build.buildpacks]]
uri = "java-maven"
[[build.buildpacks]]
uri = "paketo-buildpacks/environment-variables"
[[build.buildpacks]]
uri = "log-format"
Conclusion
CNB offers a rich capability for building OCI images. The article only scratches the CNB surface. There are various layer caching practices, image inspection, and runtime rebasing, which are helpful in day-to-day operations.
CNB allows decoupling of application development from security and operational concerns. This way, developers can focus on writing code. On the other hand, DevOps teams can enforce their policies for container generation. The different phases provide the necessary support to customize policies for the varying needs of an enterprise.
