Kubernetes Service Catalog 101
Kubernetes uses the Service Catalog to connect to the external provider using a Service Broker. Find out more about how it works in this blog post.
What is a Service Catalog and why you may need to use it?
As a Kubernetes user/operator, you’ve dealt with a lot of resources to provision different components of your infrastructure. You’ve used resources like Services, configMaps, Secrets. But sometimes you may need to use an external service like the ones typically offered by the cloud provider. Take AWS for example; they provide the RDS service, which is an abstraction layer that lets you gain access to a relational database (MySQL, Postgres,etc.) as a service. When you want to integrate RDS into your existing Kubernetes cluster, you need to be able to deal with it the same way you deal with any other Kubernetes resource. Take authentication as an example. If you want to give your cluster applications access to the RDS database, you will need to do a lot of manual work (and workarounds) to make things work as expected.
To address this need, Kubernetes was extended to include the Kubernetes Service Catalog.
What is Kubernetes Service Catalog?
In a nutshell, the Kubernetes Service Catalog is an extension API that enables applications running inside the cluster to access applications and services provided by external sources, typically the cloud provider. Prominent examples of this pattern include provisioning databases, message queuing applications, object storage services, among others. Gaining systematic access to external resources is possible when the client consumes service brokers that implement the Open Service Broker API specification.
The Kubernetes Service Catalog Resources
You can think of the Kubernetes Service Catalog as another Kubernetes cluster running inside the main one. While Kubernetes has many types of resources (Pods, Deployments, ReplicaSets, etc.), the Service Catalog offers only four resource types. You can create and configure those resources by posting YAML/JSON files to the Service-Catalog’s API server. The same way you do with Kubernetes. The resources that Kubernetes Service Catalog use are:
- The cluster service broker (ClusterServiceBroker): it refers to the external system that provisions the service.
- The cluster service class (CluserServiceClass): this is the type of service that the external system can offer.
- A service instance (ServiceInstance): this represents an instance of the provisioned service.
- A service binding (ServiceBinding): which represents the actual binding between one or more pods and the ServiceInstance.
The Kubernetes Service Catalog Components
We mentioned before that the Service Catalog could be thought of as a particular Kubernetes instance running inside the main one, it’s just designed for a specific purpose. The Service Catalog is made up of the following components:
- The API server
- The etcd datastore
- The Controller Manager
The Controller Manager is where the controllers run. The controllers are continuously watching the API server for changes. When a change is detected, like a new resource was added and needs to be created, the controller does not carry out the creation process; the appropriate Service Broker instead does it. The service broker should be already registered using the ServiceBroker resource type. The following illustration depicts the typical communication between Pods running inside a Kubernetes cluster and a service hosted on an external system through the Service Catalog.
Creating a Sample Service Catalog
To best demonstrate the different settings and use cases for the Service Catalog, let’s create a sample one. In this lab, I’ll be using a locally-installed Kubernetes cluster. You can use your own or one that’s hosted on a cloud provider. Before we start, make sure that the following prerequisites are met:
Kubernetes version must be 1.9 or higher.
The cluster must have in-cluster DNS. Most Kubernetes installations provide this feature out of the box (for example, Minikube and most cloud providers).
The API server required etcd datastore v3.
RBAC needs to be enabled in the cluster. Most Kubernetes installations use RBAC by default. However, if you deliberately disabled it, the Service Catalog may not work. If you are an older version of Minikube (v0.25 or less), you will need to enable RBAC when the application starts explicitly.
Installing Service Catalog Helm Repo
The first thing you need to do is install Helm. Helm is a configuration and package manager for Kubernetes. You can read more about it in our article. The installation instructions are pretty straightforward and fit all popular operating systems.
Once installed, you will need to initialize it by running helm --init.
Now, you need to install the Service Catalog on your cluster using a Helm Chart:
helm repo add svc-cat https://svc-catalog-charts.storage.googleapis.com
The above command will add a Helm repository to your system called svc-cat. A Helm repository can be thought of like the Ubuntu apt repo or Centos YUM repo, just a link to a location that contains the resources that we need to download and use.
You may want to ensure that the Service Catalog Repository was successfully installed on your system by running:
$ helm search service-catalog NAME CHART VERSION APP VERSION DESCRIPTION svc-cat/catalog 0.2.1 service-catalog API server and controller-manager helm chart
Installing Service Catalog Helm Chart
Now that you have the Helm Repo installed, you can use it to actually install the necessary components for running Service Catalog by using the following command:
helm install svc-cat/catalog --name catalog --namespace catalog --wait
Installing svcat Command Line Tool
Similar to core Kubernetes, Service Catalog has a command-line tool just like kubectl. The tool is called svcat, and it makes communicating with the Service Catalog API server much easier for the operator. The tool can be installed on all major operating systems (Windows, Linux, and macOS). Svcat can also be installed as a kubectl plugin. To install svcat, follow the guide on https://svc-cat.io/docs/install/#installing-the-service-catalog-cli.
Once installed, you can use the tool by itself in standalone mode, or you can further install it as a kubectl plugin by running the following command:
./svcat install plugin
The above command assumes that you placed the svcat binary in your current working directory.
For a complete example of Service Catalog to work, we need a service that implements the Open Service Broker API specification for Kubernetes Service Catalog to interact with. Such a service is typically found at cloud providers. However, we can use a tool called minibroker that acts as a broker server. Minibroker can be locally installed on your laptop and lets you test Service Broker functionality.
Installing the server is as simple as adding a Helm repo and installing the appropriate Helm chart. This can be done through the following commands:
helm repo add minibroker https://minibroker.blob.core.windows.net/charts helm install --name minibroker --namespace minibroker minibroker/minibroker
The first command adds the minibroker repository to your system. The second one actually installs the components necessary for the minibroker server to run.
Viewing the Classes and Plans for the Service Broker
Our Service Broker is up and running now. We can see view the different classes that ths broker offers (ClusterServiceClass). We can do this using the svcat command-line tool that we installed earlier:
$ svcat get classes NAME NAMESPACE DESCRIPTION +------------+-----------+---------------------------+ mariadb Helm Chart for mariadb mongodb Helm Chart for mongodb mysql Helm Chart for mysql postgresql Helm Chart for postgresql redis Helm Chart for redis
As you can see, our service broker has classes for interacting with Mariadb, MongoDB, MySQL, PostgreSQL, and Redis. Those services can be accessed the same way you access any Kubernetes resources. The difference, however, is that you don’t have to install or manage them as this is done by the provider.
Notice that you could run the previous command without the need of using the standalone svcat, you could use kubectl as follows:
$ kubectl get clusterserviceclasses NAME EXTERNAL-NAME BROKER AGE mariadb mariadb minibroker 13h mongodb mongodb minibroker 13h mysql mysql minibroker 13h postgresql postgresql minibroker 13h redis redis minibroker 13h
The output may have a different set of column names, but the list remains similar.
As mentioned before, each service that the broker provides can also have a set of plans, each offering a different quality level. Cloud providers may charge you more for using premium plans and less for standard ones, for example. Let’s have a look at the sample plans that the Redis service offers:
$ svcat describe class redis Name: redis Scope: cluster Description: Helm Chart for redis Kubernetes Name: redis Status: Active Tags: redis, keyvalue, database Broker: minibroker Plans: NAME DESCRIPTION +-----------------+--------------------------------+ 3-2-9 Open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. 4-0-10 Open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. *** the rest of the output is skipped ******
By using the described subcommand for svcat, we can learn more about what levels the service provides.
You can also use kubectl to gain more information about the ClusterServiceClass as follows:
$ kubectl get clusterserviceclasses redis -o yaml apiVersion: servicecatalog.k8s.io/v1beta1 kind: ClusterServiceClass metadata: creationTimestamp: "2019-08-01T22:55:41Z" name: redis ownerReferences: - apiVersion: servicecatalog.k8s.io/v1beta1 blockOwnerDeletion: false controller: true kind: ClusterServiceBroker name: minibroker uid: 39d34acc-b4af-11e9-8053-ca3f32181f91 resourceVersion: "7" selfLink: /apis/servicecatalog.k8s.io/v1beta1/clusterserviceclasses/redis uid: 7ca27606-b4af-11e9-8053-ca3f32181f91 spec: bindable: true bindingRetrievable: false clusterServiceBrokerName: minibroker description: Helm Chart for redis externalID: redis externalName: redis planUpdatable: false tags: - redis - keyvalue - database status: removedFromBrokerCatalog: false
If you want to view all the plans provided by the broker, you can use the following svcat command:
svcat get plans
If you prefer using kubectl, you can get the same information using the following command:
kubectl get clusterserviceplans
Now, let’s say that you need to know the specifics of a particular plan. You can view the plan properties by describing it. Let’s describe the Redis plan 3-2-9:
$ svcat describe plan 3-2-9 --scope cluster Name: 3-2-9 Description: Open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. Kubernetes Name: redis-3-2-9 Status: Active Free: true Class: redis Instances: No instances defined
Again, you can use kubectl for querying the plan information:
$ kubectl get clusterserviceplans redis-3-2-9 -o yaml [16:00:39] apiVersion: servicecatalog.k8s.io/v1beta1 kind: ClusterServicePlan metadata: creationTimestamp: "2019-08-01T22:55:43Z" name: redis-3-2-9 ownerReferences: - apiVersion: servicecatalog.k8s.io/v1beta1 blockOwnerDeletion: false controller: true kind: ClusterServiceBroker name: minibroker uid: 39d34acc-b4af-11e9-8053-ca3f32181f91 resourceVersion: "22" selfLink: /apis/servicecatalog.k8s.io/v1beta1/clusterserviceplans/redis-3-2-9 uid: 7da54720-b4af-11e9-8053-ca3f32181f91 spec: clusterServiceBrokerName: minibroker clusterServiceClassRef: name: redis description: Open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. externalID: redis-3-2-9 externalName: 3-2-9 free: true status: removedFromBrokerCatalog: false
However, notice that when using kubectl, we had to prepend the plan name with the service name. So, we don’t use 3-2-9, we must use redis-3-2-9.
Using the Service Broker services
Our infrastructure is now ready to start consuming the services our broker server provides, In our example, we will use Redis. The first thing we need to do is create a ServiceInstance resource.
Creating a ServiceInstance Resource
The notable thing about the ServiceInstance resource is that it mysg belong to a namespace. So, the first thing we need to do is create a namespace:
kubectl create namespace redis-ns
The next thing we need to do is create the definition file for the ServiceInstance resource. It may look something like the following:
apiVersion: servicecatalog.k8s.io/v1beta1 kind: ServiceInstance metadata: name: mini-instance namespace: redis-ns spec: clusterServiceClassExternalName: redis clusterServicePlanExternalName: 5-0-4
Let’s apply this configuration by running kubectl as follows:
$ kubectl apply -f serviceinstance.yaml serviceinstance.servicecatalog.k8s.io/mini-instance created
Once the above command is issued, the controller in the service catalog will start communicating with the broker server to start deploying the service. You can check the status of this operation using the following command:
$ svcat describe instance -n redis-ns mini-instance Name: mini-instance Namespace: redis-ns Status: Ready - The instance was provisioned successfully @ 2019-08-02 14:15:29 +0000 UTC Class: redis Plan: 5-0-4 Parameters: name: value Bindings: No bindings defined
Creating the ServiceBinding
So, we have the ClusterServiceBroker, the ServiceClass and the ServiceInstance. The only thing left to actually start consuming the service is to create ServiceBinding. Create the definition file as follows:
apiVersion: servicecatalog.k8s.io/v1beta1 kind: ServiceBinding metadata: name: mini-binding namespace: redis-ns spec: instanceRef: name: mini-instance
Apply the above definition through kubectl as usual:
$ kubectl apply -f servicebinding.yaml servicebinding.servicecatalog.k8s.io/mini-binding created
By applying this definition, the Service Broker controller will start creating the necessary binding to the broker service. If the service needs credentials, the broker server will issue the necessary credentials that the controller will subsequently insert it into a Secret.
Let’s double check that the binding was created:
$ svcat describe binding mini-binding -n redis-ns Name: mini-binding Namespace: redis-ns Status: Ready - Injected bind result @ 2019-08-02 15:59:12 +0000 UTC Secret: mini-binding Instance: mini-instance Parameters: No parameters defined Secret Data: Protocol 5 bytes host 51 bytes password 10 bytes port 4 bytes redis-password 10 bytes uri 76 bytes
You can also get information about the binding using the following command:
kubectl get servicebindings -n test-ns mini-binding -o yaml
The binding is ready. There are also some sensitive information that were created for us, let’s see:
$ kubectl get secrets -n redis-ns NAME TYPE DATA AGE default-token-wgskl kubernetes.io/service-account-token 3 42m mini-binding Opaque 6 28m sweet-whale-redis Opaque 1 28m
Using the Service Catalog Service
How about giving our Redis server a test drive? The first thing we need to do is determine the connection information: host, port, user, and password. All this information is stored in a Secret. Let’s decode its data:
$ kubectl -n redis-ns get secret mini-binding -o yaml apiVersion: v1 data: Protocol: cmVkaXM= host: c3dlZXQtd2hhbGUtcmVkaXMtbWFzdGVyLnJlZGlzLW5zLnN2Yy5jbHVzdGVyLmxvY2Fs password: VGdFOEhxNVNndw== port: NjM3OQ== redis-password: VGdFOEhxNVNndw== uri: cmVkaXM6Ly86VGdFOEhxNVNnd0Bzd2VldC13aGFsZS1yZWRpcy1tYXN0ZXIucmVkaXMtbnMuc 3ZjLmNsdXN0ZXIubG9jYWw6NjM3OQ== kind: Secret metadata: creationTimestamp: "2019-08-02T15:59:12Z" name: mini-binding namespace: redis-ns ownerReferences: - apiVersion: servicecatalog.k8s.io/v1beta1 blockOwnerDeletion: true controller: true kind: ServiceBinding name: mini-binding uid: 7817f2e6-b53e-11e9-a80e-92da974e749e resourceVersion: "70092" selfLink: /api/v1/namespaces/redis-ns/secrets/mini-binding uid: 788612d3-b53e-11e9-8076-025000000001 type: Opaque
OK, let’s start with the host:
$ base64 -D <<< "cmVkaXM6Ly86VGdFOEhxNVNnd0Bzd2VldC13aGFsZS1yZWRpcy1tYXN0ZXIucmVkaXMtbnMuc3Z jLmNsdXN0ZXIubG9jYWw6NjM3OQ==" redis://:TgE8Hq5Sgw@sweet-whale-redis-master.redis-ns.svc.cluster.local:6379
So, fortunately the URL contains all the information that we need:
By examining the URL, we can get the credentials and the host as follows:
Password: TgE8Hq5Sgw Host: sweet-whale-redis-master.redis-ns.svc.cluster.local Port: 6379
Connecting to the Redis instance now is as simple as spinning another Pod that has a container offering Redis client and establishing a connection to the service:
kubectl run -n redis-ns -i --tty redisclient --image=redis --restart=Never -- bash -il Port: 6379
The above command will spin out a Pod hosting a Redis container. We chose the Redis image because it already contains the redis-cli client so we don’t need to install it.
Now, let’s try to connect to our Redis service:
root@ubuntu:/data# redis-cli -h sweet-whale-redis-master.redis-ns.svc.cluster.local sweet-whale-redis-master.redis-ns.svc.cluster.local:6379> auth TgE8Hq5Sgw OK
So, using the hostname and password provided to us through the Secret, we were able to establish a successful connection to Redis. The Redis service was provided to us from outside the Kubernetes cluster, but through the Service Broker we were able to connect to it the same way we connect to any other Kubernetes-native service.
Our lab is done. Let’s delete the components that we created. First, remove the ServiceBinding:
$ svcat unbind -n redis-ns mini-instance deleted mini-binding
Notice that when we deleted the binding, the associated Secret was deleted as well:
$ kubectl -n redis-ns get secrets mini-binding Error from server (NotFound): secrets "mini-binding" not found
The next thing we need to address is the ServiceInstance, let’s remove it:
$ svcat deprovision -n redis-ns mini-instance deleted mini-instance
Next, the ServiceBroker, which can be deleted as follows:
$ kubectl delete clusterservicebrokers minibroker clusterservicebroker.servicecatalog.k8s.io "minibroker" deleted
Finally, we remove the the Service Broker Chart:
$ helm delete --purge minibroker release "minibroker" deleted
And the associate namespaces:
$ kubectl delete namespace redis-ns minibroker namespace "redis-ns" deleted namespace "minibroker" deleted
And the Service Catalog:
$ helm delete --purge catalog kubectl delete ns svc-cat $ kubectl delete ns catalog namespace "catalog" deleted
- Kubernetes Service Catalog is an extension to the API that allows external providers to provision their Services natively to Kubernetes.
- Kubernetes uses the Service Catalog to connect to the external provider using a Service Broker. The Broker acts as an interface between the Service Catalog and the internals of the provided service. The Broker must abide by the Open Service Broker API specification.
- The Service Catalog can be managed and accessed by kubectl like any other Kubernetes resource. However, there is a command-line tool designed specifically to communicate with the Service Catalog called svcat.
- Throughout the article, we created a simple lab in which we could connect to a Redis service provisioned behind a Service Broker. We used minibroker as the Service Broker.