Kubernetes Patterns: The Service Discovery Pattern
Why do we need the service discovery pattern? And what are some scenarios in which service discovery would be utilized?
Why Do We Need Service Discovery?
Kubernetes deploys applications through Pods. Pods can be placed on different hosts (nodes), scaled up by increasing their number, scaled down by killing the excess ones, and moved from one node to another. All those dynamic actions must occur while ensuring that the application remains reachable at all times. To address this critical requirement, we use the Service Discovery pattern. To understand service discovery, let’s briefly visit the Service Discovery Architecture (SOA).
In a microservices design, the application is broken down into a number of components, where each component is responsible for a specific role. Different components must communicate with each other using network protocols (for example, HTTP). So, imagine that a client component (consumer) needs to send an HTTP message to service (producer). If there’s only one instance of the producer, the solution is straightforward: place the producer connection information (like the IP address or DNS name, the protocol, and the port) in a configuration file for the client to use. However, to achieve high availability, there’s often more than one instance of a particular service available. In this case, the client cannot just hook itself to one of the producers. If this service instance was deleted or renamed, the client loses connectivity and part of the application breaks. Hence, the client needs some way to discover which service instance is healthy and available to that it can connect to it. So, back to SOA, there are two ways for a client to discover services:
Client-side discovery: in this mode, the client is responsible for determining which service instance it should connect to. It does that by contacting a service registry component, which keeps records of all the running services and their endpoints. When a new service gets added or another one dies, the Service Registry is automatically updated. It is the client’s responsibility to load-balance and distribute its request load on the available services.
Server-side discovery: in this mode, a load-balancing layer exists in front of the service instances. The client connects to the well-defined URL of the load balancer and the latter determines which backend service it shall route the request too.
In Kubernetes, the Service component is used to provide a static URL through which a client can consume. The Service component is Kubernetes's way of handling more than one connectivity scenario.
Scenario 01: Inter-App Communication
In this scenario, you have a number of Pods hosting part of the application (for example, authentication). You need other parts of the application to be able to connect to and use the authentication component. The following definition creates a ReplicaSet and a Service to address this need:
---
apiVersion: v1
kind: Service
metadata:
name: openid-svc
spec:
selector:
app: openid
ports:
- port: 80
targetPort: 9000
protocol: TCP
---
apiVersion: apps/v1
kind: ReplicaSet
metadata:
name: openid
labels:
app: openid
spec:
replicas: 3
selector:
matchLabels:
app: openid
template:
metadata:
labels:
app: openid
spec:
containers:
- name: oidc
image: qlik/simple-oidc-provider
The file is divided into two portions (in YAML you can combine more than one file using three dashes: ---). The first part defines the Service that responds to client requests. Any client that needs to send an authentication request should talk to openid-svc or - through its FQDN - openid-svc.default.svc.cluster.local.
The second part of the file defines the ReplicaSet that actually spawns the Pods which run the authentication image. In this example, we’re using the simple-oidc-provider image by qlik.
There is a number of important notices to make about this model:
- The Service knows which Pods it should route requests to by selecting their labels (lines 6-8).
- By default, the container listens on port 9000. However, you can instruct the Service to listen on a different port (80 in our example) and route the requests to the appropriate port behind the scenes (lines 9-11).
- Since this is an internal Service (nobody outside the cluster should access it), it gets assigned an IP address that’s internal to the cluster.
Let’s try this configuration by spinning out a quick Ubuntu Pod, install curl and make an HTTP request to our service:
$ kubectl run -i --tty ubuntu --image=ubuntu:18.04 --restart=Never -- bash -il
If you don't see a command prompt, try pressing enter.
root@ubuntu:/# apt update && apt install curl -y
-- output removed for brevity --
root@ubuntu:/# curl openid-svc
{"error":"invalid_request","error_description":"unrecognized route"}
So, we have the expected JSON object from one of the openidc Pods. However, our curl client didn’t know anything about which Pod returned the response, nor did it know how many Pods are backing our Service.
Notice that when connecting to our Service, we used the DNS method. Kubernetes deploys a DNS server that automatically adds an entry for new Services. Hence, we were able to communicate with our Service using its DNS name rather than it’s clusterIP address.
However, there is another way of discovering what our Service offers through environment variables. When the Service is created, any newly spawned Pods with matching labels automatically have environment variables corresponding to the Service connection details. Let’s see:
root@ubuntu:/# env | grep OPENID
OPENID_SVC_SERVICE_PORT=80
OPENID_SVC_PORT_80_TCP_PORT=80
OPENID_SVC_PORT_80_TCP_PROTO=tcp
OPENID_SVC_PORT=tcp://10.111.58.252:80
OPENID_SVC_PORT_80_TCP=tcp://10.111.58.252:80
OPENID_SVC_SERVICE_HOST=10.111.58.252
OPENID_SVC_PORT_80_TCP_ADDR=10.111.58.252
We used the env command on our temp Ubuntu container to get the environment variables that have the string OPENID. We have different Service details relayed to us like the Service IP, port and protocol.
The drawback of using environment variables when connecting to the Service is that must be created before the Pods are. So, if you create the Service after the client Pods are already running, you cannot use environment variables as they cannot be injected into a running Pod.
Services are very versatile. They can be your one-stop shop for both load-balancing and service registry:
- They provide multiple ports: for example, you can accept connections on ports 80 for unencrypted traffic and 443 for SSL-backed communication. Both exposed by the same Service.
- They support internal session affinity: using .spec.sessionAffinity: ClientIP, the Service ensures that traffic coming from a certain Pod’s IP address gets always routed to the same target Pod instead of selecting a random one. However, notice that session affinity here works on network layer 4; it cannot use HTTP cookies for example. If you need that level of affinity, you should consider using Ingress.
- Advanced health probes: any load balancer must check that the backing nodes are healthy and can respond to requests so that it can route requests to them. Services make use of health and readiness probes offered by Kubernetes to ensure that not only the Pods are in the running state, but also they return the expected response.
- You can even manually select the IP address that that Service exposes by altering the spec.ClusterIP parameter. This ability can prove to be very useful when you have a legacy application that was configured to connect to a specific IP address.
Scenario 02: Connecting To an Outside Resource
By default, a Kubernetes Service works by dynamically tracking the endpoints that were created for the matching Pods. For example, when you configure the Service resource to track Pods labeled app=web, then any healthy Pod with that label is likely to receive traffic from the Service. However, sometimes you may need your Pods to connect to an external resource. For example, you use a third-party API service that is hosted outside your cluster. Yet, you still need to use a Service interface. In this case, you can create a Service that does not have a selector and add the Endpoints manually. Let’s have an example:
---
apiVersion: v1
kind: Service
metadata:
name: external-ip
spec:
ports:
- port: 80
protocol: TCP
The above definition is very similar to our first Service example, except that we removed the selector part. Now, the Service needs its endpoints to be able to work. We can create the endpoints using a definition like the following:
apiVersion: v1
kind: Endpoints
metadata:
name: external-api
subsets:
- addresses:
- ip: 123.123.123.123
- ip: 134.134.134.134
ports:
- port: 80The most important thing to notice here is that the name of the Endpoints resource must match the Service name. Otherwise, there is no way to link a Service with its Endpoints.
Why Use a Service to Connect To an External Resource?
You might be wondering why it is wise to add a Service layer between the Pods inside your cluster and an external resource. Why not just supply the DNS name or the IP address of the external resource to the Pod configuration? For the following reasons:
- A Service can have more than one IP endpoint. So, if the external API exposes more than one IP address for high availability, you can use a Service to load balance among them.
- A Service in this sense acts as an abstraction layer between the Pods and their connection target. You can make changes to the targets without affecting or changing the client Pods configuration. For example, if the IP address of the remote server changes, you only need to make this change in one place. You may even manage to move that external service to be handled on one or more Pods inside your cluster. Having a Service in place from the start allows you to add the necessary selectors to route traffic to the new Pods. All those changes need to be made only to the Service. No Pod configuration change is needed.
It is worth noting that there is a second way of creating a Service that connects to external targets. In this method, we only supply the DNS name of the external resource. We don’t need to create Endpoints manually. However, this type of Service (called ExternalName) only creates a CNAME for the DNS record rather than proxying the connection. For example, the following definition creates a CNAME for api.example.com:
apiVersion: v1
kind: Service
metadata:
name: externalname
spec:
type: ExternalName
externalName: api.example.com
ports:
- port: 80
Scenario 03: Accepting Outside Connections
More often than not, you need to accept connections from outside the cluster. If you are hosting a web application, for example, you need your clients to be able to consume your service from their own computers. There are two ways you can expose your Pods to the outside world through a Service:
Using NodePort:
In this method, you use the external IP address of any of the cluster nodes combined with a specific port. This port is reserved on all the nodes. You can manually select an available port (must be available on all the nodes) or let Kubernetes choose a random one for you (by omitting the nodePort parameter). Any traffic that arrives at one of the nodes on the assigned port gets automatically routed by the Service to the appropriate Pod. For example, we can expose our authentication service to the world using NodePort Service that can be created using the following definition:
apiVersion: v1
kind: Service
metadata:
name: oidc-svc
spec:
type: NodePort
selector:
app: openid
ports:
- port: 80
targetPort: 9000
nodePort: 30030
protocol: TCP
The following considerations should be in your mind when using the NodePort method:
- Since you are using a local node port, you may need to configure the necessary network and firewall rules to allow external traffic. In complex infrastructures, this may be challenging.
- The client application must be made aware of all the nodes in the cluster so that if a node is down, the client knows that it can connect to the Service through another healthy node. This adds an extra layer of overhead as all the client services must change their configuration. A possible workaround for this drawback is placing a load balancer in front of the nodes to automatically route traffic to healthy nodes.
- When traffic arrives at a random node, it may not necessarily be the one where the target Pod is deployed. The Service handles routing traffic from the node where the traffic arrived at the one where the target Pod exists. However, this creates an unnecessary network hop that may cause some latency. One possible workaround for this is to ensure that all the Service Pods exist on all the nodes by deploying them through DaemonSets. The other workaround is to use .spec.externalTrafficPolicy: Local to the Service definition. This parameter ensures that no traffic is accepted on a node except if the target Pod is running on that same node. Again, this carries the burden having to configure the client entities as to which node runs which Pods. Such an approach is not encouraged as it tightly-couples clients with their target Pods, circumventing the very reason why Services exist.
- Additionally, if a node received traffic intended for a Pod running on a different node, the packets’ source IP gets changed to be the same as the node’s IP. Now, when the destination node receives the packet, it is as if it were originating from the other cluster node.
Using LoadBalancer:
To address the drawbacks of the NodePort method, the Kubernetes Service offers the LoadBalancer method. Using the LoadBalancer type, the Service automatically starts and configures a load balancer that distributes traffic among the nodes.
To use this type of Service you must have your infrastructure hosted on a cloud provider that supports Load Balancers and supports Kubernetes. Since the load balancer component is provisioned and managed by the cloud provider, the configuration specifics vary from one provider to the other. For example, you may or may not be allowed to choose the external IP address that the Load Balancer exposes. Also, some providers preserve the source IP address of the request and some others replace it with the IP address of the load balancer. The following definition demonstrates how you can create a Service of type load balancer:
---
apiVersion: v1
kind: Service
metadata:
name: openid-svc
spec:
type: LoadBalancer
clusterIP: 10.0.150.240
loadBalancerIP: 80.15.25.20 # Depends on whether the cloud provider allows it
selector:
app: openid
ports:
- port: 80
targetPort: 9000
protocol: TCP
The only part that changed here is the type, which we set to LoadBalancer. In this mode, the Service exposes an internal IP (clusterIP) that is used inside the cluster. It also exposes the external IP through the Load Balancer to accept external traffic.
Scenario 04: I Don’t Want Load-Balancing. (The Headless Service)
Providing a unified URL (or IP address) that the client Pod uses to connect to other Pods is the main reason why the Service resource exists. Traditionally, the client does not care where the response is coming from as long as it is the expected one. However, in stateful applications, this is not the case. In a stateful app, the client is interested in contacting a specific Pod (for example, the master node of a cluster, ZooKeeper’s leader node, etc.). Stateful apps in Kubernetes are handled through StatefulSets. In that case, the Service should not load-balance across the Pods. But, if a Service does not distribute traffic, what is it good for? Well, we still need the Service component to update the list of endpoints of the Pods it matches. A Service of that type is called “Headless Service”. A Headless Service definition sets the clusterIP parameter to none, effectively denying the Service from exposing an IP address. When the DNS name of the Service is queried, the Service does not return an IP. Instead, it returns a list of the Pod endpoints that it manages. It is the client’s responsibility to select the Pod that it needs to connect to. Notice that, currently, you are required to create a Headless Service if you want to create a StatefulSet. The following definition demonstrates a Headless Service:
---
apiVersion: v1
kind: Service
metadata:
name: openid-svc
spec:
selector:
app: openid
clusterIP: none
ports:
- port: 80
targetPort: 9000
protocol: TCP
Scenario 05: Serving Multiple Services (The Ingress Controller)
If your application needs only one entry point for your clients to start using it, you can go just fine using a Service of type Load Balancer. However, many applications today expose more than one endpoint. For example, we may have www.example.com which features what our company does, some testimonials, or latest offers, etc. But we may also have api.example.com for people who need programmatic access to our services, app.example.com for the GUI version of the app, blog.example.com for a community portal and so on. Inside the Kubernetes cluster, each of www, API, app, and blog would have a separate service with a number of Pods. Now, if you need to expose a Service you can use the Load Balancer service type. This means we’ll be costing ourselves four load balancers. A better solution is to have one main gateway for all your application Services that knows which URL request should be routed to which backend Service. We need an Ingress resource.
Ingress is Not Another Service Type
Ingress is a separate Kubernetes resource with its own definition and properties. To run it on your cluster you must have the Ingress Controller running first. For instructions on how to deploy Ingress Controller to your cluster, refer to this document: https://kubernetes.github.io/ingress-nginx/deploy/
Ingress can act as the main entry point for your application. It sits in front of your Services and routes traffic to them. But it’s not just a load balancer. Among the things that Ingress can do:
- SSL termination (you can install one certificate on the Ingress where SSL connection is terminated)
- Advanced routing and rewrite rules. Ingress is, in fact, a reverse proxy. You can write custom rules for which URL gets routed to which Service.
The following definition demonstrates how a single Ingress IP address can be used to route traffic to different backend Services:
apiVersion: extensions/v1beta
kind: Ingress
metadata:
name: myoid
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: openid-svc
servicePort: 9000
- path: /health
backend:
serviceName: health-svc
servicePort: 80
The most important lines to note here are:
- .metadata.annotations (Lines 5 and 6): Ingress is implemented through an Ingress Controller. The controller may require additional configuration settings that are passed through the annotations. In our example, we didn’t do much of a rewrite. If you want to see more complex rewrite examples, you can have a look at this document https://kubernetes.github.io/ingress-nginx/examples/rewrite/.
- The spec part contains a list of rules, each rule consists of a path (that’s where the request is captured), and a backend. The backend specifies which backend Service should respond to this URL request and which port it is listening on.
In practice, the above definition assigns an external IP address that you can associate with your DNS record, for example, oid.example.com. Now, whenever users hit app.example.com they’ll receive a response from the openid-svc Service. When they hit app.example.com/health, they are actually talking to the health-svc Service, which may respond with a JSON object with the application’s health status.
This way you can decouple your Services’ implementation from external exposure. The Service definition should focus only on how to route traffic to the Pods, how to select the correct ones, and on which ports and leave the external access and routing rules on the Ingress.
TL;DR
Service discovery is one of the core concepts in Kubernetes. It is the glue that connects different components with each other. The Service resource originally existed to address the need of communicating with dynamically-changing Pods through a unified, stable interface. However, as applications became more complex, Services were used for more than just that. In the following table, we summarize to choose the correct Service configuration for your scenario:
*The outline of this article outline is inspired by the book of Roland Huss and Bilgin Ibryam : Kubernetes Patterns.
