Part 4: Secure Container Firewalls

This example uses Digital Ocean, but you can just as easily create three instances in AWS, Google Cloud Platform or Microsoft Azure or any other cloud provider.

1. Create Three Droplets in Digital Ocean

Sign up or log into Digital Ocean and create three Ubuntu instances with the following specifications:

• Ubunutu 16.04
• 4GB or more of RAM per instance

Note: do not select the Private networking option for your droplets. Selecting this option will prevent the setting up of the Kubernetes cluster to fail. See section Initialize the Master for more details.

2. Add a New Weave Cloud Instance

Sign up or log into Weave Cloud.

Create a new instance or rename the default instance in Weave Cloud](https://cloud.weave.works). Weave Cloud instances are the primary workspaces for your application and provides a view onto your cluster and the application that is running on it.

3. Set up a Kubernetes Cluster with kubeadm

Kubeadm is by far the simplest way to set up a Kubernetes cluster. With only a few commands, you can deploy a complete Kubernetes cluster with a resilient and secure container network onto the Cloud Provider of your choice in only a few minutes.

kubeadm is a command line tool that and is a part of Kubernetes 1.4 and greater.

See the kubeadm reference for information on all kubeadm command-line options and for advice on automating kubeadm.

Objectives

• Install a secure Kubernetes cluster
• Install Weave Net as a pod network so that application components (pods) can communicate with one another
• Install the Sock Shop, a demo microservices application
• View the result in Weave Cloud

4. Download and install kubelet, kubeadm and Docker

To begin SSH into the machine and become root (for example, run sudo su -). Then install the required binaries onto all three instances:

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF > /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update

Next, install Docker. You can also use the official Docker packages instead of docker.io that is referenced here.

apt-get install -y docker.io

And finally, install the Kubernetes packages:

apt-get install -y kubelet kubeadm kubectl kubernetes-cni

5. Initialize the Master

Note: Before making one of your machines a master, kubelet and kubeadm must have been installed onto each of the nodes.

The master is the machine where the “control plane” components run, including etcd (the cluster database) and the API server (which the kubectl CLI communicates with).

All of these components run in pods started by kubelet.

Keep in mind that you can’t run kubeadm init twice without tearing down the cluster, see Tear Down for more information.

To initialize the master, pick one of the machines on which you previously installed kubelet and kubeadm and run:

kubeadm init

Initialization of the master may take a few minutes.

This autodetects the network interface and then advertises the master on it with the default gateway.

Note: If you want to use a different network interface, specify it with the --api-advertise-addresses=<ip-address> flag when you run kubeadm init.

Important! Special note on selecting different network interface:

When using a different network interface in Digital Ocean through the Private Network option for the droplets causes the Kubernetes cluster set up to fail. For more information, see issue #203 of kubernetes/kubeadm. This can occur with kubeadm and the default gateway that your droplets may receive at the moment of creation.

Until this issue is resolved, the default droplet networking settings must be enabled. This means that all the nodes in your cluster will be open to the world and that they communicate between each other via Internet. Ensure that you understand the implications of such a set up. You can reinforce the security of your cluster by using ufw or iptables rules.

Refer to the kubeadm reference doc to read up on the flags kubeadm init provides.

If the initialization is successful, the output should look similar to the following:

....some preflight checks and initialization

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run (as a regular user):

  sudo cp /etc/kubernetes/admin.conf $HOME/
  sudo chown $(id -u):$(id -g) $HOME/admin.conf
  export KUBECONFIG=$HOME/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  http://kubernetes.io/docs/admin/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join --token <token-id> <master-ip>

Make a record of the kubeadm join command that kubeadm init outputs. You will need this once it’s time to join the nodes. This token is used for mutual authentication between the master and any joining nodes.

This token is a secret, and so it’s important to keep it safe — anyone with this key can add authenticated nodes to your cluster.

(Optional) Scheduling Pods on the Master

By default, the cluster does not schedule pods on the master for security reasons. If you want to be able to schedule pods on the master, for example if you want a single-machine Kubernetes cluster for development, then run:

kubectl taint nodes --all dedicated-

The output will be:

node "test-01" tainted
taint key="dedicated" and effect="" not found.
taint key="dedicated" and effect="" not found.

This removes the “dedicated” taint from any nodes that have it, including the master node, meaning that the scheduler will then be able to schedule pods everywhere.

6. Set up the environment for Kubernetes

On the master run the following as a regular user:

  sudo cp /etc/kubernetes/admin.conf $HOME/
  sudo chown $(id -u):$(id -g) $HOME/admin.conf
  export KUBECONFIG=$HOME/admin.conf

7. Install Weave Net as the Pod Networking Layer

In this section, you will install a Weave Net pod network so that your pods can communicate with each other.

You must add Weave Net before deploying any applications to your cluster and before kube-dns starts up.

Note: Install only one pod network per cluster. There are two versions of the Weave Net daemonset installer. One installs Weave Net to version 1.5 of the Kubernetes binaries and the other installs to 1.6.

If you’re running Kubernetes 1.5 (and less) install Weave Net by logging onto the master and running:

kubectl apply -f https://git.io/weave-kube

If you’re running Kubernetes 1.6 (and above), install Weave Net by logging onto the master and running:

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

The output will be:

serviceaccount "weave-net" created
clusterrole "weave-net" created
clusterrolebinding "weave-net" created
daemonset "weave-net" created

Once a pod network is installed, confirm that it is working by ensuring that the kube-dns pod is running:

kubectl get pods --all-namespaces

Once the kube-dns pod is up and running, you can join all of the nodes to form the cluster.

8. Join Your Nodes to the Master

The nodes are where the workloads (containers and pods, etc) run.

Join the nodes to your cluster by running:

kubeadm join --token <token> <master-ip>

The above command, including the token and the master-ip, is output by kubeadm init that you ran earlier.

When the node has successfully joined, the output should look as follows:

preflight] Running pre-flight checks
[tokens] Validating provided token
[discovery] Created cluster info discovery client, requesting info from "http://138.197.150.135:9898/cluster-info/v1/?token-id=ad23e7"
[discovery] Cluster info object received, verifying signature using given token
[discovery] Cluster info signature and contents are valid, will use API endpoints [https://138.197.150.135:6443]
[bootstrap] Trying to connect to endpoint https://138.197.150.135:6443
[bootstrap] Detected server version: v1.6.0
[bootstrap] Successfully established connection with endpoint "https://138.197.150.135:6443"
[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request
[csr] Received signed certificate from the API server:
Issuer: CN=kubernetes | Subject: CN=system:node:node-02 | CA: false
Not before: 2017-02-20 20:33:00 +0000 UTC Not After: 2018-02-20 20:33:00 +0000 UTC
[csr] Generating kubelet configuration
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"

Node join complete:
* Certificate signing request sent to master and response
  received.
* Kubelet informed of new secure connection details.

Run 'kubectl get nodes' on the master to see this machine join.

Run kubectl get nodes on the master to display a cluster with the number of machines as you created.

(Optional) Control Your Cluster From Machines Other Than The Master

In order to get kubectl on your laptop to talk to your cluster (as an example), copy the kubeconfig file from your master to your laptop:

scp root@<master ip>:/etc/kubernetes/admin.conf .
kubectl --kubeconfig ./admin.conf get nodes

8. Install and Launch the Weave Cloud Agents

The yaml file listed below installs all of the Weave Cloud probes to a DaemonSet and launches them to your cluster.

From the master:

kubectl apply -f \
   "https://cloud.weave.works/k8s.yaml?t=<cloud-token>&k8s-version=$(kubectl version | base64 | tr -d '\n')"

The <cloud-token> is found in the settings dialog on Weave Cloud. The above command may also be cut and pasted from the setup screens in Weave Cloud.

If you mistyped or copied and pasted the command incorrectly, you can remove the DaemonSet with:

kubectl delete ns weave

Return to Weave Cloud, and click Explore and then Pods to display the Kubernetes cluster in your instance. Ensure that the All Namespaces filter is enabled from the left-hand corner.

For the next few steps, keep the instance open on Explore to watch the Sock Shop containers spin up in the cluster.

9. Install the Sock Shop onto Kubernetes

To put your cluster through its paces, install the sample microservices application, Socks Shop. Learn more about the sample microservices app by referring to the microservices-demo README.

To install the Sock Shop, run the following:

kubectl create namespace sock-shop
git clone https://github.com/microservices-demo/microservices-demo
cd microservices-demo
kubectl apply -n sock-shop -f deploy/kubernetes/manifests

Click on Explore and then Pod and enable the sock-shop namespace filter from the bottom left-hand corner.

It takes several minutes to download and start all of the containers. Watch the output of kubectl get pods -n sock-shop to see that all of the containers are successfully running.

Or view the containers as they get created in Weave Cloud.

10. View the Sock Shop in Your Browser

Find the port that the cluster allocated for the front-end service by running:

kubectl describe svc front-end -n sock-shop

The output should look like:

Name:                   front-end
Namespace:              sock-shop
Labels:                 name=front-end
Selector:               name=front-end
Type:                   NodePort
IP:                     100.66.88.176
Port:                   <unset> 80/TCP
NodePort:               <unset> 31869/TCP
Endpoints:              <none>
Session Affinity:       None

Launch the Sock Shop in your browser by going to the IP address of any of your node machines in your browser, and by specifying the NodePort. So for example, http://<master_ip>:<pNodePort>. You can find the IP address of the machines in the DigitalOcean dashboard.

In the example above, the NodePort was 31869.

If there is a firewall, make sure it exposes this port to the internet before you try to access it.

11. Create a Load on the Sock Shop

To fully appreciate the topology of the Sock Shop in Weave Scope, you’ll have to create a load on the app.

View the Sock Shop in your app with host-ip:[port number]

• <host-ip:[port number]> is the IP of the master and the port number you see when you run kubectl describe svc front-end -n sock-shop.

With the Sock Shop displayed in the browser, log in to the application with user1 and password. Select a few pairs of socks, put them inot the shopping cart, proceed to checkout and then return to Weave Cloud.

Click on the Containers view where you will see the app begin to take shape with lines appearing between each service.

Unless you are continuing onto another guide, or you are using the cluster for your own app, you may want to tear down the Sock Shop and the Kubernetes cluster you created.

• To uninstall the socks shop, run kubectl delete namespace sock-shop on the master.

• To uninstall Kubernetes on the machines, you can delete the machines you created for this tutorial, and then start over

• To uninstall a daemon set run kubectl delete ds <agent-name>.

Recreating the Cluster: Starting Over

Note: If you made an error during the install instructions, it is recommended that you delete the entire cluster and begin again.

1. Reset the cluster to the local state:

kubeadm reset

2. Run systemctl start kubelet on each of the nodes.

3. Re-initialize the master by kubeadm init on the master.

4. Then join the nodes to the master with:

kubeadm join --token <token> <master-ip>