This is Part 4 of 4 of the Weave Cloud core tutorial series.
In this guide, how to secure your app by defining Kubernetes Network Policy and having it enforced by Weave Net is demonstrated.
Securing segments of your app is simple with the application of Kubernetes-based policy that is enforced by Weave Net. Add namespaces to the policy yaml files to create software firewalls and visualize the result in Weave Cloud.
A Video Overview
Sign up for a Weave Cloud account
Go to Weave Cloud and register for an account. You’ll use the Weave Cloud token later to send metrics to Cortex and view Weave Net with it.
Deploy a Kubernetes Cluster with Weave Net and the Sample App
If you have already done this as part of one of the other tutorials, you can skip this step. Otherwise, click “Details” below to see the instructions.
1. Create Three Droplets in Digital Ocean
Sign up or log into Digital Ocean and create three Ubuntu instances with the following specifications:
- Ubunutu 16.04
- 4GB or more of RAM per instance
Note: do not select the Private networking option for your droplets. Selecting this option will prevent the setting up of the Kubernetes cluster to fail. See section Initialize the Master for more details.
2. Edit a Weave Cloud Instance Name
Sign up or log into Weave Cloud.
Before you start installing Kubernetes, you may want to rename the default instance in Weave Cloud with a more meaningful name. Weave Cloud instances are the primary workspaces from which you can Deploy, Explore and Monitor your app. In this tutorial you will use Scope in Weave Cloud to Explore the Sock Shop as it spins up on Kubernetes.
3. Set up a Kubernetes Cluster with kubeadm
Kubeadm is by far the simplest way to set up a Kubernetes cluster. With only a few commands, you can deploy a complete Kubernetes cluster with a resilient and secure container network onto the Cloud Provider of your choice in only a few minutes.
kubeadm is a command line tool that and is a part of Kubernetes 1.4 and greater.
kubeadm reference for information on all
kubeadm command-line options and for advice on automating
- Install a secure Kubernetes cluster
- Install Weave Net as a pod network so that application components (pods) can communicate with one another
- Install the Sock Shop, a demo microservices application
- View the result in Weave Cloud
4. Download and install kubelet, kubeadm and Docker
To begin SSH into the machine and become
root (for example, run
sudo su -). Then install the required binaries onto all three instances:
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - cat <<EOF > /etc/apt/sources.list.d/kubernetes.list deb http://apt.kubernetes.io/ kubernetes-xenial main EOF apt-get update
Next, install Docker. You can also use the official Docker packages instead of
docker.io that is referenced here.
apt-get install -y docker.io
And finally, install the Kubernetes packages:
apt-get install -y kubelet kubeadm kubectl kubernetes-cni
5. Initialize the Master
Note: Before making one of your machines a master,
kubeadm must have been installed onto each of the nodes.
The master is the machine where the “control plane” components run, including
etcd (the cluster database) and the API server (which the
kubectl CLI communicates with).
All of these components run in pods started by
Keep in mind that you can’t run
kubeadm init twice without tearing down the cluster, see Tear Down for more information.
To initialize the master, pick one of the machines on which you previously installed
kubeadm and run:
Initialization of the master may take a few minutes.
This autodetects the network interface and then advertises the master on it with the default gateway.
If you want to use a different network interface, you can specify one using
--api-advertise-addresses=<ip-address> flag when you run
Special note on using a different network interface:
As of the time of writing this guide, the use of a different network interface in
Digital Ocean (through the use of the Private Network option for the droplets) will cause
the setting up of the Kubernetes cluster to fail (see issue #203 of kubernetes/kubeadm).
This is related to a problem with
kubeadm and the default gateway that your droplets receive
at the moment of being created.
For now we will have to stick to the default droplet networking settings.
This means that all the nodes in your cluster will be open to the
world and that they communicate between each other via Internet. Make sure that you
understand the implications of such setup. You can reinforce the security of your cluster
Refer to the kubeadm reference doc if you want to read more about the flags
kubeadm init provides.
If the initialization is successful, the output should look similar to the following:
....some preflight checks and initialization Your Kubernetes master has initialized successfully! To start using your cluster, you need to run (as a regular user): sudo cp /etc/kubernetes/admin.conf $HOME/ sudo chown $(id -u):$(id -g) $HOME/admin.conf export KUBECONFIG=$HOME/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: http://kubernetes.io/docs/admin/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join --token <token-id> <master-ip>
Make a record of the
kubeadm join command that
kubeadm init outputs. You will need this once it’s time to join the nodes. This token is used for mutual authentication between the master and any joining nodes.
This token is a secret, and so it’s important to keep it safe — anyone with this key can add authenticated nodes to your cluster.
(Optional) Scheduling Pods on the Master
By default, the cluster does not schedule pods on the master for security reasons. If you want to be able to schedule pods on the master, for example if you want a single-machine Kubernetes cluster for development, then run:
kubectl taint nodes --all dedicated-
The output will be:
node "test-01" tainted taint key="dedicated" and effect="" not found. taint key="dedicated" and effect="" not found.
This removes the “dedicated” taint from any nodes that have it, including the master node, meaning that the scheduler will then be able to schedule pods everywhere.
6. Set up the environment for Kubernetes
On the master run the following as a regular user:
sudo cp /etc/kubernetes/admin.conf $HOME/ sudo chown $(id -u):$(id -g) $HOME/admin.conf export KUBECONFIG=$HOME/admin.conf
7. Install Weave Net as the Pod Networking Layer
In this section, you will install a Weave Net pod network so that your pods can communicate with each other.
You must add Weave Net before deploying any applications to your cluster and before
kube-dns starts up.
Note: Install only one pod network per cluster. There are two versions of the Weave Net daemonset installer. One installs Weave Net to version 1.5 of the Kubernetes binaries and the other installs to 1.6.
If you’re running Kubernetes 1.5 (and less) install Weave Net by logging onto the master and running:
kubectl apply -f https://git.io/weave-kube
If you’re running Kubernetes 1.6 (and above), install Weave Net by logging onto the master and running:
kubectl apply -f https://git.io/weave-kube-1.6
The output will be:
daemonset "weave-net" created
Once a pod network is installed, confirm that it is working by checking that the
kube-dns pod is
kubectl get pods --all-namespaces
kube-dns pod is up and running, you can join all of the nodes to form the cluster.
8. Join Your Nodes to the Master
The nodes are where the workloads (containers and pods, etc) run.
Join the nodes to your cluster by running:
kubeadm join --token <token> <master-ip>
The above command, including the token and the master-ip, is output by
kubeadm init that you ran earlier.
When the node has successfully joined, the output should look as follows:
preflight] Running pre-flight checks [tokens] Validating provided token [discovery] Created cluster info discovery client, requesting info from "http://184.108.40.206:9898/cluster-info/v1/?token-id=ad23e7" [discovery] Cluster info object received, verifying signature using given token [discovery] Cluster info signature and contents are valid, will use API endpoints [https://220.127.116.11:6443] [bootstrap] Trying to connect to endpoint https://18.104.22.168:6443 [bootstrap] Detected server version: v1.6.0 [bootstrap] Successfully established connection with endpoint "https://22.214.171.124:6443" [csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request [csr] Received signed certificate from the API server: Issuer: CN=kubernetes | Subject: CN=system:node:node-02 | CA: false Not before: 2017-02-20 20:33:00 +0000 UTC Not After: 2018-02-20 20:33:00 +0000 UTC [csr] Generating kubelet configuration [kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf" Node join complete: * Certificate signing request sent to master and response received. * Kubelet informed of new secure connection details. Run 'kubectl get nodes' on the master to see this machine join.
kubectl get nodes on the master to display a cluster with the number of machines as you created.
(Optional) Control Your Cluster From Machines Other Than The Master
In order to get
kubectl on your laptop to talk to your cluster (as an example), copy the
kubeconfig file from your master to your laptop:
scp root@<master ip>:/etc/kubernetes/admin.conf . kubectl --kubeconfig ./admin.conf get nodes
8. Install and Launch the Weave Cloud Agents
yaml file below installs all of the Cloud probes in a DaemonSet and launches them to your cluster. This allows you to use: Flux for Continuous Delivery, Scope for Troubleshooting and Cortex for Prometheus Monitoring.
From the master:
kubectl apply -n kube-system \ -f "https://cloud.weave.works/k8s/v1.6/weave-cloud?t=<cloud-token>"
<cloud-token> is found in the settings dialog on Weave Cloud.
If you mistyped or copied and pasted the command incorrectly, you can remove the DaemonSet with:
kubectl delete -n kube-system \ -f "https://cloud.weave.works/k8s/v1.6/weave-cloud?t=?t=anything"
Return to Weave Cloud, and click Explore to display Scope and then Pods to show the Kubernetes cluster. Ensure that the All Namespaces filter is enabled from the left-hand corner.
In these next steps, you can watch as the Sock Shop containers start appearing in Scope in Weave Cloud.
9. Install the Sock Shop onto Kubernetes
To put your cluster through its paces, install the sample microservices application, Socks Shop. Learn more about the sample microservices app by referring to the microservices-demo README.
To install the Sock Shop, run the following:
kubectl create namespace sock-shop git clone https://github.com/microservices-demo/microservices-demo cd microservices-demo kubectl apply -n sock-shop -f deploy/kubernetes/manifests
Click on the Pod view and then enable the sock-shop namespace filter from the bottom left-hand corner in the Weave Cloud user interface.
It takes several minutes to download and start all of the containers. Watch the output of
kubectl get pods -n sock-shop to see that all of the containers are successfully running.
Or view the containers as they get created in Weave Cloud.
10. View the Sock Shop in Your Browser
Find the port that the cluster allocated for the front-end service by running:
kubectl describe svc front-end -n sock-shop
The output should look like:
Name: front-end Namespace: sock-shop Labels: name=front-end Selector: name=front-end Type: NodePort IP: 100.66.88.176 Port: <unset> 80/TCP NodePort: <unset> 31869/TCP Endpoints: <none> Session Affinity: None
Launch the Sock Shop in your browser by going to the IP address of any of your node machines in your browser, and by specifying the NodePort. So for example,
http://<master_ip>:<pNodePort>. You can find the IP address of the machines in the DigitalOcean dashboard.
In the example above, the NodePort was
If there is a firewall, make sure it exposes this port to the internet before you try to access it.
11. Create a Load on the Sock Shop
To fully appreciate the topology of the Sock Shop in Weave Scope, you’ll have to create a load on the app.
View the Sock Shop in your app with
<host-ip:[port number]>is the IP of the master and the port number you see when you run
kubectl describe svc front-end -n sock-shop.
With the Sock Shop displayed in the browser, log in to the application with
password. Select a few pairs of socks, put them inot the shopping cart, proceed to checkout and then return to Weave Cloud.
Click on the Containers view where you will see the app begin to take shape with lines appearing between each service.
Apply a Network Policy, and Enforce with Weave Net
In the above guide, you should have deployed the socks shop. However, the different components are not isolated.
Let’s start by testing that. Load up Weave Cloud and make sure you’re in the containers view, then observe that the catalogue service can talk to the shipping service.
Search for the “catalogue” container and then click the
>_ icon from the details panel. This opens up a convenient shell for that container.
Type the following:
You should get:
wget: server returned error: HTTP/1.1 404
This is not good! The catalogue service can speak to the shipping service. For a hacker who managed to infiltrate the catalogue service, they could now get direct access to the shipping service and attack that too.
So, let’s apply some network policy. SSH into the master, or where ever you run
cd microservices-demo kubectl apply -f deploy/kubernetes/manifests-policy/
Now run the wget inside the terminal in Weave Cloud again:
And you’ll see the connection just times out. Those packets are being dropped. The app is now more secure!
Unless you are continuing onto another guide, or you are using the cluster for your own app, you may want to tear down the Sock Shop and the Kubernetes cluster you created.
To uninstall the socks shop, run
kubectl delete namespace sock-shopon the master.
To uninstall Kubernetes on the machines, you can delete the machines you created for this tutorial, and then start over
To uninstall a daemon set run
kubectl delete ds <agent-name>.
Recreating the Cluster: Starting Over
Note: If you made an error during the install instructions, it is recommended that you delete the entire cluster and begin again.
1. Reset the cluster to the local state:
systemctl start kubelet on each of the nodes.
3. Re-initialize the master by
kubeadm init on the master.
4. Then join the nodes to the master with:
kubeadm join --token <token> <master-ip>
You’ve seen that Kubernetes network policy allows you to define flexible and dynamic security policies, and Weave Net allows you to enforce them.
Join the Weave Community
If you have any questions or comments you can reach out to us on our Slack channel. To invite yourself to the Community Slack channel, visit Weave Community Slack invite or contact us through one of these other channels at Help and Support Services.