This is Part 3 of 4 of the Weave Cloud core tutorial series.

Containerized applications are by nature dynamic and in a state of constant change. Applications may be spread across multiple cloud provides or they can span both data centers and cloud providers (hybrid clouds). Because of these factors, traditional server-focused monitoring systems designed for relatively static systems are not adequate.

Weave Cloud monitoring is extends the open source project Prometheus and when the agent is deployed to your cluster, it listens for changes within your Kubernetes cluster and pushes metrics to Weave Cloud, where they can be viewed and queried.

In this example, you will use the sample app, ‘The Sock Shop’, deploy it to three virtual machines running Docker and Kubernetes and then monitor metrics from within Weave Cloud.

This tutorial takes approximately 25 minutes to complete.

« Go to previous part: Part 2 – Deploy: Continuous Delivery
Go to next part: Part 4 – Secure: Container Firewalls »

A Video Overview

What You Will Use

Sign Up for Weave Cloud

Sign up for a Weave Cloud account and get a Weave Cloud token:

1. Go to Weave Cloud

2. Sign up using either a Github, or Google account or use an email address.

3. Make a note of the cloud service token from the User settings screen:

Note: If you are continuing from one of the other tutorials in this series, then the Weave Cloud agents should already be running on your hosts. If not, use your Cloud token to set up Weave Cloud including Prometheus monitoring with Weave Cloud.

Deploy a Kubernetes Cluster with Weave Net and the Sample App

If you have already done this as part of one of the other tutorials, skip these steps.

This example uses Digital Ocean, but you can just as easily create three instances in AWS, Google Cloud Platform or Microsoft Azure or any other cloud provider.

1. Create Three Droplets in Digital Ocean

Sign up or log into Digital Ocean and create three Ubuntu instances with the following specifications:

  • Ubunutu 16.04
  • 4GB or more of RAM per instance

Note: do not select the Private networking option for your droplets. Selecting this option will prevent the setting up of the Kubernetes cluster to fail. See section Initialize the Master for more details.

2. Add a New Weave Cloud Instance

Sign up or log into Weave Cloud.

Create a new instance or rename the default instance in Weave Cloud]( Weave Cloud instances are the primary workspaces for your application and provides a view onto your cluster and the application that is running on it.

3. Set up a Kubernetes Cluster with kubeadm

Kubeadm is by far the simplest way to set up a Kubernetes cluster. With only a few commands, you can deploy a complete Kubernetes cluster with a resilient and secure container network onto the Cloud Provider of your choice in only a few minutes.

kubeadm is a command line tool that and is a part of Kubernetes 1.4 and greater.

See the kubeadm reference for information on all kubeadm command-line options and for advice on automating kubeadm.


  • Install a secure Kubernetes cluster
  • Install Weave Net as a pod network so that application components (pods) can communicate with one another
  • Install the Sock Shop, a demo microservices application
  • View the result in Weave Cloud

4. Download and install kubelet, kubeadm and Docker

To begin SSH into the machine and become root (for example, run sudo su -). Then install the required binaries onto all three instances:

curl -s | apt-key add -
cat <<EOF > /etc/apt/sources.list.d/kubernetes.list
deb kubernetes-xenial main
apt-get update

Next, install Docker. You can also use the official Docker packages instead of that is referenced here.

apt-get install -y

And finally, install the Kubernetes packages:

apt-get install -y kubelet kubeadm kubectl kubernetes-cni

5. Initialize the Master

Note: Before making one of your machines a master, kubelet and kubeadm must have been installed onto each of the nodes.

The master is the machine where the “control plane” components run, including etcd (the cluster database) and the API server (which the kubectl CLI communicates with).

All of these components run in pods started by kubelet.

Keep in mind that you can’t run kubeadm init twice without tearing down the cluster, see Tear Down for more information.

To initialize the master, pick one of the machines on which you previously installed kubelet and kubeadm and run:

kubeadm init

Initialization of the master may take a few minutes.

This autodetects the network interface and then advertises the master on it with the default gateway.

Note: If you want to use a different network interface, specify it with the --api-advertise-addresses=<ip-address> flag when you run kubeadm init.

Important! Special note on selecting different network interface:

When using a different network interface in Digital Ocean through the Private Network option for the droplets causes the Kubernetes cluster set up to fail. For more information, see issue #203 of kubernetes/kubeadm. This can occur with kubeadm and the default gateway that your droplets may receive at the moment of creation.

Until this issue is resolved, the default droplet networking settings must be enabled. This means that all the nodes in your cluster will be open to the world and that they communicate between each other via Internet. Ensure that you understand the implications of such a set up. You can reinforce the security of your cluster by using ufw or iptables rules.

Refer to the kubeadm reference doc to read up on the flags kubeadm init provides.

If the initialization is successful, the output should look similar to the following:

....some preflight checks and initialization

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run (as a regular user):

  sudo cp /etc/kubernetes/admin.conf $HOME/
  sudo chown $(id -u):$(id -g) $HOME/admin.conf
  export KUBECONFIG=$HOME/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

You can now join any number of machines by running the following on each node
as root:

  kubeadm join --token <token-id> <master-ip>

Make a record of the kubeadm join command that kubeadm init outputs. You will need this once it’s time to join the nodes. This token is used for mutual authentication between the master and any joining nodes.

This token is a secret, and so it’s important to keep it safe — anyone with this key can add authenticated nodes to your cluster.

(Optional) Scheduling Pods on the Master

By default, the cluster does not schedule pods on the master for security reasons. If you want to be able to schedule pods on the master, for example if you want a single-machine Kubernetes cluster for development, then run:

kubectl taint nodes --all dedicated-

The output will be:

node "test-01" tainted
taint key="dedicated" and effect="" not found.
taint key="dedicated" and effect="" not found.

This removes the “dedicated” taint from any nodes that have it, including the master node, meaning that the scheduler will then be able to schedule pods everywhere.

6. Set up the environment for Kubernetes

On the master run the following as a regular user:

  sudo cp /etc/kubernetes/admin.conf $HOME/
  sudo chown $(id -u):$(id -g) $HOME/admin.conf
  export KUBECONFIG=$HOME/admin.conf

7. Install Weave Net as the Pod Networking Layer

In this section, you will install a Weave Net pod network so that your pods can communicate with each other.

You must add Weave Net before deploying any applications to your cluster and before kube-dns starts up.

Note: Install only one pod network per cluster. There are two versions of the Weave Net daemonset installer. One installs Weave Net to version 1.5 of the Kubernetes binaries and the other installs to 1.6.

If you’re running Kubernetes 1.5 (and less) install Weave Net by logging onto the master and running:

kubectl apply -f

If you’re running Kubernetes 1.6 (and above), install Weave Net by logging onto the master and running:

kubectl apply -f "$(kubectl version | base64 | tr -d '\n')"

The output will be:

serviceaccount "weave-net" created
clusterrole "weave-net" created
clusterrolebinding "weave-net" created
daemonset "weave-net" created

Once a pod network is installed, confirm that it is working by ensuring that the kube-dns pod is running:

kubectl get pods --all-namespaces

Once the kube-dns pod is up and running, you can join all of the nodes to form the cluster.

8. Join Your Nodes to the Master

The nodes are where the workloads (containers and pods, etc) run.

Join the nodes to your cluster by running:

kubeadm join --token <token> <master-ip>

The above command, including the token and the master-ip, is output by kubeadm init that you ran earlier.

When the node has successfully joined, the output should look as follows:

preflight] Running pre-flight checks
[tokens] Validating provided token
[discovery] Created cluster info discovery client, requesting info from ""
[discovery] Cluster info object received, verifying signature using given token
[discovery] Cluster info signature and contents are valid, will use API endpoints []
[bootstrap] Trying to connect to endpoint
[bootstrap] Detected server version: v1.6.0
[bootstrap] Successfully established connection with endpoint ""
[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request
[csr] Received signed certificate from the API server:
Issuer: CN=kubernetes | Subject: CN=system:node:node-02 | CA: false
Not before: 2017-02-20 20:33:00 +0000 UTC Not After: 2018-02-20 20:33:00 +0000 UTC
[csr] Generating kubelet configuration
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"

Node join complete:
* Certificate signing request sent to master and response
* Kubelet informed of new secure connection details.

Run 'kubectl get nodes' on the master to see this machine join.

Run kubectl get nodes on the master to display a cluster with the number of machines as you created.

(Optional) Control Your Cluster From Machines Other Than The Master

In order to get kubectl on your laptop to talk to your cluster (as an example), copy the kubeconfig file from your master to your laptop:

scp root@<master ip>:/etc/kubernetes/admin.conf .
kubectl --kubeconfig ./admin.conf get nodes

8. Install and Launch the Weave Cloud Agents

The yaml file listed below installs all of the Weave Cloud probes to a DaemonSet and launches them to your cluster.

From the master:

kubectl apply -n kube-system -f \
   "<cloud-token>&k8s-version=$(kubectl version | base64 | tr -d '\n')"

The <cloud-token> is found in the settings dialog on Weave Cloud. The above command may also be cut and pasted from the setup screens in Weave Cloud.

If you mistyped or copied and pasted the command incorrectly, you can remove the DaemonSet with:

kubectl delete -n kube-system \
  -f ""

Return to Weave Cloud, and click Explore and then Pods to display the Kubernetes cluster in your instance. Ensure that the All Namespaces filter is enabled from the left-hand corner.

For the next few steps, keep the instance open on Explore to watch the Sock Shop containers spin up in the cluster.

9. Install the Sock Shop onto Kubernetes

To put your cluster through its paces, install the sample microservices application, Socks Shop. Learn more about the sample microservices app by referring to the microservices-demo README.

To install the Sock Shop, run the following:

kubectl create namespace sock-shop
git clone
cd microservices-demo
kubectl apply -n sock-shop -f deploy/kubernetes/manifests

Click on Explore and then Pod and enable the sock-shop namespace filter from the bottom left-hand corner.

It takes several minutes to download and start all of the containers. Watch the output of kubectl get pods -n sock-shop to see that all of the containers are successfully running.

Or view the containers as they get created in Weave Cloud.

10. View the Sock Shop in Your Browser

Find the port that the cluster allocated for the front-end service by running:

kubectl describe svc front-end -n sock-shop

The output should look like:

Name:                   front-end
Namespace:              sock-shop
Labels:                 name=front-end
Selector:               name=front-end
Type:                   NodePort
Port:                   <unset> 80/TCP
NodePort:               <unset> 31869/TCP
Endpoints:              <none>
Session Affinity:       None

Launch the Sock Shop in your browser by going to the IP address of any of your node machines in your browser, and by specifying the NodePort. So for example, http://<master_ip>:<pNodePort>. You can find the IP address of the machines in the DigitalOcean dashboard.

In the example above, the NodePort was 31869.

If there is a firewall, make sure it exposes this port to the internet before you try to access it.

11. Create a Load on the Sock Shop

To fully appreciate the topology of the Sock Shop in Weave Scope, you’ll have to create a load on the app.

View the Sock Shop in your app with host-ip:[port number]

  • <host-ip:[port number]> is the IP of the master and the port number you see when you run kubectl describe svc front-end -n sock-shop.

With the Sock Shop displayed in the browser, log in to the application with user1 and password. Select a few pairs of socks, put them inot the shopping cart, proceed to checkout and then return to Weave Cloud.

Click on the Containers view where you will see the app begin to take shape with lines appearing between each service.

Configuring Monitor for Your Production Environment

Next, enable the Weave Cloud agent to start pushing metrics to Weave Cloud.

1. Log onto the master Kubernetes node and run the following to launch the all of the Weave Cloud agents onto your hosts. Keep your Cloud service token handy and paste it into the command:

kubectl apply -n kube-system -f \
   "[cloud-token]&k8s-version=$(kubectl version | base64 | tr -d '\n')"


  • [cloud-token] is the token you obtained when you signed up for Weave Cloud.

If you mistyped or copied and pasted the command incorrectly, you can remove the DaemonSet with:

kubectl delete -n kube-system \
  -f ""

The Weave Cloud monitor agent runs in its own Docker container and it may take a few minutes for it to download and appear on the server. You can watch for it to appear in Weave Cloud.

2. Or check that Cortex is running on one of the Kubernetes nodes with:

kubectl get pods -n kube-system -l weave-cloud-component=cortex

Where you should see something similar to:

NAME                                 READY     STATUS    RESTARTS   AGE
weave-cortex-agent-459196232-a83mc   1/1       Running   0          10s
weave-cortex-node-exporter-clofd     1/1       Running   0          12s
weave-cortex-node-exporter-jeyrt     1/1       Running   0          12s
weave-cortex-node-exporter-klp2h     1/1       Running   0          12s
weave-cortex-node-exporter-uqwx5     1/1       Running   0          12s

Viewing Sock Shop Metrics in Weave Cloud Monitor

Go to Weave Cloud and click ‘Monitor’ from the header. You should see the Monitor UI where you can display metrics from the Sock Shop app.

Select CPU usage in % by pod from the Kubernetes section in the Detected Metrics section of Prometheus System Queries, where something similar to the screen capture shown below will be shown (note this is a stacked view of the chart):

Run the Load Test

To view metrics in Weave Cloud Monitor under a load, run:

docker run -ti --rm --name=LOAD_TEST  weaveworksdemos/load-test -r 100 -c 2 -h <host-ip:[port number]>

Running Queries with the Prometheus Query Language

Custom queries can be built with the Prometheus Query Language builder. For example detailed metrics can be viewed by Node, by orchestrator such as Kubernetes or you can view metrics about your Weave Net container network.

For more information on using the Prometheus Query Language, see Prometheus Query Examples

Monitor Weave Net with Weave Cloud

Weave Net supports Prometheus monitoring, so you can monitor your Weave Net network in Weave Cloud.

Go to the Cortex page in Weave Cloud to view network specific metrics, such as:

  • IP address space exhaustion in %
  • Number of local DNS entries per each host
  • Connection termination rate per second
  • Number of blocked connections per transport-layer protocol
  • Frequent protocol-dport combinations of blocked connections such as IP address exhaustion

For more information on Weave Net metrics see Monitoring with Prometheus.

Tear Down

Unless you are continuing onto another guide, or you are using the cluster for your own app, you may want to tear down the Sock Shop and the Kubernetes cluster you created.

  • To uninstall the socks shop, run kubectl delete namespace sock-shop on the master.

  • To uninstall Kubernetes on the machines, you can delete the machines you created for this tutorial, and then start over

  • To uninstall a daemon set run kubectl delete ds <agent-name>.

Recreating the Cluster: Starting Over

Note: If you made an error during the install instructions, it is recommended that you delete the entire cluster and begin again.

1. Reset the cluster to the local state:

kubeadm reset

2. Run systemctl start kubelet on each of the nodes.

3. Re-initialize the master by kubeadm init on the master.

4. Then join the nodes to the master with:

kubeadm join --token <token> <master-ip>


You’ve seen how Weave Cloud can be used to implement Prometheus monitoring to manage cloud based applications.

Join the Weave Community

If you have any questions or comments you can reach out to us on our Slack channel. To invite yourself to the Community Slack channel, visit Weave Community Slack invite or contact us through one of these other channels at Help and Support Services.

« Go to previous part: Part 2 – Deploy: Continuous Delivery
Go to next part: Part 4 – Secure: Container Firewalls »