Istio and Weave Cloud

Istio and Weave Cloud can work together to achieve several goals:

Use Weave Cloud Explore to visualize Istio in action. This means that you’ll be able to see how Istio deploys sidecars for each of your application containers and you’ll be able to verify how all the application traffic goes via these sidecars. You’ll be able to see a before and after of the changes Istio’s kube-inject makes to your app in real time that helps visualize, understand and learn how Istio works.
Have Weave Cloud Monitor handle the long-term storage of Prometheus metrics, as well as providing a nicer GUI for exploring available metrics. This means you don’t have to worry about scalability of your Prometheus instance, or managing storage within your Kubernetes cluster – we handle that for you. You can configure the same Grafana dashboard that Istio provides out of the box to query Weave Cloud.

Demo!

Try it yourself – Setup

For convenience, this guide assumes you’re using a GKE cluster. If you don’t have one, go and get one. Tested with 3 n1-highcpu-4 instances, smaller clusters may run out of capacity.
Sign up for a Weave Cloud account but do not connect it to your GKE cluster yet. We’ll use a slightly modified config to get Prometheus deployed in the Weave Cloud bundle to scrape the Istio Mixer.
Install Istio on your cluster. This will also deploy Prometheus but it’s fine to have two of them running.

Short version of installing Istio:

kubectl apply -f install/kubernetes/istio.yaml
kubectl apply -f install/kubernetes/addons/prometheus.yaml
kubectl apply -f install/kubernetes/addons/grafana.yaml
kubectl apply -f install/kubernetes/addons/servicegraph.yaml

Customized config

We will customize the Weave Cloud config to configure it to scrape the Istio Mixer. Get <your-weave-cloud-token> from the right hand side of the Weave Cloud web interface.

export WEAVE_CLOUD_TOKEN=<your-weave-cloud-token>
wget -O weave-cloud.yaml "https://cloud.weave.works/k8s.yaml?service-token=$WEAVE_CLOUD_TOKEN&k8s-version=$(kubectl version | base64 | tr -d '\n')"
vim weave-cloud.yaml # or use the text editor of your choice

Add the following stanza to the scrape_configs section (don’t forget to :set paste if you’re using vim):

          - job_name: 'mixer'
            # Override the global default and scrape targets from this job every 5 seconds.
            scrape_interval: 5s
            # metrics_path defaults to '/metrics'
            # scheme defaults to 'http'.
            static_configs:
            - targets: ['istio-mixer:42422']

Now run:

kubectl apply -f weave-cloud.yaml

Deploy sample app and visualize in Weave Cloud Explore

Deploy the Bookinfo sample app without kube-inject. From your Istio directory:

kubectl apply -f samples/apps/bookinfo/bookinfo.yaml

Find out what the LoadBalancer IP is:

kubectl get ingress -o wide

If there’s nothing in the ADDRESS column, you may have to wait a while.

Open the page in your browser, e.g. http://35.184.222.222/productpage

Refresh the page a few times, and observe the app join up in Weave Cloud, in the pods view:

OK, so this is how the app behaves without Istio. Let’s see how it works with Istio enabled!

kubectl delete -f samples/apps/bookinfo/bookinfo.yaml
source istio.VERSION
kubectl apply -f <(bin/istioctl kube-inject -f samples/apps/bookinfo/bookinfo.yaml)

Cool, we can see how Istio works in Weave Cloud Explore. We can see the sidecars in the pod view:

And we can drill down to see the individual containers in each pod, including the proxy sidecar:

And we can see how the proxy containers talk to the mixer:

As an exercise for the reader, try changing request routing and observe that Weave Cloud Explore can show you where the requests are going in pod view.

Using Weave Cloud’s hosted Prometheus for Istio metrics

The fact that Istio can export metrics to Prometheus is great, but Prometheus is single-server and therefore has limited storage and availability. Let’s ship our Istio metrics to Weave Cloud instead, where they can scale infinitely and get looked after by the nice folks at Weaveworks.

Go to the Monitor tab in Weave Cloud and notice that we’re already collecting mixer metrics with our enhanced Prometheus GUI:

We can also reconfigure the Grafana dashboard to go get metrics from Weave Cloud:

kubectl port-forward $(kubectl get pod -l app=grafana -o jsonpath='{.items[0].metadata.name}') 3000:3000 & sleep 1
open http://localhost:3000/dashboard/db/istio-dashboard

See Integrating Grafana for how to hook Grafana up to Weave Cloud:

Then you can set the panels in your dashboards to source data from Weave Cloud, rather than the cluster-local Prometheus instance, for better scalability, resilience, and easier operational management.

Now your Istio metrics can be served from Weave Cloud:

Huzzah!