The Long Tail: Tools To Investigate High Long Tail Latency

Figuring out what's causing slow requests or ops can be tricky. Tom Wilkie shares the tools he used to investigate high long tail latency.

Kubernetes Rolling Upgrades One By One

Have a semi-stateful service running on Kubernetes and want to do rolling upgrades? Tom Wilkie outlines how to make deployments upgrade one by one.

The Red Method: Key Metrics For Microservices Architecture

Tom Wilkie shares Weaveworks monitoring philosophy and the three most important metrics to use in your microservices architecture.

Websockets Are Evil And Must Die

In previous blog posts I’ve mention my dislike for WebSockets. In this quick blog post I attempt to explain why. Unfortunately I think this raises more questions than it answers! Be aware, these opinions are my own. It’s the monitoring,...

Running Consul On Kubernetes And Monitoring It With Prometheus

We run a couple of replicated Consul services for Weave Cloud: one acts as a coordinator for the distributed WebSocket router backing Scope’s terminals feature, and the other for storing the state of Cortex’s consistent hash ring. This...

Aggregating Pod Resource (cpu, Memory) Usage By Arbitrary Labels With Prometheus

How would you answer the questions like “how much CPU is my service consuming?” using Prometheus and Kubernetes? In this quick post, I’ll show you how… First we need to think about where to get the information from. cAdvisor (from Google)...

Cloud Provider Options For A Container-friendly Environment

The advent Docker and containers, has simplified packaging, deploying and supporting applications. However, with this advancement come challenges, as you also need a robust environment in which to deploy the containers. And that requires...

Dashboards As Code

If you use Prometheus, then you probably use Grafana. At Weave, we have Grafana dashboards for all of our microservices. When we want to understand our system, our Grafana dashboards are the first things we look at. To make the most out of...

Configuring Container Capabilities With Kubernetes

In this blog post, I discuss how to configure capabilities for Docker containers running on Kubernetes. Even though Linux containers run in their own isolated bubble, they aren’t just allowed to do anything they like. Some actions, such as...

Weave Mesh In Prometheus Alertmanager

tl;dr: Weave Mesh, an open-source gossip and CRDT communications library, gave the Prometheus Alertmanager exactly what it needed to be reliable and highly-available. As a former SoundCloud engineer, I’ve had the privilege of using...

Monitoring Your Kubernetes Infrastructure With Prometheus

In the last post in our series “Prometheus and Kubernetes”, Tom talked about how we use Prometheus to monitor the applications and services we deploy to Kubernetes. In this post, I want to talk about how we use Prometheus to monitor our...