Never Lose Data with Persistent Volume Snapshots in Weave Scope 1.10
Weave Scope 1.10 just landed with many improvements. Kubernetes Persistent Volumes clone/snapshot operations are now supported. Scope got faster, bugs were fixed and its community is more active than it’s ever been.
It’s been a while since we last talked about one of our most popular open source projects. Weave Scope is used by a lot of people out there to make what’s happening in your cluster observable. At the Weaveworks booth at various conferences I immediately noticed, how people regularly stop by when they see Weave Scope in action.
Bryan Boreham just released Weave Scope 1.10 and the changes the team landed are very impressive. Some of the highlights are:
- Snapshot and Clone operations for Kubernetes Persistent Volumes.
- Kubernetes objects can be reported just once in a cluster, instead of reporting the same data from every node.
- The App now supports HTTP basic auth.
And lots more. I’m particularly happy about all the great news from our community.
Snapshot and Clone Operations
One big part of Scope 1.10 are the Snapshot and Clone operations for Kubernetes Persistent Volumes. Satyam Zode, Senior Software Engineer at MayaData Inc. and freshly baked maintainer of Scope says this about one of new features that just landed in Scope:
"Kubernetes has come a long way in dealing with Stateful applications. Right from In-tree volume plugins, to a more generic out-of-tree dynamic provisioners and achieved a great adoption of this architecture. One challenge that will always remain with stateful applications is the management of data operations. While the recent versions of Kubernetes have adopted a generic design to deal with provisioning snapshots and promoting snapshots to persistent volumes, it just increases the number of components to deal with the regular Kubernetes cli 'kubectl' for day to day operations and troubleshooting of data operations.
We recently added Persistent Volume support to Weave Scope to make the visualization and basic troubleshooting of stateful applications simpler. The next big addition to Weave Scope in terms of stateful applications come in terms of snapshot and clone support. The adjacency management and end-user workflow for snapshots and clones is made very simple and clean. You can now see, which snapshot belongs to which PV (and hence which application) and which new clone is promoted from which snapshot etc. All with nice visualization. We have also provided a filter to turn on/off the snapshot visualization as it can get quite cumbersome when taking snapshots at regular intervals through cron jobs as is the regular practice in production workloads.
A simple example use-case is , if you are running a cron job for taking snapshot on regular intervals and at some point you need to recover a data, you can go to Weave Scope, search the required application pod, get the persistent volume and scan through the available snapshots. Pick the required snapshot, click on the clone control and wait for a new cloned PV to be created from the given snapshot and you automatically get a new PVC associated with this PV as well. Now you can use this PVC in the application, mount it at a new path, browse it to recover the required data."
(Satyam worked with his colleague Akash Srivastava on this. Bianca Cheng Costanzo and Filip Barl from Weaveworks helped out with UI bits.)
Scope 1.10 is faster, prettier, more consistent, and has more tests
Bryan Boreham put quite a bit of work into the performance optimizations of Scope. Scope’s probes now use at least 10% less CPU. In general quite a bit of memory/CPU is saved and quite a few filtering optimizations make Scope even more lightweight.
A team of people came together and brought initial support for the Container Runtime Interface (CRI): thanks Lili Cosic, Damien Lespiau, gotjosh and Ajay Kelkar.
Marcus Cobden and Bryan worked on adding Opentracing (Jaeger) distributed tracing for profiling the Scope app. Akash Srivastava, Julien Garcia Gonzalez and Satyam fixed the Kubernetes example deployment of Scope. Many many more bug fixes landed as well.
On the UI side of things a big team of people came together to make Scope more consistent, less cluttered and easier to use: It’s been Bianca Cheng Costanzo, Tomasz Skowroński, Filip Barl, Simon Howe, Guy Edwards and Rakesh Singh.
In the build and test department we saw a big bunch of changes as well. Scope now builds on the Power CPU architecture, the Dockerfiles were updated and slimmed down, and many dependencies were also updated.
Since the last release (1.9.1), about 45% of the files were changed,54 people contributed in Github issues and 30 contributed code, of these 15 were first-timers. As one would expect, 10 landed their patches during Hacktoberfest. Thanks a lot everyone for your great work!
If you want to hack on Scope and join the team, take a look at our list of “good first issues”. We are happy to help you get started. Just talk to us on Slack.
We have grown our community!
Weave Scope always was open source, but a few months back, Alfonso Acosta, one of Scope’s maintainers started the process of making it easier for new contributors to get involved.
Since then the Weave Scope community met regularly in Zoom calls (visit the link to see videos and minutes of past meetings). We discuss upcoming changes, review PRs and new issues. It’s been great to see new contributors from all parts of the globe came together to push things forward.
Another big step was to put Scope’s governance on broader feet. Since then Filip Barl and Satyam Zode have joined Alfonso and Bryan as maintainers and have been reviewing and approving PRs. We are very grateful for both of them stepping up to this! Here’s a short introduction:
"I am Satyam, I work as Senior Software Engineer at MayaData Inc. I am excited about distributed systems, Kubernetes, OpenEBS, Weave Scope and Go.
I find Weave Scope interesting because Weave Scope is an excellent example of an intuitive engineering. Weave Scope is very useful platform for application developers to see how their microservices and cloud native workloads are working. I am working towards making Weave Scope more powerful by bringing Kubernetes storage resources."
Filip Barl is Software Engineer at Weaveworks and works from the Berlin office:
"I’ve been contributing to the UI of Weave Scope for the last 2 years.
One of the things I like most about Scope is its very straightforward idea and implementation. A single screen which gives a big picture overview of all the interactions in your system as well as letting you pin down to individual nodes covers a wide span of use cases ranging from live presentations to debugging specific containers.
I also believe Scope has a lot of potential to grow into a more complete cross-functional product by making use of data persistence and making the UX more clean and intuitive. Having the latter in mind, since the last minor release, Bianca and I started moving Scope towards a unified Design System:
- Colors and Fonts have been standardized
Some atomic UI components have been identified and standardized
Except for that, I’ve been working on removing the remaining bits of Weave Cloud legacy logic from the Scope UI codebase, to make it into a cleaner OSS product."
Next meeting coming up Tuesday, 11th December 2018
If you like Scope, and want to be involved, have your say or just learn about what’s coming up, please come and join us at the next meeting. It happens every second Tuesday - here is the info you need.
As always, find us on the #scope channel - here’s the invite link. We are looking forward to seeing you there!
