Announcing Weave Scope: Container Visibility
Introducing Weave Scope Today we are launching the second big piece of Weave. Weave Scope automatically generates a map of your containers, enabling you to intuitively understand, monitor, and control your applications. It is...
Introducing Weave Scope
Today we are launching the second big piece of Weave.
Weave Scope automatically generates a map of your containers, enabling you to intuitively understand, monitor, and control your applications. It is completely open source, and a big part of Weaveworks’ plan to get more Docker users into production in 2015. And the first version has been built by some terrific people who have just joined us via an acquisition. Read all about it below…
We need a new way to do monitoring, visibility and control
There are a lot of monitoring solutions for operations, but not many that are aimed at application developers. We think this is a big gap that needed to be filled. Scope gives developers a new and much more understandable way to monitor applications. And it is designed to work at container scale, for any application, anywhere. Just like Weave.
But there’s more. Weave to date has offered a container network with extras such as service discovery. One key use case for this, is to help customers build cloud native applications. In other words applications that are: container packaged, dynamically scheduled and microservices oriented. Customers want these. The idea is they improve developer productivity, and that ops can get big cost reductions. Efficiencies come from automation, and from having more flexible apps in relation to resources.
Great, yeah, FTW, etc. But then why isn’t everyone already doing this?
- Because of all the questions, such as: if I create a container based application, and run it for while, then how do I know if it is still healthy? When and where do I need to add more capacity? Should I move the application to a different infrastructure or location?
- And because of scale – consider splitting up a monolithic application into multiple parts – micro-services running in containers. Now imagine that this application can grow and shrink, or move, according to need. Keeping track of this is harder than with one big app that never changes.
Hence we concluded this was one of the most important missing pieces of the container landscape.
The requirements of a solution, or: why us?
OK – so we know that customers want monitoring, maps, alerts and other shiny things. And we want this to work for cloud native applications as defined above. What makes us the right people to do it?
We identified the following requirements that, if met, would provide something new and valuable.
- Works with docker, including the emerging open platform & plugins
- Helps developers, because the magic is derived automatically from their application
- Allows ops teams to use any of their familiar tools (eg statsd)
- Understands the application architecture and can change it
- 100% open source
So the fun part is when you realise that all these requirements are already true of Weave.
Weave provides a programmable network, where service discovery and address allocation are all part of the “fabric”. Using this, we could also provide visibility, insight and control into your application. For example, we know about the network topology and service names; if we could also measure QoS like roundtrips for load and health, then we could possibly quarantine bad containers and reroute to new ones.
We decided some time ago that we should do this. But we were not sure how, or quite when to start other than “asap”. And then something really cool happened.
We met an awesome team and made an acquisition
We were interviewing a terrific developer called Peter Bourgon, who while at SoundCloud, had made his mark in the broader community. Some bold strokes included his work on the Roshi CRDT for processing streams of timestamped events, and his talk at QConSF, and his ideas for microservices in the Go community ie. Gokit.
We were not at all surprised to find that Peter had very developed thoughts on monitoring and visualization. But things got interesting when it turned out that he had reached very similar conclusions to our own, but from a different starting point, i.e. operating large scale systems himself. And then he showed us a demo that did exactly what we hoped for 🙂
Peter’s demo blew us away because of two major innovations:
- A beautifully designed front end that gives developers real insight
- A unique CRDT based design that enables scalability from small to large cases
It was not long before we had met the whole team and were discussing an acquisition. Two other developers had created the software with Peter. Their names are David Kaltschmidt, and Harmen Bus. David was the UX guy, and Harmen built the backend tiers with Peter.
Working mainly from Berlin, this team had iterated through serious product development while keeping the lights on with customers. Needless to say, we were impressed – and you will be too – because as part of this acquisition Peter and David have joined Weaveworks.
How to use Weave Scope, and what’s next
The code is open source as of today and lives here. We are integrating it into Weave as fast as we can and expect a first version release in a couple of weeks. Until then, expect things to break often! You can also learn a bit more about it here.
If you are using Weave then please DO check out Scope. We have a bunch of things going on here, all designed to make Scope work really well for Weave users first. Some hints about the roadmap have already been dropped in this blog post. Do please get in touch if you want to help, or have questions.
If you are an SDN vendor or using some network other than Weave, do not despair. We’d love for you to use Scope too, and have no plans to make that awkward for you. Indeed, it should be very easy. So get started or get in touch. With Weave’s growing adoption among Docker developers, and friendly features like automated service discovery and address allocation, it is an ideal way to combine your favourite network with cloud native application development.
Talk to us please! Put issues onto Github, or email us. Or – meet the team: If you are at SREcon this week, you can meet Peter. He and David will be around Berlin and Vienna as well as Hamburg shortly.
