Visualizing Cloud Foundry with Weave Scope

Stev Witzel (Pivotal) will cover how to Visualize Cloud Foundry with Weave Scope Stev will demo an experimental BOSH release for Weave Scope that helps reveal Cloud Foundry's topology and all applications running on top of the platform.

Pivotal’s Stev Witzel joined a recent Weave Online User Group meeting to talk about Cloud Foundry. In his presentation, “Visualizing Cloud Foundry with Weave Scope” Stev explained the many components of Cloud Foundry and how to use Weave Scope to reveal all the applications running on top of the platform.

What is Cloud Foundry?

Cloud Foundry is an application-centric cloud platform where a user simply pushes the application to the cloud and does not care about how the application is run. This is often  called PaaS (Platform as a Service). 

Cloud Foundry is very easy to use, with a single CLI command “cf push” an application can be pushed to the cloud and run. Stev gave an overview of the main elements of Cloud Foundry:

BOSH

One of the core components of Cloud Foundry is the deployment system, BOSH. BOSH enables Cloud Foundry to be deployed to the following Cloud platforms::

• AWS
  
• Azure
  
• OpenStack
  
• vSphere
  
• vCloud Air
  

Within Bosh there are some key elements that are used to deploy a cluster. These are:

BOSH Director:

Stev described BOSH Director as the remote control into the Infrastructure as a Service that hosts Cloud Foundry. The BOSH Director runs as a virtual machine and its function is to spin up other virtual machines that make up the application running on Cloud Foundry. The BOSH documentation has a detailed description of BOSH Director.

Stemcell:

Stemcell defines the operating system of the virtual machine that runs the application on Cloud Foundry. Pre-packaged Stemcells for different platforms can be found at the Pivotal website and are built with security in mind. The BOSH documentation has a more information on  Stemcells.

Release:

A BOSH release is a big tarball that bundles up all the binaries which are necessary to build the cluster. It includes configuration files, startup scripts, and any other information that is necessary to start the virtual machine. In other words, a BOSH release defines “how” a virtual machine is to be deployed. The specific configuration file, startup scripts, and other information for one virtual machine are together called a job. See the BOSH documentation for further details on  BOSH release.

Deployment manifest:

A deployment manifest is defined as a YAML file. It does all the wiring to put all the required resources to work together. This sample YAML file is taken from the BOSH website:

name: redis-us

releases:

- name: redis

version: 15.9 # <--- bump version

resource_pools:

- name: all-machines

stemcell:

name: BOSH-aws-xen-ubuntu-trusty-go_agent

version: 2972

network: default

cloud_properties:

instance_type: m1.small

availability_zone: us-east-1a

...

Further further background see the BOSH documentation.

Deploying a cluster on Cloud Foundry

With all the necessary components t in place, an application can be deployed with these four commands:

bosh upload stemcell stemcell.tgz

bosh upload release cluster-release.tgz

bosh deployment cluster-manifest.yml

bosh deploy cluster

Monitoring and Troubleshooting a Cloud Foundry Cluster

Monitoring and troubleshooting with Weave Scope

BOSH has a few ways to monitor clusters running on Cloud Foundry, for example:

$ bosh vms

$ bosh instances --ps

For troubleshooting using BOSH, we can ssh into the virtual machine or download logs to the local machine for investigation. You would do this with:

$ bosh ssh slave 1

Then with:

$ bosh logs master 0

To explore a system further we can use Weaveworks’ open source project  Weave Scope. Weave Scope is great for monitoring and troubleshooting and is primarily used for container-based systems.

Stev discussed the work he’d done to make Weave Scope work with Cloud Foundry. His solution is to create a BOSH release for Weave Scope with two functions running on the same binary:

• Scope App – a web server to display the data
  
• Scope probe – to collect data from the different virtual machines
  

A new deployment manifest is defined and deployed to the cluster. This manifest will create a virtual machine to run the Scope App for the user to interact with the system.

For the Scope probe to work with the virtual machines in the cluster, Stev created a new YAML file. You use the YAML file to take advantage of a BOSH feature for runtime configuration: the ability to define add-ons for the running virtual machines. This allows the Scope probe to co-locate and run on all of the virtual machines in the cluster once you re-deploy the cluster.

Stev demoed how the Weave Scope App was deployed to the cluster. He showed the details of the VM running the Scope web server for data display:

Then to re-deploy the cluster with the Scope probe running, follow these steps below:

With this integration, a user is now able to use Weave Scope to look at different aspects of the Cloud Foundry cluster such as the interconnection of virtual machines, the status, and vitals of individual virtual machines.

You can use the open-source Weave Scope App running on the cluster locally at the infrastructure hosting Cloud Foundry. Weave Scope is rich in features. Besides the graphical view, it can also provide table views for some of the information.

But you can also send data to Weaveworks’ commercial Weave Cloud product for added functionality such as stored data over time:

Weave Scope plugin

Stev also used the Weave Scope plugin to gather BOSH-specific information such as deployment ID, index, network info, package information, and BOSH template information.

If you want to gather data for containers running in Cloud Foundry, the Weave Scope plugin is also useful. The container format in Cloud Foundry is somewhat different than the format for Docker containers. Regardless, the plugin allows access to Cloud Foundry container-specific information:

Not only does the Weave Scope plugin show the containers in Cloud Foundry, it can also display what applications are running in the containers.

Weave Scope Release

You can check out the Weave Scope Release for Cloud Foundry on GitHub. This is still work in progress and is not ready to be used in production, but Stev and Weaveworks Head of Developer Experience Luke Marsden are actively working on updates. Stay tuned for more information toward the end of summer.

We also have a recording of the full talk for you available:

Thank you for reading our blog. We build Weave Cloud, which is a hosted add-on to your clusters. It helps you iterate faster on microservices with continuous delivery, visualization & debugging, and Prometheus monitoring to improve observability.

Try it out, join our online user group for free talks and trainings, and come and hang out with us on Slack.