Pivotal’s Stev Witzel joined a recent Weave Online User Group meeting to talk about Cloud Foundry. In his presentation, “Visualizing Cloud Foundry with Weave Scope” Stev explained the many components of Cloud Foundry and how to use Weave Scope to reveal all the applications running on top of the platform.
What is Cloud Foundry?
Cloud Foundry is an application-centric cloud platform where a user simply pushes the application to the cloud and does not care about how the application is run. This is often called PaaS (Platform as a Service).
Cloud Foundry is very easy to use, with a single CLI command “cf push” an application can be pushed to the cloud and run. Stev gave an overview of the main elements of Cloud Foundry:
One of the core components of Cloud Foundry is the deployment system, BOSH. BOSH enables Cloud Foundry to be deployed to the following Cloud platforms::
- vCloud Air
Within Bosh there are some key elements that are used to deploy a cluster. These are:
Stev described BOSH Director as the remote control into the Infrastructure as a Service that hosts Cloud Foundry. The BOSH Director runs as a virtual machine and its function is to spin up other virtual machines that make up the application running on Cloud Foundry. The BOSH documentation has a detailed description of BOSH Director.
Stemcell defines the operating system of the virtual machine that runs the application on Cloud Foundry. Pre-packaged Stemcells for different platforms can be found at the Pivotal website and are built with security in mind. The BOSH documentation has a more information on Stemcells.
A BOSH release is a big tarball that bundles up all the binaries which are necessary to build the cluster. It includes configuration files, startup scripts, and any other information that is necessary to start the virtual machine. In other words, a BOSH release defines “how” a virtual machine is to be deployed. The specific configuration file, startup scripts, and other information for one virtual machine are together called a job. See the BOSH documentation for further details on BOSH release.
A deployment manifest is defined as a YAML file. It does all the wiring to put all the required resources to work together. This sample YAML file is taken from the BOSH website:
- name: redis
version: 15.9 # <--- bump version
- name: all-machines
Further further background see the BOSH documentation.
Deploying a cluster on Cloud Foundry
With all the necessary components t in place, an application can be deployed with these four commands:
bosh upload stemcell stemcell.tgz
bosh upload release cluster-release.tgz
bosh deployment cluster-manifest.yml
bosh deploy cluster
Monitoring and Troubleshooting a Cloud Foundry Cluster
Monitoring and troubleshooting with Weave Scope
BOSH has a few ways to monitor clusters running on Cloud Foundry, for example:
$ bosh vms
$ bosh instances --ps
For troubleshooting using BOSH, we can ssh into the virtual machine or download logs to the local machine for investigation. You would do this with:
$ bosh ssh slave 1
$ bosh logs master 0
To explore a system further we can use Weaveworks’ open source project Weave Scope. Weave Scope is great for monitoring and troubleshooting and is primarily used for container-based systems.
Stev discussed the work he’d done to make Weave Scope work with Cloud Foundry. His solution is to create a BOSH release for Weave Scope with two functions running on the same binary:
- Scope App – a web server to display the data
- Scope probe – to collect data from the different virtual machines
A new deployment manifest is defined and deployed to the cluster. This manifest will create a virtual machine to run the Scope App for the user to interact with the system.
For the Scope probe to work with the virtual machines in the cluster, Stev created a new YAML file. You use the YAML file to take advantage of a BOSH feature for runtime configuration: the ability to define add-ons for the running virtual machines. This allows the Scope probe to co-locate and run on all of the virtual machines in the cluster once you re-deploy the cluster.
Stev demoed how the Weave Scope App was deployed to the cluster. He showed the details of the VM running the Scope web server for data display:
Then to re-deploy the cluster with the Scope probe running, follow these steps below:
With this integration, a user is now able to use Weave Scope to look at different aspects of the Cloud Foundry cluster such as the interconnection of virtual machines, the status, and vitals of individual virtual machines.
You can use the open-source Weave Scope App running on the cluster locally at the infrastructure hosting Cloud Foundry. Weave Scope is rich in features. Besides the graphical view, it can also provide table views for some of the information.
But you can also send data to Weaveworks’ commercial Weave Cloud product for added functionality such as stored data over time:
Weave Scope plugin
Stev also used the Weave Scope plugin to gather BOSH-specific information such as deployment ID, index, network info, package information, and BOSH template information.
If you want to gather data for containers running in Cloud Foundry, the Weave Scope plugin is also useful. The container format in Cloud Foundry is somewhat different than the format for Docker containers. Regardless, the plugin allows access to Cloud Foundry container-specific information:
Not only does the Weave Scope plugin show the containers in Cloud Foundry, it can also display what applications are running in the containers.
Weave Scope Release
You can check out the Weave Scope Release for Cloud Foundry on GitHub. This is still work in progress and is not ready to be used in production, but Stev and Weaveworks Head of Developer Experience Luke Marsden are actively working on updates. Stay tuned for more information toward the end of summer.
We also have a recording of the full talk for you available:
Thank you for reading our blog. We build Weave Cloud, which is a hosted add-on to your clusters. It helps you iterate faster on microservices with continuous delivery, visualization & debugging, and Prometheus monitoring to improve observability.