Monasca comes to Kolla

Back near the dawn of time in December, 2016, Sam Yaple created a spec to add Monasca containers to Kolla. Aeons later Kolla-Ansible finally supports deploying Monasca out-of-the-box. Much like crossing the Magic Roundabout in Swindon, many things had to line up to make it happen. The ground was paved by adding support for Apache Kafka, Zookeeper, Storm and Logstash. Then came the Monasca services, rolled out one-by-one until the Fluentd firehose was coupled up to the Monasca Log API. The CI system creaked, the lights went dim and the core reviewers groaned as Zuul unleashed a colossal chunk of Ansible. No longer did one have to carefully deploy, configure and maintain an uncountable number of services. Injurious crashes were reduced by three quarters and sanity returned to the Monasca sysadmins. So what exactly did the end result look like?

Monasca overview

At this stage you might be thinking that Graphviz has just exploded on your screen, or even, has anyone keeled over and died from looking at the diagram? But if you defocus your eyes a little further, you'll see that there can actually be three, or even more of everything. Three APIs, three instances of almost anything you can see, with traffic pirouetting through a Kafka cluster in between. The only things which spoil the fun are InfluxDB which requires an enterprise license for clustering, and the Monasca fork of Grafana, which just doesn't seem to play nicely with load balancing.

So what is it like to run this monster in production? Does it deliver? Why on Earth would you want to do it? We actually have some compelling reasons which we'll summarise below:

  • Horizontally scalable
    We love working with small deployments, and supporting these matters greatly to us, but in the world of HPC, machines can get really huge. Indeed, it's not uncommon for small deployments to morph into large ones, and with Monasca, no matter where you start, you can seamlessly scale with demand.
  • Multi-tenant
    Add value to your OpenStack deployment. Through the power of automation it's true that you could stamp out a monitoring and logging solution per tenant without too much fuss. However, it's hard to beat simply logging in via a public endpoint with your OpenStack credentials.
  • Highly available / fault tolerant
    Kolla Monasca has been designed to provide a single pane of glass for monitoring the health of your OpenStack deployment. If a wheel falls off, we don't want you scrambling for the spare tyre. All critical monitoring and logging services can be deployed in a highly available and fault tolerant configuration.
  • Support for push-metrics
    In big systems there are often complex interactions and understanding these is part of the art of HPC. What's more, complex interactions don't tend to happen at fixed time intervals. Support for push-metrics allows users to stream batches of data into Monasca with a sampling frequency of whatever they like. So whether you're tuning traffic flows in your network fabric, or optimising your MPI routine, Monasca has you covered.

So without further ado, we're going to hand you over to the Kolla documentation. Unlike the Magic Roundabout you'll have two paths to follow: The brave can enable Monasca in their existing Kolla Ansible deployment, and the cautious can choose to deploy Monasca standalone and integrate, if they wish, with an external instance of Keystone which doesn't need to be provided by Kolla. We hope that you like it, and most of all we hope that you find it useful.