We always enjoy attending the UKRI Cloud Working Group Workshop held annually at the
awesome Francis Crick Institute. The
sizeable crowd it draws and the high quality of content are both
healthy signs of the vitality of cloud for research computing.
This year's workshop demonstrated a maturing approach to use of
cloud, with some notable focus on various methods for harnessing
hybrid and public clouds for dynamic and bursting workloads. Public
cloud companies presented on new and forthcoming HPC-aware features,
while research organisations presented on mobility to avoid lock-in
to cloud vendors. How these two contrasting tensions play out will
be interesting over the next few years.
There was also a welcome focus on operating and sustaining cloud-hosted
infrastructure and platforms. In particular, Matt Pryor from
STFC/JASMIN presented their current
project on a user-friendly application portal, coupled with
Cluster-as-a-Service deployments of Slurm and Kubernetes, with focus
on both usability for scientists and day-2 operations for administrators.
StackHPC is proud to be working with the JASMIN team on implementing this
well-considered initiative and we hope to write more about it in due course.
We always particpate as much as possible, and this year StackHPC was
more involved than we have ever been before. Five members of our team
attended, and in a one-day programme three presentations were delivered
by the team - a real achievement for a ten-person company.
We presented three prominent areas of recent work. John Garbutt spoke about our recent work on storage
for the software-defined supercomputer, in particular SKA SDP buffer
prototyping and the Cambridge Data Accelerator.
Pictured here with David Yuan of EMBL and Matt Pryor of STFC
Mark Goddard presented our work on Kayobe, a free and open source
deployment tool for containerised OpenStack control planes, based
on Kolla and
and embodying current best practices. Kayobe is seeing broad
adoption for research computing configurations and use cases.
Bharat Kunwar delivered a demonstration
of Pangeo, the second of the day after Jacob
Tomlinson presented the work of the Met Office Informatics Lab.
With a focus on data-intensive analytics on private cloud infrastructure,
Bharat demonstrated the deployment of Pangeo on a bare metal HPC
OpenStack deployment, using Kubernetes deployed by Magnum. In addition
to demonstrating containers running on bare metal, Bharat demonstrated
storage attachments backed by Ceph and RDMA-enabled BeeGFS. All of that in ten minutes!