HPCAC Conference End Note: The Case for a Scientific OpenStack

The HPC Advisory Council held its 2017 workshop in Lugano in the beautiful region of Ticino, Switzerland.

InsideHPC has recorded footage and gathered presentation slides from much of the conference, available here. Thanks Rich!

I was able to attend this year, and was thoroughly delighted to do so. The conference content included some stimulating presentations outlining the future directions of HPC. Three general themes were strong in much of the content.

Deep Learning & AI

It is clear that Deep Learning algorithms are transforming many areas of scientific computing. With presentations ranging from the giant corporations (such as IBM) to new and nimble startups (such as DeepCube), there is huge interest and clear potential for where Deep Learning techniques can be applied.

GPUs and Accelerators

If Deep Learning holds great promise for researchers looking for solutions to the problems of modern science, it also holds great promise for the bottom line of companies developing accelerators such as GPUs and more exotic hardware architectures! The highly computationally-intensive nature of Deep Learning algorithms was the subject of several very interesting talks, including refactoring software with awareness of PCI bus congestion, direct communication between GPUs and HPC networks, and the new OpenCAPI initiative for OpenPOWER.


OpenStack is clearly thriving in this space. DK Panda from Ohio State University presented two compelling keynotes describing their latest work on enhancing MVAPICH2 MPI for virtualisation and big data software stacks. Francis Lam from Huawei talked about HPC hardware, citing OpenStack use cases.

Mike Lowe and Dave Hancock from Indiana University presented the OpenStack journey they took with getting Jetstream into production. With Jetstream's deployment Mike took a very hands-on approach, and as a result they have a system that performs very well for research computing requirements, but is also well understood by Mike and his team at a fundamental level.

Saverio Proto from SWITCH described how they are able to integrate cloud instances into the layer-2 data centre networks of the research faculties they are supporting, giving seamless scalability without any inconvenience to the researchers using the platform.

I had the honour of the closing end note address, and I took the opportunity to lay out our case for using OpenStack for meeting the needs of modern research computing.

Stig talking at HPCAC Lugano 2017

InsideHPC covered it here. Their footage of the presentation is also available on their YouTube channel:

An HPC conference can bring together hot technologies such as accelerators, deep learning and virtualisation, and appraise them in the context of HPC's long tradition of maximising the capability of computation at scale.

An exciting time for StackHPC to be standing at this crossroads!