MPI Is Not Perfect … Yet

By Daniel Holmes

August 3, 2016

The Message Passing Interface (MPI) is the standard definition of a communication API that has underpinned traditional HPC for decades. The message passing programming represents distributed-memory hardware architectures using processes that send messages to each other. When first standardised in 1993-4, MPI was a major step forward from the many proprietary, system-dependent, and semantically different message-passing libraries that came before it.

However, software engineering has moved on since the 1990s, and so have computer hardware, languages, and mass-market expectations. Is MPI still relevant? Do we need something else to augment it or to replace it?

Last year, Jonathan Dursi wrote a popular blog post entitled “HPC is dying and MPI is killing it”.

Rather than dying, I believe HPC is expanding, evolving, and fragmenting. And MPI can, and should, play a significant role in nurturing the new HPC into being.

Dursi had a few misconceptions on MPI but also some very valid and valuable insight. During the months after that blog post, Dursi interacted with Jeff Squyres; an active member of the MPI Forum and core developer of the Open MPI library. Squyres challenged him on various points and asked for further information on his perceptions of MPI. Dursi has written several follow-up blog posts, both positive and negative towards MPI (http://www.dursi.ca).

For me, these exchanges highlight the need to involve the wider community in the development of MPI. As an active member of the MPI Forum myself, and having written an implementation of the MPI API in C# for my PhD, I need that perspective from outside the already-converted MPI community. We often respond to the stuff below MPI (hardware and system software) but rarely to the stuff above MPI, to which we need to start paying more attention. There is a lot of useful information in those blogs and the discussions that happened offline. There is general appeal that the community should be aware of and partake in. This discussion needs to continue.

I want MPI to be challenged. Especially at the MPI Forum and the EuroMPI conference, the focus is often on how good MPI is: all the problems that are perceived with it are not really MPI’s fault. If only people knew more about to use MPI properly, then they would be singing its praises too. But MPI is not actually perfect – why would the MPI Forum meet four times a year if it were? Why do 20 organisations pay to send representatives? What are we talking about? Why do we still need the EuroMPI conference series?

This year I am the Programme Chair for EuroMPI 2016, so I should have an answer for at least the last of those questions!

I set the theme for this year’s conference as “Modern Challenges to MPI’s Dominance in HPC” and built a programme that allows the MPI community to begin to identify, and to meet, those challenges.

I invited Jonathan Dursi as a keynote speaker because, through that blog, he put his head above the parapet and started a discussion that we urgently need to have. To broaden the ‘from outside MPI’ perspective, I invited experienced technology journalist Nicole Hemsoth.

I am relying on Bill Gropp’s keynote talk to defend MPI – MPI was, is, and always will be, a fantastic thing, both good and necessary for HPC. Both the MPI Standard and the MPI library implementations, embody a wealth of knowledge, experience, thought, and innovation over decades, including before it was called MPI. It would be a huge disservice, and a big mistake, to just throw it away.

Kathryn Mohror, chair of the Tools working group in the MPI Forum, and David Lecomber, CEO of Allinea Software Ltd, focus on one of the most user-oriented aspects of MPI, its support for tools that aid development, debugging, and optimisation.

The key discussion I want to take place is between the establishment and the radicals, on what MPI is right now, and the different perspectives on what is needed for the future.

Why doesn’t Google use MPI for search, or Facebook in their datacentres? Why isn’t MPI used in high frequency trading? These are all things where high performance communication and extreme scale matter.

We heard at ISC this year that the AI/Deep Learning/Machine Learning community have discovered MPI and are starting to use it and rely on it. Why did this take so long? Why have they only just discovered it? It has been around since the 90s. Is it really the best tool for the job or just the closest thing to hand?

We need these opinions to be heard so that MPI can address the current and future challenges of high-end computing.

MPI is already changing.  Every year there are optimisations of existing functions. Every year we talk about tweaks to the API to make it slightly better, more understandable, or less ambiguous. But the MPI Forum is also talking about three much bigger new concepts: endpoints, fault-tolerance and sessions.

MPI Endpoints is a reaction to the increasing number of cores and threads in a shared memory node (below MPI) but also to the increased usage of shared memory programming models in concert with MPI (same level as MPI). Fault tolerance is a response to the decreasing mean time before failure with the ever-increasing system scale (below MPI) but also a realisation that MPI cannot just take the whole system down at the first fault when there are other components that can continue (above MPI). MPI Sessions is about managing MPI’s interaction with increasingly complex runtimes (below MPI), but is also a response to modern programming techniques, such as modularisation using libraries (above MPI).

As with many previous changes in MPI these changes respond to increasing complexity and capability of hardware and system-level software. They are responses to the stuff below MPI, but they are also mindful of the stuff above MPI.

I hope that EuroMPI this year will be a crucible for these ideas, a meeting-point between MPI’s past and its future.

Everyone who is involved in HPC should have an interest in the outcome.

I hope that the discussions held at this meeting will influence the path that MPI takes and it is vital that everyone has a voice in this.

About EuroMPI

http://www.eurompi2016.ed.ac.uk/

Now in its 23rd year, EuroMPI is the leading conference for users and developers of MPI. The aim of this conference is to bring together all of the stakeholders involved in developments and applications related to the Message Passing Interface (MPI). As the preeminent meeting for users, developers and researchers to interact and discuss new developments and applications of message-passing parallel computing, the annual meeting has a long, rich tradition. This year’s conference will be held in Edinburgh, Scotland, UK from 25th to 28th September 2016.

There is still an opportunity to contribute to the technical program content by submitting a poster by 26th August 2016. The organisers particularly encourage those who use MPI and work with it every-day but would not otherwise consider themselves part of the traditional EuroMPI community to submit as we wish to hear this group’s experience of MPI.

About the Author

An Applications Consultant in HPC Research at the EPCC, The University of Edinburgh, Daniel Holmes’ research focuses on the message passing interface (MPI) and interoperability with other programming models and systems. His work involves implementing MPI libraries in a variety of different languages, designing and implementing new features for effective message passing and improving basic parallelism strategies in a range of situations from shared memory to PGAS systems. He is an active member of the MPI Forum, chairing a working group and chapter committee. Prior to his research career Daniel gained twelve years experience as an IT professional in a wide variety of companies and industries.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Nvidia Touts Strong Results on Financial Services Inference Benchmark

February 3, 2023

The next-gen Hopper family may be on its way, but that isn’t stopping Nvidia’s popular A100 GPU from leading another benchmark on its way out. This time, it’s the STAC-ML inference benchmark, produced by the Securi Read more…

Quantum Computing Firm Rigetti Faces Delisting

February 3, 2023

Quantum computing companies are seeing their market caps crumble as investors patiently await out the winner-take-all approach to technology development. Quantum computing firms such as Rigetti Computing, IonQ and D-Wave went public through mergers with blank-check companies in the last two years, with valuations at the time of well over $1 billion. Now the market capitalization of these companies are less than half... Read more…

US and India Strengthen HPC, Quantum Ties Amid Tech Tension with China

February 2, 2023

Last May, the United States and India announced the “Initiative on Critical and Emerging Technology” (iCET), aimed at expanding the countries’ partnerships in strategic technologies and defense industries across th Read more…

Pittsburgh Supercomputing Enables Transparent Medicare Outcome AI

February 2, 2023

Medical applications of AI are replete with promise, but stymied by opacity: with lives on the line, concerns over AI models’ often-inscrutable reasoning – and as a result, possible biases embedded in those models Read more…

Europe’s LUMI Supercomputer Has Officially Been Accepted

February 1, 2023

“LUMI is officially here!” proclaimed the headline of a blog post written by Pekka Manninen, director of science and technology for CSC, Finland’s state-owned IT center. The EuroHPC-organized supercomputer’s most Read more…

AWS Solution Channel

Shutterstock 2069893598

Cost-effective and accurate genomics analysis with Sentieon on AWS

This blog post was contributed by Don Freed, Senior Bioinformatics Scientist, and Brendan Gallagher, Head of Business Development at Sentieon; and Olivia Choudhury, PhD, Senior Partner Solutions Architect, Sujaya Srinivasan, Genomics Solutions Architect, and Aniket Deshpande, Senior Specialist, HPC HCLS at AWS. Read more…

Microsoft/NVIDIA Solution Channel

Shutterstock 1453953692

Microsoft and NVIDIA Experts Talk AI Infrastructure

As AI emerges as a crucial tool in so many sectors, it’s clear that the need for optimized AI infrastructure is growing. Going beyond just GPU-based clusters, cloud infrastructure that provides low-latency, high-bandwidth interconnects and high-performance storage can help organizations handle AI workloads more efficiently and produce faster results. Read more…

Intel’s Gaudi3 AI Chip Survives Axe, Successor May Combine with GPUs

February 1, 2023

Intel's paring projects and products amid financial struggles, but AI products are taking on a major role as the company tweaks its chip roadmap to account for more computing specifically targeted at artificial intellige Read more…

Quantum Computing Firm Rigetti Faces Delisting

February 3, 2023

Quantum computing companies are seeing their market caps crumble as investors patiently await out the winner-take-all approach to technology development. Quantum computing firms such as Rigetti Computing, IonQ and D-Wave went public through mergers with blank-check companies in the last two years, with valuations at the time of well over $1 billion. Now the market capitalization of these companies are less than half... Read more…

US and India Strengthen HPC, Quantum Ties Amid Tech Tension with China

February 2, 2023

Last May, the United States and India announced the “Initiative on Critical and Emerging Technology” (iCET), aimed at expanding the countries’ partnership Read more…

Intel’s Gaudi3 AI Chip Survives Axe, Successor May Combine with GPUs

February 1, 2023

Intel's paring projects and products amid financial struggles, but AI products are taking on a major role as the company tweaks its chip roadmap to account for Read more…

Roadmap for Building a US National AI Research Resource Released

January 31, 2023

Last week the National AI Research Resource (NAIRR) Task Force released its final report and roadmap for building a national AI infrastructure to include comput Read more…

PFAS Regulations, 3M Exit to Impact Two-Phase Cooling in HPC

January 27, 2023

Per- and polyfluoroalkyl substances (PFAS), known as “forever chemicals,” pose a number of health risks to humans, with more suspected but not yet confirmed Read more…

Multiverse, Pasqal, and Crédit Agricole Tout Progress Using Quantum Computing in FS

January 26, 2023

Europe-based quantum computing pioneers Multiverse Computing and Pasqal, and global bank Crédit Agricole CIB today announced successful conclusion of a 1.5-yea Read more…

Critics Don’t Want Politicians Deciding the Future of Semiconductors

January 26, 2023

The future of the semiconductor industry was partially being decided last week by a mix of politicians, policy hawks and chip industry executives jockeying for Read more…

Riken Plans ‘Virtual Fugaku’ on AWS

January 26, 2023

The development of a national flagship supercomputer aimed at exascale computing continues to be a heated competition, especially in the United States, the Euro Read more…

Leading Solution Providers

Contributors

SC22 Booth Videos

AMD @ SC22
Altair @ SC22
AWS @ SC22
Ayar Labs @ SC22
CoolIT @ SC22
Cornelis Networks @ SC22
DDN @ SC22
Dell Technologies @ SC22
HPE @ SC22
Intel @ SC22
Intelligent Light @ SC22
Lancium @ SC22
Lenovo @ SC22
Microsoft and NVIDIA @ SC22
One Stop Systems @ SC22
Penguin Solutions @ SC22
QCT @ SC22
Supermicro @ SC22
Tuxera @ SC22
Tyan Computer @ SC22
  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire