Distinguished Member of the Technical Staff
Scalable Algorithms Department
Computing Research Center
In an effort to develop a more relevant metric for ranking HPC systems, Dr. Michael Heroux of Sandia National Laboratory has joined forces with LINPACK creator Dr. Jack Dongarra to create the High Performance Conjugate Gradients (HPCG) benchmark that is expected to further come into its own this year.
The goals of the new benchmark are to address the primary shortcoming of LINPACK, wherein real application performance is seen as increasingly inconsistent with the narrow scope of the benchmark. In addition, HPCG aims to stress the balance between floating point and communication bandwidth and latency, and to tighten the focus on messaging, memory, and parallelization.
As the benchmark continues to evolve, the team is not looking for the new benchmark to upend LINPACK, but rather to act as a complementary bookend aligned with actual application performance.
HPCwire: Hello Michael. You have been closely involved in work with the prolific Dr. Jack Dongarra creating a new benchmark called the High Performance Conjugate Gradients (HPCG) that is set to make waves this year. A benchmark shift has been talked about in HPC circles for a long time – what have been the biggest challenges to get to this point, and are we finally there?
The biggest challenges have been to create a simple, easy-to-understand benchmark that provides meaningful information that complements LINPACK and assures community buy-in. We have worked closely with many members of the HPC community in determining our basic strategy and implementation, including workshops and many detailed technical conversations. We also established working relationships with vendors from the beginning. These conversations have led to numerous improvements, and we expect to continue making adjustments in HPCG over the next year or so, even as we build up the list of ranked systems.
We still have some challenges. As you might imagine, there are many people who have opinions about benchmarking the TOP 500 systems, and whether or not there should even be such a list. Articulating the importance of the TOP500 list, the relevance and longevity of HPCG, and the value of the LINPACK-HPCG combination is essential. However, the rapid growth in submissions, the availability of optimized versions of HPCG and the interesting results from the present collection of HPCG runs give us strong reasons to be optimistic.
HPCwire: How does the HPCG benchmark differ from LINPACK? The new benchmark is said to be complimentary, not competitive with LINPACK. Can you explain a little bit about the rationale behind this, and the value of using both benchmarks?
HPCG presents numerous challenges to HPC systems beyond LINPACK. First, HPCG stresses the need for high-performance memory systems. Sparse operations, present in many HPC applications, are strongly impacted by memory system performance. Everyone agrees that HPCG does a good job of representing this major requirement, while LINPACK does not. Some people have prematurely determined that HPCG is essentially a memory bandwidth benchmark, but there is much more to it.
Optimal performance from HPCG also requires fast, well-balanced global collective capabilities across the entire computer system. HPCG also needs excellent point-to-point communication for executing so-called neighborhood collectives. Many of our highly engineered systems support both of these requirements well, but there are some interesting anomalies, and lower quality systems can be extremely challenged by these operations.
Lesser known system features that HPCG probes are fine-grain collaborative threading capabilities, which are essential as we continue to rely more heavily on massive concurrency for performance improvements. Optimized implementations of HPCG are starting to explore this aspect of system performance, with very interesting results. The ability for thread-parallel computations to interact with each other on a fine-grain computation is essential to addressing strong scaling performance requirements that many applications will have in the future. HPCG rewards investments in these types of features, while LINPACK does not.
Regarding LINPACK-HPCG complementarity, to be honest, when we started the HPCG effort, we expected that we would eventually replace LINPACK. But we have come to realize anew the tremendous historical value of LINPACK. Furthermore, with the complementarity that HPCG provides, LINPACK can now be viewed as a high-water performance mark, while HPCG is close to the low-water mark. For many application teams, they will be able to take the LINPACK-HPCG pair of results for a given system and derive an approximate performance estimate for their own application somewhere in between.
HPCwire: In terms of ranking systems, can you offer any insight into how the HPCG benchmark might shake things up? Are there any overnight winners or losers in the Top500 based on the new standard?
So far we have seen some interesting re-ranking of systems, with systems that are generally deemed as “more balanced” appearing higher on the list than what LINPACK dictates. Also, we see a general compression across the top tier of systems, which is again something we see in many real applications.
I don’t see that we will have overnight winners or losers. For the most part, despite the high profile of getting a good LINPACK number, HPC systems are designed to run real applications. What HPCG does is confirm the investment in system features that are intended for improving real application performance and provide further incentive as we go forward. The real impact of HPCG will be from the increased attention we are seeing in both the hardware and software features that make HPCG, and many applications, faster. I am very encouraged by the changes in emphasis we have seen so far.
HPCwire: On a more intimate level, what can you tell us about yourself – personal life, family, background, hobbies?
My personal life revolves around my family and rural living. I grew up in a small farming community and today I live in the woods where my wife and I are raising our children. I enjoy fishing, swimming, cutting wood and making maple syrup. I also enjoy working with junior members of the scientific community. I teach and mentor at all education levels and consider it a privilege to do so.
HPCwire: One last question – What can you share about yourself that you can share that you think your colleagues would be surprised to learn?
Although I work for Sandia National Laboratories in Albuquerque, New Mexico, I live 1,000 miles away in rural central Minnesota, which helps make sense of my personal interests and hobbies. I have always worked remotely for Sandia, while teaching part-time as a Scientist in Residence at St. John’s University, near my home.