GAINESVILLE, Va., Oct. 28, 2021 — The Standard Performance Evaluation Corporation (SPEC) today announced a set of new application benchmark suites designed to provide a comprehensive measure of real-world performance for state-of-the-art High Performance Computing (HPC) systems. The SPEChpc 2021 Benchmark Suites provide researchers, developers, hardware and software vendors, compiler vendors, data center operators and end users with fair comparisons of the performance of different HPC systems.

Modern HPC systems are built using numerous different accelerators that add tremendous computing power but also make performance evaluation and characterization difficult. Even more complexity arises from the use of multiple parallel and accelerator programming models, each supporting only a subset of computing devices. The SPEChpc 2021 Benchmark Suites offer a broad selection of science and engineering codes that represent real-world HPC workloads and that are portable across CPUs and accelerators. The suites also include vendor-neutral comparative performance metrics.

With the SPEChpc 2021 Benchmark Suites, developers and researchers can evaluate different programming models to assess which model would be best for their application or system configuration. Hardware and software vendors can use it to stress test their solutions. And compiler vendors can use it to improve general code performance and their support for directive-based programming models. The new suites can also be used by data center operators and other end users to make procurement decisions.

“Building on our experience in developing the SPEC MPI 2007 benchmark, the SPEC OMP 2012 benchmark, and the SPEC ACCEL benchmark suites, SPEC designed a new set of benchmark suites that keeps pace with the rapidly evolving HPC market,” said Ron Lieberman, SPEC High Performance Group (HPG) Chair. “The high portability of the SPEChpc 2021 Benchmark Suites, along with a strict result review process and rich SPEC result repository, enables us to deliver vendor-neutral performance comparisons for evaluating and studying modern HPC platforms.”

Sunita Chandrasekaran, Associate Professor at the University of Delaware and Principal Investigator of the SOLLVE project within the U.S. Department of Energy’s Exascale Computing Project, notes that “The SPEChpc 2021 Benchmark Suites are critically important to the research community as the suites offer a variety of research opportunities. Students can learn a lot from using performance analysis tools to investigate how an application is mapped to the target architecture, how to use profilers, and how to perform code optimizations based on profiler info, therefore addressing the gap between theoretical and achieved performance of a given architecture. Besides these benefits, the suites provide an excellent opportunity to learn about parallel programming models – a topic of utmost importance to the next generation workforce in the field of computer/computational science.”

SPEChpc 2021 Benchmark Suites Features

• A tested set of benchmarks with performance measurement and validation built into the test harness.
  Benchmarks include full and mini applications covering a wide range of scientific domains and Fortran/C/C++ programming languages.
• Comprehensive support for multiple programming models, including MPI, MPI+OpenACC, MPI+OpenMP, and MPI+OpenMP with target offload.
• Support for most major compilers, MPI libraries, and different flavors of Linux operating systems.
• Four suites, Tiny, Small, Medium, and Large, with increasing workload sizes, allows for appropriate evaluation of different sized HPC systems, ranging from a single node to many thousands of nodes.

The SPEChpc 2021 Benchmarks Suites are developed and maintained by the SPEC HPG Committee, which includes representatives from AMD, Argonne National Lab, Bull Atos Technology, Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Hewlett Packard Enterprise, IBM, Indiana University, Intel, Lawrence Berkeley National Laboratory, Lenovo Global Technology, NVIDIA, Oak Ridge National Laboratory, RWTH Aachen University Germany, Stony Brook University USA, Technische Universität Dresden, Texas Advanced Computing Center, the University of Texas at Austin, University of Basel Switzerland and University of Delaware.

Available Now for Download

The SPEChpc 2021 Benchmark Suites are available for immediate download on the SPEC website under a two-tiered pricing structure: free for non-profit and educational organizations and $2,500 for sellers of computer-related products and services. SPEC HPG members receive benchmark licenses as a membership benefit.

About SPEC

SPEC is a non-profit organization that establishes, maintains and endorses standardized benchmarks and tools to evaluate performance for the newest generation of computing systems. Its membership comprises more than 120 leading computer hardware and software vendors, educational institutions, research organizations, and government agencies worldwide.

---

Source: SPEC