Co-Design Center Develops Next-Gen Simulation Tools

By Scott Gibson, Oak Ridge National Laboratory

November 8, 2017

Editor’s Note: Delivering concrete tools to make exascale computing both practical and worthwhile is one of the U.S. Exascale Computing Project’s imperatives. This article from ECP examines the need for and early output from the Center for Efficient Exascale Discretizations, one of ECP’s five co-design centers.   

The exascale architectures on the horizon for supercomputing pose intricate challenges to the developers of large-scale scientific applications. Running efficiently at exascale will not only require porting and optimizing code but also rethinking the design of core algorithms to take advantage of the greater parallelism and more efficient calculation capabilities that exascale systems will offer.

With the aim of efficiently exploiting exascale resources, the Center for Efficient Exascale Discretizations (CEED) of the Exascale Computing Project (ECP) is working closely with a wide variety of application scientists, vendors, and software technology developers to create highly optimized discretization libraries and next-generation mini apps based on what are called advanced high-order finite element methods.

CEED is one of five ECP co-design centers established to enable developers of software technology, hardware technology, and computational science applications to forge multidisciplinary collaborations.

The co-design centers are targeting the most important computational algorithms and patterns of communication in scientific research, referred to as scientific application motifs. By taking advantage of sophisticated mathematical algorithms and advanced hardware such as GPUs, for example, the high-order methods being developed in CEED are delivering improved simulation quality without an increase in computational time.

The computational motif addressed by CEED pertains to discretization, the process of dividing a large simulation into smaller components (finite elements) in preparation for computer analysis. Computer modeling and simulation of physical phenomena such as fluid flow, vibration, and heat is central to understanding the behavior of real-world systems such as solar, wind, combustion, and nuclear fusion and fission. Meshes (or grids) that represent the domain of interest (a wind farm, for example) are created for finite element analysis, in which the equations governing the physics are approximated by polynomials on each element.

CEED’s focus is on efficient implementations of high-order polynomial approximations on unstructured meshes. These methods require advanced mathematical theory and algorithms, but can provide high-fidelity approximations of the fields inside each element, leading to higher-quality solutions with fewer unknown coefficients to be determined. The resulting smaller linear systems need less data movement, which is a critical performance feature of high-performance computing today and on future exascale systems, allowing the CEED algorithms to better exploit the hardware and deliver a significant performance gain over conventional low-order methods, said CEED director Tzanio Kolev of Lawrence Livermore National Laboratory (LLNL).

Members of the Center for Efficient Exascale Discretizations (CEED) team gathered in August at Lawrence Livermore National Laboratory for the center’s first meeting with representatives of projects, vendors, and industry associated with the Exascale Computing Project. CEED plans to have the meeting annually. (Image credit: CEED)

To serve as an integration hub for the ECP efforts, CEED is organized around four interconnected research-and-development thrusts focused on its customers: applications, hardware, software, and finite elements, which ties together and coordinates the efforts of the other three thrusts.

The center is assisting the applications groups through the development of discretization libraries that extract maximal performance out of new hardware as it is deployed and provides this performance to multiple applications without duplication of development effort. Members of the CEED team are designated as liaisons to selected applications, ensuring that the center accounts for their needs and that algorithmic improvements are integrated into applications when the new machines are delivered.

“CEED helps ECP applications by providing them with leading-edge simulation algorithms that can extract much more of the performance from exascale hardware than what’s currently available,” Kolev said.

The first wave of ECP application projects with which CEED has been actively collaborating includes ExaSMR, a coupled Monte Carlo neutronics and fluid flow simulation tool of small modular nuclear reactors, and MARBL, a next-generation multiphysics code at LLNL.

“A discretization library can help a wide variety of applications to bridge the gap to exascale hardware,” Kolev said. “Once we’ve made the connection and the library has been integrated into an application, we can quickly deliver new benefits, such as improved kernels [the core components of numerical algorithms], in the future by upgrading the application to an updated version of the library.”

In addition to libraries of highly performant kernels, key products of the CEED project are new and improved mini apps designed to highlight performance critical paths and provide simple examples of meaningful high-order computations.

The mini apps can be used for interactions with vendors and software technologies projects, and for procurement activities. Two of the mini apps that CEED has developed for those purposes are called Nekbone and Laghos, which are part of the recently released ECP Proxy Applications Suite 1.0. Nekbone and Laghos represent subcomponents of interest from ExaSMR and MARBL.

The Nekbone mini app has a long history, including use in a recent procurement to represent incompressible flow simulations with implicit time stepping. Under CEED, Nekbone is being updated to run on GPUs, an important component of the Titan supercomputer and the upcoming Summit and Sierra machines.

In contrast, Laghos is a new mini app developed in CEED as a proxy for compressible flow simulations with explicit time stepping. “Laghos consists of a particular class of algorithms that pertains to compressible flow with unstructured moving meshes that represents interactions of materials and shock waves as extreme densities and pressures,” Kolev said. “This is the first time we’ve had a really good mini app that captures high-order discretizations for these types of problems. Laghos is important for the activities at many of the National Nuclear Security Administration labs, and we have a lot of interest already from vendors who want to optimize the mini app.”

(The video of a simulation below is an example of the type of high-order calculations that CEED addresses. Shown is a Lagrangian compressible hydrodynamics simulation of triple-point shock interaction in axisymmetric coordinates. The new Laghos mini app is the first proxy for these types of problems. For more details and additional examples of CEED-related simulations, see CEED Publications and OutreachBLAST: High-Order Finite Element Hydrodynamics,  NEK5000, and MFEM. (Simulation movie credit: CEED))

 

Kolev described CEED’s interactions with vendors as a two-way connection. The center is working closely with them to ensure its discretization libraries will run efficiently on the hardware in development. At the same time, CEED is providing feedback to the vendors concerning hardware changes that can improve the performance of high-order algorithms.

“We represent many physicists and applications scientists when we interact with vendors,” Kolev said. “We learn from them how to make low-level optimizations and discover which memory models and programming models are good for our libraries’ algorithms. But we also advocate to the vendors that they should consider certain types of algorithms in their designs.”

When exascale becomes a reality, CEED wants high-order application developers to feel comfortable that they have full support for all tools that are part of their simulation pipeline.

“That means we’re thinking not only about discretization but also meshing, visualization, and the solvers that work with high order at exascale,” Kolev said. “We are collaborating with the teams in the ECP’s Software Technology focus area to develop new mathematical algorithms and expand the tools they are developing to meet the needs of CEED’s computational motif.”

CEED has also proposed important lower-level benchmarking problems (known as bake-off problems) to compare different high-order approaches and engage with the external high-order community via GitHub.

“These problems are designed to exercise really hot kernels that are central to the performance of high-order methods,” Kolev said. “This is an activity we use internally to push each other and learn from each other and make sure we are delivering to the users the best possible performance based on our collective experience. Engaging with the community is a win-win for everyone, and we are already starting to do this: a project in Germany picked up our benchmark problems from GitHub, and ran some very interesting tests. The engagement is ultimately going to benefit the applications because we are going to use the kernels that stem from such interactions in our libraries.”

CEED is a research partnership of two US Department of Energy labs and five universities. The partners are Lawrence Livermore National Laboratory; Argonne National Laboratory; the University of Illinois Urbana-Champaign; Virginia Tech; the University of Tennessee, Knoxville; the University of Colorado, Boulder; and the Rensselaer Polytechnic Institute. The center held its first project meeting in August at LLNL, which it plans to hold annually. The meeting brought together more than 50 researchers from ECP projects, vendors, and industry.

Link to the original article on the ECP web site: https://www.exascaleproject.org/co-design-center-develops-next-generation-simulation-libraries-and-mini-apps/

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

GTC21: Dell Building Cloud Native Supercomputers at U Cambridge and Durham

April 14, 2021

In conjunction with GTC21, Dell Technologies today announced new supercomputers at universities across DiRAC (Distributed Research utilizing Advanced Computing) in the UK with plans to explore use of Nvidia BlueField DPU Read more…

The Role and Potential of CPUs in Deep Learning

April 14, 2021

Deep learning (DL) applications have unique architectural characteristics and efficiency requirements. Hence, the choice of computing system has a profound impact on how large a piece of the DL pie a user can finally enj Read more…

GTC21: Nvidia Launches cuQuantum; Dips a Toe in Quantum Computing

April 13, 2021

Yesterday Nvidia officially dipped a toe into quantum computing with the launch of cuQuantum SDK, a development platform for simulating quantum circuits on GPU-accelerated systems. As Nvidia CEO Jensen Huang emphasized i Read more…

Nvidia Aims Clara Healthcare at Drug Discovery, Imaging via DGX

April 12, 2021

Nvidia Corp. continues to expand its Clara healthcare platform with the addition of computational drug discovery and medical imaging tools based on its DGX A100 platform, related InfiniBand networking and its AGX develop Read more…

Nvidia Serves Up Its First Arm Datacenter CPU ‘Grace’ During Kitchen Keynote

April 12, 2021

Today at Nvidia’s annual spring GPU technology conference, held virtually once more due to the ongoing pandemic, the company announced its first ever Arm-based CPU, called Grace in honor of the famous American programmer Grace Hopper. Read more…

AWS Solution Channel

Volkswagen Passenger Cars Uses NICE DCV for High-Performance 3D Remote Visualization

 

Volkswagen Passenger Cars has been one of the world’s largest car manufacturers for over 70 years. The company delivers more than 6 million automobiles to global customers every year, from 50 production locations on five continents. Read more…

Nvidia Debuts BlueField-3 – Its Next DPU with Big Plans for an Expanded Role

April 12, 2021

Nvidia today announced its next generation data processing unit (DPU) – BlueField-3 – adding more substance to its evolving concept of the DPU as a full-fledged partner to CPUs and GPUs in delivering advanced computi Read more…

GTC21: Dell Building Cloud Native Supercomputers at U Cambridge and Durham

April 14, 2021

In conjunction with GTC21, Dell Technologies today announced new supercomputers at universities across DiRAC (Distributed Research utilizing Advanced Computing) Read more…

The Role and Potential of CPUs in Deep Learning

April 14, 2021

Deep learning (DL) applications have unique architectural characteristics and efficiency requirements. Hence, the choice of computing system has a profound impa Read more…

Nvidia Serves Up Its First Arm Datacenter CPU ‘Grace’ During Kitchen Keynote

April 12, 2021

Today at Nvidia’s annual spring GPU technology conference, held virtually once more due to the ongoing pandemic, the company announced its first ever Arm-based CPU, called Grace in honor of the famous American programmer Grace Hopper. Read more…

Nvidia Debuts BlueField-3 – Its Next DPU with Big Plans for an Expanded Role

April 12, 2021

Nvidia today announced its next generation data processing unit (DPU) – BlueField-3 – adding more substance to its evolving concept of the DPU as a full-fle Read more…

Nvidia’s Newly DPU-Enabled SuperPod Is a Multi-Tenant, Cloud-Native Supercomputer

April 12, 2021

At GTC 2021, Nvidia has announced an upgraded iteration of its DGX SuperPods, calling the new offering “the first cloud-native, multi-tenant supercomputer.” Read more…

Tune in to Watch Nvidia’s GTC21 Keynote with Jensen Huang – Recording Now Available

April 12, 2021

Join HPCwire right here on Monday, April 12, at 8:30 am PT to see the Nvidia GTC21 keynote from Nvidia’s CEO, Jensen Huang, livestreamed in its entirety. Hosted by HPCwire, you can click to join the Huang keynote on our livestream to hear Nvidia’s expected news and... Read more…

The US Places Seven Additional Chinese Supercomputing Entities on Blacklist

April 8, 2021

As tensions between the U.S. and China continue to simmer, the U.S. government today added seven Chinese supercomputing entities to an economic blacklist. The U Read more…

Habana’s AI Silicon Comes to San Diego Supercomputer Center

April 8, 2021

Habana Labs, an Intel-owned AI company, has partnered with server maker Supermicro to provide high-performance, high-efficiency AI computing in the form of new Read more…

Julia Update: Adoption Keeps Climbing; Is It a Python Challenger?

January 13, 2021

The rapid adoption of Julia, the open source, high level programing language with roots at MIT, shows no sign of slowing according to data from Julialang.org. I Read more…

Intel Launches 10nm ‘Ice Lake’ Datacenter CPU with Up to 40 Cores

April 6, 2021

The wait is over. Today Intel officially launched its 10nm datacenter CPU, the third-generation Intel Xeon Scalable processor, codenamed Ice Lake. With up to 40 Read more…

CERN Is Betting Big on Exascale

April 1, 2021

The European Organization for Nuclear Research (CERN) involves 23 countries, 15,000 researchers, billions of dollars a year, and the biggest machine in the worl Read more…

Programming the Soon-to-Be World’s Fastest Supercomputer, Frontier

January 5, 2021

What’s it like designing an app for the world’s fastest supercomputer, set to come online in the United States in 2021? The University of Delaware’s Sunita Chandrasekaran is leading an elite international team in just that task. Chandrasekaran, assistant professor of computer and information sciences, recently was named... Read more…

HPE Launches Storage Line Loaded with IBM’s Spectrum Scale File System

April 6, 2021

HPE today launched a new family of storage solutions bundled with IBM’s Spectrum Scale Erasure Code Edition parallel file system (description below) and featu Read more…

10nm, 7nm, 5nm…. Should the Chip Nanometer Metric Be Replaced?

June 1, 2020

The biggest cool factor in server chips is the nanometer. AMD beating Intel to a CPU built on a 7nm process node* – with 5nm and 3nm on the way – has been i Read more…

Saudi Aramco Unveils Dammam 7, Its New Top Ten Supercomputer

January 21, 2021

By revenue, oil and gas giant Saudi Aramco is one of the largest companies in the world, and it has historically employed commensurate amounts of supercomputing Read more…

Quantum Computer Start-up IonQ Plans IPO via SPAC

March 8, 2021

IonQ, a Maryland-based quantum computing start-up working with ion trap technology, plans to go public via a Special Purpose Acquisition Company (SPAC) merger a Read more…

Leading Solution Providers

Contributors

Can Deep Learning Replace Numerical Weather Prediction?

March 3, 2021

Numerical weather prediction (NWP) is a mainstay of supercomputing. Some of the first applications of the first supercomputers dealt with climate modeling, and Read more…

Livermore’s El Capitan Supercomputer to Debut HPE ‘Rabbit’ Near Node Local Storage

February 18, 2021

A near node local storage innovation called Rabbit factored heavily into Lawrence Livermore National Laboratory’s decision to select Cray’s proposal for its CORAL-2 machine, the lab’s first exascale-class supercomputer, El Capitan. Details of this new storage technology were revealed... Read more…

New Deep Learning Algorithm Solves Rubik’s Cube

July 25, 2018

Solving (and attempting to solve) Rubik’s Cube has delighted millions of puzzle lovers since 1974 when the cube was invented by Hungarian sculptor and archite Read more…

African Supercomputing Center Inaugurates ‘Toubkal,’ Most Powerful Supercomputer on the Continent

February 25, 2021

Historically, Africa hasn’t exactly been synonymous with supercomputing. There are only a handful of supercomputers on the continent, with few ranking on the Read more…

The History of Supercomputing vs. COVID-19

March 9, 2021

The COVID-19 pandemic poses a greater challenge to the high-performance computing community than any before. HPCwire's coverage of the supercomputing response t Read more…

AMD Launches Epyc ‘Milan’ with 19 SKUs for HPC, Enterprise and Hyperscale

March 15, 2021

At a virtual launch event held today (Monday), AMD revealed its third-generation Epyc “Milan” CPU lineup: a set of 19 SKUs -- including the flagship 64-core, 280-watt 7763 part --  aimed at HPC, enterprise and cloud workloads. Notably, the third-gen Epyc Milan chips achieve 19 percent... Read more…

HPE Names Justin Hotard New HPC Chief as Pete Ungaro Departs

March 2, 2021

HPE CEO Antonio Neri announced today (March 2, 2021) the appointment of Justin Hotard as general manager of HPC, mission critical solutions and labs, effective Read more…

Microsoft, HPE Bringing AI, Edge, Cloud to Earth Orbit in Preparation for Mars Missions

February 12, 2021

The International Space Station will soon get a delivery of powerful AI, edge and cloud computing tools from HPE and Microsoft Azure to expand technology experi Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire