Co-Design Center Develops Next-Gen Simulation Tools

By Scott Gibson, Oak Ridge National Laboratory

November 8, 2017

Editor’s Note: Delivering concrete tools to make exascale computing both practical and worthwhile is one of the U.S. Exascale Computing Project’s imperatives. This article from ECP examines the need for and early output from the Center for Efficient Exascale Discretizations, one of ECP’s five co-design centers.   

The exascale architectures on the horizon for supercomputing pose intricate challenges to the developers of large-scale scientific applications. Running efficiently at exascale will not only require porting and optimizing code but also rethinking the design of core algorithms to take advantage of the greater parallelism and more efficient calculation capabilities that exascale systems will offer.

With the aim of efficiently exploiting exascale resources, the Center for Efficient Exascale Discretizations (CEED) of the Exascale Computing Project (ECP) is working closely with a wide variety of application scientists, vendors, and software technology developers to create highly optimized discretization libraries and next-generation mini apps based on what are called advanced high-order finite element methods.

CEED is one of five ECP co-design centers established to enable developers of software technology, hardware technology, and computational science applications to forge multidisciplinary collaborations.

The co-design centers are targeting the most important computational algorithms and patterns of communication in scientific research, referred to as scientific application motifs. By taking advantage of sophisticated mathematical algorithms and advanced hardware such as GPUs, for example, the high-order methods being developed in CEED are delivering improved simulation quality without an increase in computational time.

The computational motif addressed by CEED pertains to discretization, the process of dividing a large simulation into smaller components (finite elements) in preparation for computer analysis. Computer modeling and simulation of physical phenomena such as fluid flow, vibration, and heat is central to understanding the behavior of real-world systems such as solar, wind, combustion, and nuclear fusion and fission. Meshes (or grids) that represent the domain of interest (a wind farm, for example) are created for finite element analysis, in which the equations governing the physics are approximated by polynomials on each element.

CEED’s focus is on efficient implementations of high-order polynomial approximations on unstructured meshes. These methods require advanced mathematical theory and algorithms, but can provide high-fidelity approximations of the fields inside each element, leading to higher-quality solutions with fewer unknown coefficients to be determined. The resulting smaller linear systems need less data movement, which is a critical performance feature of high-performance computing today and on future exascale systems, allowing the CEED algorithms to better exploit the hardware and deliver a significant performance gain over conventional low-order methods, said CEED director Tzanio Kolev of Lawrence Livermore National Laboratory (LLNL).

Members of the Center for Efficient Exascale Discretizations (CEED) team gathered in August at Lawrence Livermore National Laboratory for the center’s first meeting with representatives of projects, vendors, and industry associated with the Exascale Computing Project. CEED plans to have the meeting annually. (Image credit: CEED)

To serve as an integration hub for the ECP efforts, CEED is organized around four interconnected research-and-development thrusts focused on its customers: applications, hardware, software, and finite elements, which ties together and coordinates the efforts of the other three thrusts.

The center is assisting the applications groups through the development of discretization libraries that extract maximal performance out of new hardware as it is deployed and provides this performance to multiple applications without duplication of development effort. Members of the CEED team are designated as liaisons to selected applications, ensuring that the center accounts for their needs and that algorithmic improvements are integrated into applications when the new machines are delivered.

“CEED helps ECP applications by providing them with leading-edge simulation algorithms that can extract much more of the performance from exascale hardware than what’s currently available,” Kolev said.

The first wave of ECP application projects with which CEED has been actively collaborating includes ExaSMR, a coupled Monte Carlo neutronics and fluid flow simulation tool of small modular nuclear reactors, and MARBL, a next-generation multiphysics code at LLNL.

“A discretization library can help a wide variety of applications to bridge the gap to exascale hardware,” Kolev said. “Once we’ve made the connection and the library has been integrated into an application, we can quickly deliver new benefits, such as improved kernels [the core components of numerical algorithms], in the future by upgrading the application to an updated version of the library.”

In addition to libraries of highly performant kernels, key products of the CEED project are new and improved mini apps designed to highlight performance critical paths and provide simple examples of meaningful high-order computations.

The mini apps can be used for interactions with vendors and software technologies projects, and for procurement activities. Two of the mini apps that CEED has developed for those purposes are called Nekbone and Laghos, which are part of the recently released ECP Proxy Applications Suite 1.0. Nekbone and Laghos represent subcomponents of interest from ExaSMR and MARBL.

The Nekbone mini app has a long history, including use in a recent procurement to represent incompressible flow simulations with implicit time stepping. Under CEED, Nekbone is being updated to run on GPUs, an important component of the Titan supercomputer and the upcoming Summit and Sierra machines.

In contrast, Laghos is a new mini app developed in CEED as a proxy for compressible flow simulations with explicit time stepping. “Laghos consists of a particular class of algorithms that pertains to compressible flow with unstructured moving meshes that represents interactions of materials and shock waves as extreme densities and pressures,” Kolev said. “This is the first time we’ve had a really good mini app that captures high-order discretizations for these types of problems. Laghos is important for the activities at many of the National Nuclear Security Administration labs, and we have a lot of interest already from vendors who want to optimize the mini app.”

(The video of a simulation below is an example of the type of high-order calculations that CEED addresses. Shown is a Lagrangian compressible hydrodynamics simulation of triple-point shock interaction in axisymmetric coordinates. The new Laghos mini app is the first proxy for these types of problems. For more details and additional examples of CEED-related simulations, see CEED Publications and OutreachBLAST: High-Order Finite Element Hydrodynamics,  NEK5000, and MFEM. (Simulation movie credit: CEED))

 

Kolev described CEED’s interactions with vendors as a two-way connection. The center is working closely with them to ensure its discretization libraries will run efficiently on the hardware in development. At the same time, CEED is providing feedback to the vendors concerning hardware changes that can improve the performance of high-order algorithms.

“We represent many physicists and applications scientists when we interact with vendors,” Kolev said. “We learn from them how to make low-level optimizations and discover which memory models and programming models are good for our libraries’ algorithms. But we also advocate to the vendors that they should consider certain types of algorithms in their designs.”

When exascale becomes a reality, CEED wants high-order application developers to feel comfortable that they have full support for all tools that are part of their simulation pipeline.

“That means we’re thinking not only about discretization but also meshing, visualization, and the solvers that work with high order at exascale,” Kolev said. “We are collaborating with the teams in the ECP’s Software Technology focus area to develop new mathematical algorithms and expand the tools they are developing to meet the needs of CEED’s computational motif.”

CEED has also proposed important lower-level benchmarking problems (known as bake-off problems) to compare different high-order approaches and engage with the external high-order community via GitHub.

“These problems are designed to exercise really hot kernels that are central to the performance of high-order methods,” Kolev said. “This is an activity we use internally to push each other and learn from each other and make sure we are delivering to the users the best possible performance based on our collective experience. Engaging with the community is a win-win for everyone, and we are already starting to do this: a project in Germany picked up our benchmark problems from GitHub, and ran some very interesting tests. The engagement is ultimately going to benefit the applications because we are going to use the kernels that stem from such interactions in our libraries.”

CEED is a research partnership of two US Department of Energy labs and five universities. The partners are Lawrence Livermore National Laboratory; Argonne National Laboratory; the University of Illinois Urbana-Champaign; Virginia Tech; the University of Tennessee, Knoxville; the University of Colorado, Boulder; and the Rensselaer Polytechnic Institute. The center held its first project meeting in August at LLNL, which it plans to hold annually. The meeting brought together more than 50 researchers from ECP projects, vendors, and industry.

Link to the original article on the ECP web site: https://www.exascaleproject.org/co-design-center-develops-next-generation-simulation-libraries-and-mini-apps/

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Amid Upbeat Earnings, Intel to Cut 1% of Employees, Add as Many

January 24, 2020

For all the sniping two tech old timers take, both IBM and Intel announced surprisingly upbeat earnings this week. IBM CEO Ginny Rometty was all smiles at this week’s World Economic Forum in Davos, Switzerland, after  Read more…

By Doug Black

Indiana University Dedicates ‘Big Red 200’ Cray Shasta Supercomputer

January 24, 2020

After six months of celebrations, Indiana University (IU) officially marked its bicentennial on Monday – and it saved the best for last, inaugurating Big Red 200, a new AI-focused supercomputer that joins the ranks of Read more…

By Staff report

What’s New in HPC Research: Tsunamis, Wildfires, the Large Hadron Collider & More

January 24, 2020

In this bimonthly feature, HPCwire highlights newly published research in the high-performance computing community and related domains. From parallel programming to exascale to quantum computing, the details are here. Read more…

By Oliver Peckham

Toshiba Promises Quantum-Like Advantage on Standard Hardware

January 23, 2020

Toshiba has invented an algorithm that it says delivers a 10-fold improvement for a select class of computational problems, without the need for exotic hardware. In fact, the company's simulated bifurcation algorithm is Read more…

By Tiffany Trader

Energy Research Combines HPC, 3D Manufacturing

January 23, 2020

A federal energy research initiative is gaining momentum with the release of a contract award aimed at using supercomputing to harness 3D printing technology that would boost the performance of power generators. Partn Read more…

By George Leopold

AWS Solution Channel

Challenging the barriers to High Performance Computing in the Cloud

Cloud computing helps democratize High Performance Computing by placing powerful computational capabilities in the hands of more researchers, engineers, and organizations who may lack access to sufficient on-premises infrastructure. Read more…

IBM Accelerated Insights

Intelligent HPC – Keeping Hard Work at Bay(es)

Since the dawn of time, humans have looked for ways to make their lives easier. Over the centuries human ingenuity has given us inventions such as the wheel and simple machines – which help greatly with tasks that would otherwise be extremely laborious. Read more…

TACC Highlights Its Upcoming ‘IsoBank’ Isotope Database

January 22, 2020

Isotopes – elemental variations that contain different numbers of neutrons – can help researchers unearth the past of an object, especially the few hundred isotopes that are known to be stable over time. However, iso Read more…

By Oliver Peckham

Toshiba Promises Quantum-Like Advantage on Standard Hardware

January 23, 2020

Toshiba has invented an algorithm that it says delivers a 10-fold improvement for a select class of computational problems, without the need for exotic hardware Read more…

By Tiffany Trader

In Advanced Computing and HPC, Dell EMC Sets Sights on the Broader Market Middle 

January 22, 2020

If the leading advanced computing/HPC server vendors were in the batting lineup of a baseball team, Dell EMC would be going for lots of singles and doubles – Read more…

By Doug Black

DNA-Based Storage Nears Scalable Reality with New $25 Million Project

January 21, 2020

DNA-based storage, which involves storing binary code in the four nucleotides that constitute DNA, has been a moonshot for high-density data storage since the 1960s. Since the first successful experiments in the 1980s, researchers have made a series of major strides toward implementing DNA-based storage at scale, such as improving write times and storage density and enabling easier file identification and extraction. Now, a new $25 million... Read more…

By Oliver Peckham

AMD Recruits Intel, IBM Execs; Pending Layoffs Reported at Intel Data Platform Group

January 17, 2020

AMD has raided Intel and IBM for new senior managers, one of whom will replace an AMD executive who has played a prominent role during the company’s recharged Read more…

By Doug Black

Atos-AMD System to Quintuple Supercomputing Power at European Centre for Medium-Range Weather Forecasts

January 15, 2020

The United Kingdom-based European Centre for Medium-Range Weather Forecasts (ECMWF), a supercomputer-powered weather forecasting organization backed by most of Read more…

By Oliver Peckham

Julia Programming’s Dramatic Rise in HPC and Elsewhere

January 14, 2020

Back in 2012 a paper by four computer scientists including Alan Edelman of MIT introduced Julia, A Fast Dynamic Language for Technical Computing. At the time, t Read more…

By John Russell

White House AI Regulatory Guidelines: ‘Remove Impediments to Private-sector AI Innovation’

January 9, 2020

When it comes to new technology, it’s been said government initially stays uninvolved – then gets too involved. The White House’s guidelines for federal a Read more…

By Doug Black

IBM Touts Quantum Network Growth, Improving QC Quality, and Battery Research

January 8, 2020

IBM today announced its Q (quantum) Network community had grown to 100-plus – Delta Airlines and Los Alamos National Laboratory are among most recent addition Read more…

By John Russell

Using AI to Solve One of the Most Prevailing Problems in CFD

October 17, 2019

How can artificial intelligence (AI) and high-performance computing (HPC) solve mesh generation, one of the most commonly referenced problems in computational engineering? A new study has set out to answer this question and create an industry-first AI-mesh application... Read more…

By James Sharpe

SC19: IBM Changes Its HPC-AI Game Plan

November 25, 2019

It’s probably fair to say IBM is known for big bets. Summit supercomputer – a big win. Red Hat acquisition – looking like a big win. OpenPOWER and Power processors – jury’s out? At SC19, long-time IBMer Dave Turek sketched out a different kind of bet for Big Blue – a small ball strategy, if you’ll forgive the baseball analogy... Read more…

By John Russell

Julia Programming’s Dramatic Rise in HPC and Elsewhere

January 14, 2020

Back in 2012 a paper by four computer scientists including Alan Edelman of MIT introduced Julia, A Fast Dynamic Language for Technical Computing. At the time, t Read more…

By John Russell

Cray, Fujitsu Both Bringing Fujitsu A64FX-based Supercomputers to Market in 2020

November 12, 2019

The number of top-tier HPC systems makers has shrunk due to a steady march of M&A activity, but there is increased diversity and choice of processing compon Read more…

By Tiffany Trader

Crystal Ball Gazing: IBM’s Vision for the Future of Computing

October 14, 2019

Dario Gil, IBM’s relatively new director of research, painted a intriguing portrait of the future of computing along with a rough idea of how IBM thinks we’ Read more…

By John Russell

Intel Debuts New GPU – Ponte Vecchio – and Outlines Aspirations for oneAPI

November 17, 2019

Intel today revealed a few more details about its forthcoming Xe line of GPUs – the top SKU is named Ponte Vecchio and will be used in Aurora, the first plann Read more…

By John Russell

Dell Ramps Up HPC Testing of AMD Rome Processors

October 21, 2019

Dell Technologies is wading deeper into the AMD-based systems market with a growing evaluation program for the latest Epyc (Rome) microprocessors from AMD. In a Read more…

By John Russell

D-Wave’s Path to 5000 Qubits; Google’s Quantum Supremacy Claim

September 24, 2019

On the heels of IBM’s quantum news last week come two more quantum items. D-Wave Systems today announced the name of its forthcoming 5000-qubit system, Advantage (yes the name choice isn’t serendipity), at its user conference being held this week in Newport, RI. Read more…

By John Russell

Leading Solution Providers

SC 2019 Virtual Booth Video Tour

AMD
AMD
ASROCK RACK
ASROCK RACK
AWS
AWS
CEJN
CJEN
CRAY
CRAY
DDN
DDN
DELL EMC
DELL EMC
IBM
IBM
MELLANOX
MELLANOX
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
SIX NINES IT
SIX NINES IT
VERNE GLOBAL
VERNE GLOBAL
WEKAIO
WEKAIO

IBM Unveils Latest Achievements in AI Hardware

December 13, 2019

“The increased capabilities of contemporary AI models provide unprecedented recognition accuracy, but often at the expense of larger computational and energet Read more…

By Oliver Peckham

SC19: Welcome to Denver

November 17, 2019

A significant swath of the HPC community has come to Denver for SC19, which began today (Sunday) with a rich technical program. As is customary, the ribbon cutt Read more…

By Tiffany Trader

Jensen Huang’s SC19 – Fast Cars, a Strong Arm, and Aiming for the Cloud(s)

November 20, 2019

We’ve come to expect Nvidia CEO Jensen Huang’s annual SC keynote to contain stunning graphics and lively bravado (with plenty of examples) in support of GPU Read more…

By John Russell

Top500: US Maintains Performance Lead; Arm Tops Green500

November 18, 2019

The 54th Top500, revealed today at SC19, is a familiar list: the U.S. Summit (ORNL) and Sierra (LLNL) machines, offering 148.6 and 94.6 petaflops respectively, Read more…

By Tiffany Trader

51,000 Cloud GPUs Converge to Power Neutrino Discovery at the South Pole

November 22, 2019

At the dead center of the South Pole, thousands of sensors spanning a cubic kilometer are buried thousands of meters beneath the ice. The sensors are part of Ic Read more…

By Oliver Peckham

Azure Cloud First with AMD Epyc Rome Processors

November 6, 2019

At Ignite 2019 this week, Microsoft's Azure cloud team and AMD announced an expansion of their partnership that began in 2017 when Azure debuted Epyc-backed instances for storage workloads. The fourth-generation Azure D-series and E-series virtual machines previewed at the Rome launch in August are now generally available. Read more…

By Tiffany Trader

Intel’s New Hyderabad Design Center Targets Exascale Era Technologies

December 3, 2019

Intel's Raja Koduri was in India this week to help launch a new 300,000 square foot design and engineering center in Hyderabad, which will focus on advanced com Read more…

By Tiffany Trader

Summit Has Real-Time Analytics: Here’s How It Happened and What’s Next

October 3, 2019

Summit – the world’s fastest publicly-ranked supercomputer – now has real-time streaming analytics. At the 2019 HPC User Forum at Argonne National Laborat Read more…

By Oliver Peckham

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This