Advancing the Additive Manufacturing Revolution: ExaAM’s Simulation Codes Bring Exascale Power to Industrial 3D Printing

December 22, 2022

Dec. 22, 2022 — The methods used to fabricate metal parts have changed remarkably little in the past few centuries. Although machining technology has certainly advanced, it still follows basic metalworking methods to produce the parts in our vehicles, our appliances, our buildings, and more: pour metal into a mold, carve it away from a block, stamp it, extrude it, or roll it. But all those legacy processes introduce limitations to the metal parts being made—from their potential geometries to their physical properties.

The ECP’s ExaAM team gathers in front of a screen from ExaAM, a simulation toolset for exascale supercomputers that can model entire metal parts being constructed through the 3D printing process. From left, standing: Sam Reeve, Alex Plotkowski, Matt Bement, Stephen DeWitt, Austin Isner, Matt Rolchigo. From left, sitting: Gerry Knapp, Adrian Sabau, Philip Fackler. Credit: Kelley Smith/ORNL.

What if a new approach could create metal parts in shapes so complex that they’d otherwise be impossible to construct? What if their microstructures were so accurately designed and fabricated that different sections of a single part could yield different material properties?

That scenario is no longer hypothetical with the advent of additive manufacturing (AM). Rather than taking a piece of metal and reforming it into something else, AM fabricators make parts from scratch by sequentially adding layers of liquid metal into patterns until they form 3D shapes that solidify as they cool. If that sounds like a consumer-grade 3D printer, that’s because the basic technology is the same. However, taken to an industrial scale, this fabrication method can go far beyond printing plastic models on your desktop. AM heralds a new era of precision manufacturing that could make a big impact on the aerospace, biomedical, energy, and automotive industries, where it has already been making inroads.

The ExaAM project, which is a collection of simulation codes optimized for exascale-class supercomputers (i.e., 1 quintillion or more calculations per second), aims to advance AM as a viable method for industrial manufacturing. Supported by the US Department of Energy’s (DOE’s) Exascale Computing Project (ECP), ExaAM consists of a team of researchers at DOE’s Oak Ridge National Laboratory (ORNL), Lawrence Livermore National Laboratory, Los Alamos National Laboratory, the National Institute of Standards and Technology, and several academic collaborators. The team has developed the ExaAM tool set to provide insights into how metals behave throughout the AM process (i.e., melting, solidifying) and how they perform. This is critical information for designing new parts as well as gauging their reliability and functionality.

“The purpose of ExaAM is to model the AM manufacturing process—you’ve got heat being deposited, metal melting, metal solidifying, and then you’re going to the next layer with more metal remelting and then solidifying. So, you get all this thermal cycling, which makes for a very complicated process,” said Matt Bement, head of ORNL’s Scalable Algorithms and Coupled Physics group and the project’s principal investigator.

“Being able to understand how that process gives you a microstructure and what that microstructure tells you about its properties—that’s what we’re trying to do,” Bement said. “If you can do that accurately, then you could start printing 3D parts and qualify them for critical missions because you really do understand what it is you just made.”

Predicting Entire Parts Before They’re Made

AM and 3D printing technologies have existed for a few decades, but AM has taken longer to mature due to its complexity—metal is much more complicated to print than plastic. When it cools and solidifies, crystals within the metal align to form microstructures that determine its as a material, which will ultimately affect the of the part.

“Everything we know about how most parts have historically been made doesn’t help us as much as we would like with planning an AM part. The microstructure that you get from an AM part is very different from traditionally manufactured parts,” Bement said. “How the metallic crystals end up packing themselves together during AM makes a really big difference in what that metallic part is going to do—how it’s going to react to fatigue loading, how it’s going to react to impact loading, and how it’s going to react to different temperatures.”

And that’s where ExaAM comes in: it models how these parts can be precisely constructed through AM. It achieves this by combining individual software tools that each simulate different stages of AM to collectively capture the relevant physics of the process.

Some of the tools have been adapted from preexisting codes, such as the open-source fluid dynamics simulator OpenFOAM, which the team extended to model AM’s melting stage and redubbed as additiveFOAM. Solidification and its resulting microstructures can be simulated with a few different tools: ExaCA provides a fast-running but lower-accuracy simulation of solidification, whereas the phase-field codes AMPE, MEUMAPPS, and Tusas provide higher accuracy but more computationally expensive solidification modeling. ExaConstit handles predictions of mechanical properties, and Diablo provides a simulation of the whole part.

Overcoming Timescales

To model an entire part that’s constructed through AM, the team had to overcome a major computational hurdle: the time difference between the physical processes being simulated (i.e., melting, solidifying) vs. the actual time it takes to construct a part with an AM fabricator.

ORNL researchers Matt Bement (left) and Alex Plotkowski lead the effort to take the ExaAM simulation toolset into the industrial world of additive manufacturing. They are standing in front of MedUSA, located at ORNL’s Manufacturing Demonstration Facility. MedUSA is a hybrid manufacturing machine that enables both additive and subtractive manufacturing capabilities in one unique system. Credit: Kelley Smith/ORNL.

“One of the big challenges of doing a whole simulation is that things that happen in the melt pool are measured in microseconds. But if you were to sit there and watch the printer build the whole part, it would take hours,” Bement said. “There are a lot of microseconds in hours, so the sheer number of timesteps that you have to take becomes an enormous problem.”

A similar computational issue was one of physical scale. AM’s melting and solidifying processes occur in a length scale of a few microns, but the team’s ultimate goal for ExaAM is to simulate large, whole parts.

“I don’t want to simulate something that’s just a few microns across or even a few hundred microns. I want something that’s centimeters long, a part that you can hold in your hand. So, we had to find a couple of ways to adapt those things,” said Alex Plotkowski, an R&D staff member of ORNL’s Materials Science and Technology group and researcher at ORNL’s Manufacturing Demonstration Facility (MDF).

The team had to devise a way to parallelize the time domain and scale up the simulations to handle entire parts—and they succeeded.

“We developed a time-parallel approach to take those types of simulations and scale them to very large supercomputers. That gives us much larger scale, and that’s one of the reasons why it’s well suited for the ECP as a whole—we can make good use of those large computing resources to be able to do these problems at a scale that is maybe not feasible with more conventional computing platforms,” Plotkowski said.

Creating a predictive model of a whole AM part—not just sections of it—is a breakthrough achievement in itself.

“If you had tried to do this with a previous system or a desktop computer, it would have been completely intractable. The amount of time for the computations would have been measured in years. But with an exascale machine, we can get highly resolved, whole-part simulations done on the order of several hours to a couple days,” Bement said.

Taking ExaAM to the Factory Floor

Plotkowski works at ORNL’s MDF, a 110,000 square-foot, off-campus building where the lab tests its advanced manufacturing technologies and shows them to industry partners. His background in physical metallurgy and alloy design for manufacturing gives him a factory floor perspective on assessing ExaAM’s potential for industrial applications.

“We’ve been developing capabilities for ExaAM that enable us to predict useful things about the AM process in terms of the construction properties of these materials. That has a pretty obvious application to a lot of industrial companies that are interested in being able to optimize AM processes by bringing those things in house and being able to fabricate components for themselves,” Plotkowski said.

ExaAM has been successfully tested at MDF by making parts on an industrial-scale laser powder-bed printer, which puts down layers of metal powder and then melts them with lasers. This has enabled the team to take measurements of the constructed parts’ microstructures and properties as well as the dynamics of the process while they’re being fabricated. They then compare those measurements against the predictions from the software tools to help guide the software development and confirm that it’s making reasonable predictions.

“We’re in the process of trying to develop a workflow in which we can take actual data that comes from the systems about how they’re performing. Then we can use that to drive the simulation tools so we’re doing a one-to-one simulation—a digital twin to get simulation data that corresponds one-to-one with components that we’re fabricating in real life,” Plotkowski said.

Ultimately, the team hopes to make ExaAM’s entire tool set public and has already put some of the codes online for anyone to download.

“Each of those is a scalable code, so it’s written in such a way that it will run on laptops or workstations but will also run very large things on supercomputers,” Plotkowski said. “It takes a little expertise to go all the way from one end to the other if you’re trying to use those codes, but it’s totally possible, and they’re written that way to make it usable for industrial partners in their own workstations or clusters.”

With the ECP scheduled to wind down in 2023 and 2024, the ExaAM project is on track to successfully deliver AM simulation capabilities to assist other DOE and industry efforts. The team’s next step is to use these tools to accelerate AM adoption for critical energy applications. The ExaAM codes are being incorporated into the MDF Digital Factory—which is funded by the DOE’s Advanced Materials & Manufacturing Technologies Office (AMMTO)—to help accelerate qualification of AM components for nuclear energy in the DOE Office of Nuclear Energy’s Advanced Manufacturing and Materials Technology program.

Meanwhile, ORNL researchers are working with General Motors to understand the AM process’s effects on the properties of lightweight automotive components via a DOE program called High-Performance Computing for Manufacturing that’s also funded by AMMTO. This effort may soon bring AM-printed parts to our daily lives.

Video

A laser powder-bed 3D printer at Oak Ridge National Laboratory’s Manufacturing Demonstration Facility begins the process of constructing a metal part. Not unlike a desktop 3D printer, additive manufacturing fabricators make parts from scratch by sequentially adding layers of liquid metal into patterns until they form 3D shapes that solidify as they cool. Video by Kelley Smith/ORNL.


Source: Coury Turczyn, Oak Ridge National Laboratory & Exascale Computing Project

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

ASC23: Application Results

June 2, 2023

The ASC23 organizers put together a slate of fiendishly difficult applications for the students this year. The apps were a mix of traditional HPC packages, like WRF-Hydro and FVCOM, plus machine learning centric programs Read more…

Q&A with Marco Pistoia, an HPCwire Person to Watch in 2023

June 2, 2023

HPCwire Person to Watch Marco Pistoia wears a lot of hats at JPMorgan Chase & Co.: managing director, distinguished engineer, head of global technology applied research and head of quantum computing. That work with J Read more…

HPC Career Notes: June 2023 Edition

June 1, 2023

In this monthly feature, we’ll keep you up-to-date on the latest career developments for individuals in the high-performance computing community. Whether it’s a promotion, new company hire, or even an accolade, we’ Read more…

Intersect360: HPC Market ‘Returning to Stable Growth’

June 1, 2023

The folks at Intersect360 Research released their latest report and market update just ahead of ISC 2023, which was held in Hamburg, Germany, last week. The headline: “We’re returning to stable growth,” per Addison Read more…

Lori Diachin to Lead the Exascale Computing Project as It Nears Final Milestones

May 31, 2023

The end goal is in sight for the multi-institutional Exascale Computing Project (ECP), which launched in 2016 with a mandate from the Department of Energy (DOE) and National Nuclear Security Administration (NNSA) to achi Read more…

AWS Solution Channel

Shutterstock 1493175377

Introducing GPU health checks in AWS ParallelCluster 3.6

GPU failures are relatively rare but when they do occur, they can have severe consequences for HPC and deep learning tasks. For example, they can disrupt long-running simulations and distributed training jobs. Read more…

 

Shutterstock 1415788655

New Thoughts on Leveraging Cloud for Advanced AI

Artificial intelligence (AI) is becoming critical to many operations within companies. As the use and sophistication of AI grow, there is a new focus on the infrastructure requirements to produce results fast and efficiently. Read more…

ASC23: LINPACK Results

May 30, 2023

With ISC23 now in the rearview mirror, let’s get back to the results from the ASC23 Student Cluster Competition. In our last articles, we looked at the competition and applications, plus introduced the teams, now it’ Read more…

ASC23: Application Results

June 2, 2023

The ASC23 organizers put together a slate of fiendishly difficult applications for the students this year. The apps were a mix of traditional HPC packages, like Read more…

Intersect360: HPC Market ‘Returning to Stable Growth’

June 1, 2023

The folks at Intersect360 Research released their latest report and market update just ahead of ISC 2023, which was held in Hamburg, Germany, last week. The hea Read more…

Lori Diachin to Lead the Exascale Computing Project as It Nears Final Milestones

May 31, 2023

The end goal is in sight for the multi-institutional Exascale Computing Project (ECP), which launched in 2016 with a mandate from the Department of Energy (DOE) Read more…

At ISC, Sustainable Computing Leaders Discuss HPC’s Energy Crossroads

May 30, 2023

In the wake of SC22 last year, HPCwire wrote that “the conference’s eyes had shifted to carbon emissions and energy intensity” rather than the historical Read more…

Nvidia Announces Four Supercomputers, with Two in Taiwan

May 29, 2023

At the Computex event in Taipei this week, Nvidia announced four new systems equipped with its Grace- and Hopper-generation hardware, including two in Taiwan. T Read more…

Nvidia to Offer a ‘1 Exaflops’ AI Supercomputer with 256 Grace Hopper Superchips

May 28, 2023

We in HPC sometimes roll our eyes at the term “AI supercomputer,” but a new system from Nvidia might live up to the moniker: the DGX GH200 AI supercomputer. Read more…

Closing ISC Keynote by Sterling and Suarez Looks Backward and Forward

May 25, 2023

ISC’s closing keynote this year was given jointly by a pair of distinguished HPC leaders, Thomas Sterling of Indiana University and Estela Suarez of Jülich S Read more…

The Grand Challenge of Simulating Nuclear Fusion: An Overview with UKAEA’s Rob Akers

May 25, 2023

As HPC and AI continue to rapidly advance, the alluring vision of nuclear fusion and its endless zero-carbon, low-radioactivity energy is the sparkle in many a Read more…

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

Leading Solution Providers

Contributors

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

ISC 2023 Booth Videos

Cornelis Networks @ ISC23
Dell Technologies @ ISC23
Intel @ ISC23
Lenovo @ ISC23
ISC23 Playlist
  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire