Accelerating HPC in Formula One

By Tiffany Trader

November 3, 2015

The Formula One racing community has embraced HPC as an essential part of its workflow to build ever-speedier race cars, but suggestions that CFD should obviate the need for wind tunnel testing continue to draw heated commentary. One of the aerodynamics software vendors that supports this move is Exa Corporation. The Burlington, Mass.-headquartered company serves the international automotive and transportation markets and maintains a global customer roster that includes NASA, BMW, Ford, Jaguar and Tesla, and more than one F1 racing team.

Motor sports are specialty for Exa, which positions its PowerFLOW software as uniquely suited to the demands of racing-oriented computation.

“PowerFLOW fits into the rapidly changing environment of racing by providing a broad range of data and flow visualization, which can be used to tune the vehicle design for optimal performance,” the company asserts on its website. “You can compute the dependence on ride heights, vehicle speed, wheel positions, and crosswind or slip angles by simulating key points on the track circuit and evaluate the effects of your aerodynamic tuning and design choices.”

At the heart of Exa’s PowerFLOW software suite is a patented Lattice Boltzmann-based implementation of computational fluid dynamics (CFD) that is known for its high accuracy in low Mach number regimes. Founded in 1991, Exa has spent more than two decades perfecting its use of Lattice Boltzmann-based physics to perform simulations that accurately predict the performance of designs and provide a route to further optimizations.

HPCwire recently spoke with the Exa team about how CFD generally as well as their approach specifically offers benefits over expensive physical testing and reduces the need for costly late-stage changes. Much of our conversation centered on the limitations of wind tunnels with respect to complexities of the testing process and specific shortfalls that arise due to its being “a simulated version of reality that creates wind in a building around a car that’s not moving and a road surface that’s not moving underneath it.”

Exa finds itself having to counter the perception that wind tunnels provide this perfect representation of reality, but the fact is that wind tunnel testing is still a facsimile, a physical simulation as opposed to a digital one. Dr. Brad Duncan, aerodynamics applications director at Exa, explained further.

“The primary problem with a wind tunnel,” Duncan began, “is that you’re trying to model a moving car in a stationary laboratory. In order to make the experience road-like you have to blow the air at the right speed that the car would be driving; you also have to model the fact that the airflow is an open space, but the lab is a confined space; and there’s the fact that everything about the wind tunnel is fixed, but in the real-world the car is on a road and is moving, so you would have belts moving and wind blowing under the car. To capture all of this requires quite a few approximations and complex systems that have a lot of issues.

“One of the main challenges relates to air flow. When blowing air through a duct, trying to get the speed of that air right is a very tricky thing. One of the leading order errors is that the flow speed around the car doesn’t exactly match what you are trying to model, which is the car driving at a certain speed, say 150-200 miles per hour, down the track. The solution is to pick a point of calibration, but if the speed is correct at two lengths in front of the car, it doesn’t mean it’s accurate at the car. In aerodynamics flow speed is very important for accuracy, but in this simulated environment, forces will be too high or too low based on an error in the test speed calibration.

“There’s another problem because when you are testing in the wind tunnel with a Formula One car, you are not able to test at the true speed. A lot of wind tunnels will go faster than highway speed but max out at a speed that is lower than true track speed. Straightaway speed, depending on the particular circuit, can be as high as 200 miles per hour. When these high-speeds are attempted, the tires and the air heat up, which also skews the data. There are a stack of correction factors and adjustments that have to be made to try to address these effects and standardize them, and that creates an error stack up that can add additional tolerance to every number.”

Another factor is the force of the air flow coming off the rear wing, Duncan explained. Very much like how takeoff works on an airplane (but upside down), the flow that goes past the rear car wing is getting pushed up in order to create a tremendous down force on the back of the car. The problem is that the wind tunnel is feeling the presence of this flow that is getting pushed upward far away from the car but starting to hit the walls of the wind tunnel in the collector, the part of the wind tunnel where the air goes back in. That upwash from the wing makes the car look bigger to the air, making an even bigger obstacle to the airflow in the wind tunnel, according to Exa’s Duncan, who adds that all of these individual variances accumulate and will shift up or down the forces that engineering teams are trying to measure.

Sometimes the car itself is also a model at about 60 percent scale (injecting even more approximations into the process), but there is testing where the actual car is used, however even then considerations must be made to how vehicles are attached to the wind tunnel. For example, the car may be attached to a central sting that goes up to the roof (generally used in a scale model) or it can be secured at the struts.

Traditionally, the entire process was based on physical testing where all design was done in a wind tunnel. Now each car company has its own unique workflow and CFD testing can be used in all stages of the process and wind tunnels can also be used throughout. Most automotive companies are using a mix of both; a typical workflow may be heavy in CFD in the beginning and then wind tunnel testing comes in due to compliance and regulations dictated by the EPA (in the US) and the NEDC (in Europe). Even in this area, changes are coming, say Exa, referencing a recent Jaguar model that was completely developed using Exa CFD software and then only the final validation was done in wind tunnel. Discounting the artificial restrictions set down by the Formula One management body, wind tunnel testing is a more limited resource than CFD, but it is still essential for meeting regulatory requirements around fuel emissions. (There is some overlap with safety regulations, but according to Exa, that tends to be its own domain: crash testing and crash simulation.)

“The real value of simulation for aerodynamics is about designing the vehicle so it performs well. What we’re trying to do is make the world a better place — improve fuel economy, reduce emissions, and still make it possible for car companies to sell those cars because their ability to sell is based on the aesthetics and performance of the car and all the other things that consumers are looking for. They’re not necessarily going pay for fuel economy as much as some other things but we want the car companies to be able to deliver a competitive vehicle that performs well in every way, is still beautiful, still has the interior space, still has the safety, and meets all the regulations. We want to give car companies a way to be able to be best in class in aerodynamics and then they can say that this car has better fuel economy than all of its competitors,” said Duncan.

“Where aerodynamics simulation fits in is it allows you to design for fuel economy and emissions. Because you are using a digital test environment you are able to do that in collaboration with the other work that the car company is doing, which is designing the car for style and checking that it meets all of their other engineering requirements.”

“When it comes to engineering simulations, aerodynamics is the one that is most closely coupled to the aesthetics of the styling of the car,” Duncan shared. “Brand identity and what determines if a car sells has so much more to do with styling.

“They say that form follows function, but that’s not really true in the automotive market.

“And for race cars it’s a different story, it’s about winning the next race. For that the simulation has to be accurate and it has to give them a better design solution.

“In automotive racing, you have these engineers that are competitive and then they have to find the best answer to the design problem as fast as possible, which means they need accurate data but they also need something more like what you would call insight simulation can give you that insight because it offers so much more data — visual data and and practical data that helps them make a design decision.

“That’s one of the key limitations to the wind tunnel; you can’t get insight into why something works or doesn’t work.”

Duncan also noted that doing something on a computer allows engineers to optimize the testing process in a way that just isn’t possible with physical testing.

“The workflow is complex and involves adding and removing parts on a car, positioning them differently, combining them in different ways,” he explained. “When you remove material or reposition a part, if you then decide to put it back, if you’re trying to do three different things and find the combinations means you have to keep putting part A back to test part B and C in their positions. That can be challenging unless you have a very well-designed test, designed for efficient modularity. All these steps also involve a lot of time delays and the result is a number on the screen. Automated optimization is on a computer is much more straight-forward.”

Duncan reserved his heartiest endorsement of CFD for F1 design for Exa’s PowerFLOW software, a fully-transient time-accurate solver. PowerFLOW’s transient property means that the software can model how turbulence changes with time. According to Exa, the software provides unmatched simulation accuracy of ±1 count (equivalent to a Cd of 0.001), as opposed to an estimated ±30 counts for the rival steady-state codes. With Lattice Boltzmann physics, fluid is treated as mesoscopic particles (not molecules). “We’re using a very different approach to the physics,” he added, “based on IP and advanced research. You can look up Lattice Boltzmann in a text book, but you can’t look up what’s in our solver.”

“Some of the older CFD codes based on steady-state solvers are providing only one time-averaged flow result and it’s not able to tell you that flow is changing in time, so it’s not getting buffeting and oscillations or the changes that are happening due to a realistic flow effect. A flow around a car is not actually steady, so they are making this giant assumption by using steady-state, one-shot simulations. So they are not able to get as much benefit out of it. This built-in inaccuracy has traditionally been accepted as a tradeoff made to get the answer faster.”

“PowerFLOW doesn’t have a steady-state mode; it only runs in what we call real-mode,” said Duncan. “You see these small vortices and wakes and structures in the airflow that give you that look and feel that the flow is real and also is much more accurate because of that. It’s more similar to the wind tunnel and the track in that sense that the flow is realistically modeled because we have a significant competitive advantage in how we model airflow.

Most CFD codes use Navier-Stokes equations, but Lattice-Boltzman achieves very realistic flow simulation that works for these complex flows that flow around the F1 car. There are other codes based on LBM, inluding XFlow released by Next Limit Technologies, but Exa claims its PowerFLOW software has much more development behind it.

The tradeoff of this additional accuracy is that PowerFLOW is more computationally expensive to run a fully time accurate solver. “It’s so much more accurate that it’s hard to say that that’s actually a problem because if you run a steady-state solver and you get the wrong answer it might cost you more because you’re trying to work around the limitation and you might test something and find that it doesn’t work and then you have expensive delays and rebuilds because the information it gave you wasn’t very helpful. So in the end, you might say that a more accurate solver is priceless.”

In line with these benefits, Exa is not known for being a low-cost solution, Duncan said, but he pointed to Exa’s IP, higher levels of expertise and service and support. “We’re not just providing software in a box, we’re providing the whole engineering solution, so I think the customer is getting a lot more.”

Every simulation typically runs on a compute cluster of 200-400 cores for about one-to-two days, with turnaround time being the critical parameter. “The code scales very well, better than most other CFD solvers,” Duncan said, explaining that because the LB is particle-based there’s a lot of independent operations that make it very condusive to scaling up to thousands of processors. “Most customers are running the code on commodity machines with the individual core performance being the most important parameter along with good networking since overall performance is limited by the data bandwidth between nodes. We try to keep up on the latest Intel racks and interconnect fabrics, InfiniBand and the like,” said Duncan.

The F1 regulations currently enforce limitations on both wind tunnel testing and CFD work with an inverse relationship between the two. Testing is limited no matter how you do it, although workarounds have been rumored to exist. Given this limitation there is an argument to be made that having a more accurate test would make the most use out of the limited time available. As for those who point to poor results with teams who have used a pure-play CFD approach, Duncan suggested that this could be do to those traditional steady-state codes that make accuracy tradeoffs for the sake of speed. Looked at another way, the faster run times mean more runs can be completed within the mandated envelope, but there’s a catch-22 here. If the rules cap CPU hours, teams can do a lot more tests if they have a less accurate method, but a less accurate method is less valuable.

Clarifying an earlier point that he’d made, Duncan noted that he wouldn’t advise that just any CFD solution would be sufficient to completely replace wind tunnels, but an accurate simulation method could replace wind tunnels completely. “There are so many ways that you can get better data from CFD than wind tunnels in terms of predicting the performance of the car on the road. You can make the case that a team would benefit from spending their entire allocation on a more accurate solution.”

One gets the sense that these limitations while intentioned to enhance the competitiveness are hampering the sport and reducing potential innovations that could set the stage for technology transfer.

Duncan responded that automotive companies are very competitive right now to improve aerodynamics. With the CAFE requirements from the Obama administration, car makers are in a tight spot because they have to perform better and still make a car that people want to buy. They can’t give up performance or any other attributes of the car in order to meet these fuel economy mandates. The solution, said Duncan, has to be innovation.

“The only way to get a more aerodynamic car and still meet all these other constraints and make a beautiful car is a truly innovative design process,” he continued. “So that’s where certainly in motor sports and especially Formula One things that they try are always leading-edge. They have to do something else or they are not going to win. The strides that they would make would advance aerodynamics and fuel economy for the rest of the world.”

PowerFLOW is available via annual license or pay-as-you-go on-demand, with the option to run on secure, hosted, high-performance systems as part of its ExaCLOUD offering.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

What’s New in HPC Research: Wind Farms, Gravitational Lenses, Web Portals & More

February 19, 2019

In this bimonthly feature, HPCwire highlights newly published research in the high-performance computing community and related domains. From parallel programming to exascale to quantum computing, the details are here. Read more…

By Oliver Peckham

Insights from Optimized Codes on Cineca’s Marconi

February 15, 2019

What can you do with 381,392 CPU cores? For Cineca, it means enabling computational scientists to expand a large part of the world’s body of knowledge from the nanoscale to the astronomic, from calculating quantum effe Read more…

By Ken Strandberg

What Will IBM’s AI Debater Learn from Its Loss?

February 14, 2019

The utility of IBM’s latest man-versus-machine gambit is debatable. At the very least its Project Debater got us thinking about the potential uses of artificial intelligence as a way of helping humans sift through al Read more…

By George Leopold

HPE Extreme Performance Solutions

HPE Systems With Intel Omni-Path: Architected for Value and Accessible High-Performance Computing

Today’s high-performance computing (HPC) and artificial intelligence (AI) users value high performing clusters. And the higher the performance that their system can deliver, the better. Read more…

IBM Accelerated Insights

Medical Research Powered by Data

“We’re all the same, but we’re unique as well. In that uniqueness lies all of the answers….”

  • Mark Tykocinski, MD, Provost, Executive Vice President for Academic Affairs, Thomas Jefferson University

Getting the answers to what causes some people to develop diseases and not others is driving the groundbreaking medical research being conducted by the Computational Medicine Center at Thomas Jefferson University in Philadelphia. Read more…

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst of bankruptcy proceedings. According to Dutch news site Drimb Read more…

By Tiffany Trader

Insights from Optimized Codes on Cineca’s Marconi

February 15, 2019

What can you do with 381,392 CPU cores? For Cineca, it means enabling computational scientists to expand a large part of the world’s body of knowledge from th Read more…

By Ken Strandberg

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

UC Berkeley Paper Heralds Rise of Serverless Computing in the Cloud – Do You Agree?

February 13, 2019

Almost exactly ten years to the day from publishing of their widely-read, seminal paper on cloud computing, UC Berkeley researchers have issued another ambitious examination of cloud computing - Cloud Programming Simplified: A Berkeley View on Serverless Computing. The new work heralds the rise of ‘serverless computing’ as the next dominant phase of cloud computing. Read more…

By John Russell

Iowa ‘Grows Its Own’ to Fill the HPC Workforce Pipeline

February 13, 2019

The global workforce that supports advanced computing, scientific software and high-speed research networks is relatively small when you stop to consider the magnitude of the transformative discoveries it empowers. Technical conferences provide a forum where specialists convene to learn about the latest innovations and schedule face-time with colleagues from other institutions. Read more…

By Elizabeth Leake, STEM-Trek

Trump Signs Executive Order Launching U.S. AI Initiative

February 11, 2019

U.S. President Donald Trump issued an Executive Order (EO) today launching a U.S Artificial Intelligence Initiative. The new initiative - Maintaining American L Read more…

By John Russell

Celebrating Women in Science: Meet Four Women Leading the Way in HPC

February 11, 2019

One only needs to look around at virtually any CS/tech conference to realize that women are underrepresented, and that holds true of HPC. SC hosts over 13,000 H Read more…

By AJ Lauer

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

Assessing Government Shutdown’s Impact on HPC

February 6, 2019

After a 35-day federal government shutdown, the longest in U.S. history, government agencies are taking stock of the damage -- and girding for a potential secon Read more…

By Tiffany Trader

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

Cray Unveils Shasta, Lands NERSC-9 Contract

October 30, 2018

Cray revealed today the details of its next-gen supercomputing architecture, Shasta, selected to be the next flagship system at NERSC. We've known of the code-name "Shasta" since the Argonne slice of the CORAL project was announced in 2015 and although the details of that plan have changed considerably, Cray didn't slow down its timeline for Shasta. Read more…

By Tiffany Trader

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

AMD Sets Up for Epyc Epoch

November 16, 2018

It’s been a good two weeks, AMD’s Gary Silcott and Andy Parma told me on the last day of SC18 in Dallas at the restaurant where we met to discuss their show news and recent successes. Heck, it’s been a good year. Read more…

By Tiffany Trader

Intel Reportedly in $6B Bid for Mellanox

January 30, 2019

The latest rumors and reports around an acquisition of Mellanox focus on Intel, which has reportedly offered a $6 billion bid for the high performance interconn Read more…

By Doug Black

US Leads Supercomputing with #1, #2 Systems & Petascale Arm

November 12, 2018

The 31st Supercomputing Conference (SC) - commemorating 30 years since the first Supercomputing in 1988 - kicked off in Dallas yesterday, taking over the Kay Ba Read more…

By Tiffany Trader

Looking for Light Reading? NSF-backed ‘Comic Books’ Tackle Quantum Computing

January 28, 2019

Still baffled by quantum computing? How about turning to comic books (graphic novels for the well-read among you) for some clarity and a little humor on QC. The Read more…

By John Russell

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

Contract Signed for New Finnish Supercomputer

December 13, 2018

After the official contract signing yesterday, configuration details were made public for the new BullSequana system that the Finnish IT Center for Science (CSC Read more…

By Tiffany Trader

Deep500: ETH Researchers Introduce New Deep Learning Benchmark for HPC

February 5, 2019

ETH researchers have developed a new deep learning benchmarking environment – Deep500 – they say is “the first distributed and reproducible benchmarking s Read more…

By John Russell

IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans

January 10, 2019

IBM made three significant quantum computing announcements at CES this week. One was introduction of IBM Q System One; it’s really the integration of IBM’s Read more…

By John Russell

HPC Reflections and (Mostly Hopeful) Predictions

December 19, 2018

So much ‘spaghetti’ gets tossed on walls by the technology community (vendors and researchers) to see what sticks that it is often difficult to peer through Read more…

By John Russell

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

Nvidia’s Jensen Huang Delivers Vision for the New HPC

November 14, 2018

For nearly two hours on Monday at SC18, Jensen Huang, CEO of Nvidia, presented his expansive view of the future of HPC (and computing in general) as only he can do. Animated. Backstopped by a stream of data charts, product photos, and even a beautiful image of supernovae... Read more…

By John Russell

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

Intel Confirms 48-Core Cascade Lake-AP for 2019

November 4, 2018

As part of the run-up to SC18, taking place in Dallas next week (Nov. 11-16), Intel is doling out info on its next-gen Cascade Lake family of Xeon processors, specifically the “Advanced Processor” version (Cascade Lake-AP), architected for high-performance computing, artificial intelligence and infrastructure-as-a-service workloads. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This