Accelerating HPC in Formula One

By Tiffany Trader

November 3, 2015

The Formula One racing community has embraced HPC as an essential part of its workflow to build ever-speedier race cars, but suggestions that CFD should obviate the need for wind tunnel testing continue to draw heated commentary. One of the aerodynamics software vendors that supports this move is Exa Corporation. The Burlington, Mass.-headquartered company serves the international automotive and transportation markets and maintains a global customer roster that includes NASA, BMW, Ford, Jaguar and Tesla, and more than one F1 racing team.

Motor sports are specialty for Exa, which positions its PowerFLOW software as uniquely suited to the demands of racing-oriented computation.

“PowerFLOW fits into the rapidly changing environment of racing by providing a broad range of data and flow visualization, which can be used to tune the vehicle design for optimal performance,” the company asserts on its website. “You can compute the dependence on ride heights, vehicle speed, wheel positions, and crosswind or slip angles by simulating key points on the track circuit and evaluate the effects of your aerodynamic tuning and design choices.”

At the heart of Exa’s PowerFLOW software suite is a patented Lattice Boltzmann-based implementation of computational fluid dynamics (CFD) that is known for its high accuracy in low Mach number regimes. Founded in 1991, Exa has spent more than two decades perfecting its use of Lattice Boltzmann-based physics to perform simulations that accurately predict the performance of designs and provide a route to further optimizations.

HPCwire recently spoke with the Exa team about how CFD generally as well as their approach specifically offers benefits over expensive physical testing and reduces the need for costly late-stage changes. Much of our conversation centered on the limitations of wind tunnels with respect to complexities of the testing process and specific shortfalls that arise due to its being “a simulated version of reality that creates wind in a building around a car that’s not moving and a road surface that’s not moving underneath it.”

Exa finds itself having to counter the perception that wind tunnels provide this perfect representation of reality, but the fact is that wind tunnel testing is still a facsimile, a physical simulation as opposed to a digital one. Dr. Brad Duncan, aerodynamics applications director at Exa, explained further.

“The primary problem with a wind tunnel,” Duncan began, “is that you’re trying to model a moving car in a stationary laboratory. In order to make the experience road-like you have to blow the air at the right speed that the car would be driving; you also have to model the fact that the airflow is an open space, but the lab is a confined space; and there’s the fact that everything about the wind tunnel is fixed, but in the real-world the car is on a road and is moving, so you would have belts moving and wind blowing under the car. To capture all of this requires quite a few approximations and complex systems that have a lot of issues.

“One of the main challenges relates to air flow. When blowing air through a duct, trying to get the speed of that air right is a very tricky thing. One of the leading order errors is that the flow speed around the car doesn’t exactly match what you are trying to model, which is the car driving at a certain speed, say 150-200 miles per hour, down the track. The solution is to pick a point of calibration, but if the speed is correct at two lengths in front of the car, it doesn’t mean it’s accurate at the car. In aerodynamics flow speed is very important for accuracy, but in this simulated environment, forces will be too high or too low based on an error in the test speed calibration.

“There’s another problem because when you are testing in the wind tunnel with a Formula One car, you are not able to test at the true speed. A lot of wind tunnels will go faster than highway speed but max out at a speed that is lower than true track speed. Straightaway speed, depending on the particular circuit, can be as high as 200 miles per hour. When these high-speeds are attempted, the tires and the air heat up, which also skews the data. There are a stack of correction factors and adjustments that have to be made to try to address these effects and standardize them, and that creates an error stack up that can add additional tolerance to every number.”

Another factor is the force of the air flow coming off the rear wing, Duncan explained. Very much like how takeoff works on an airplane (but upside down), the flow that goes past the rear car wing is getting pushed up in order to create a tremendous down force on the back of the car. The problem is that the wind tunnel is feeling the presence of this flow that is getting pushed upward far away from the car but starting to hit the walls of the wind tunnel in the collector, the part of the wind tunnel where the air goes back in. That upwash from the wing makes the car look bigger to the air, making an even bigger obstacle to the airflow in the wind tunnel, according to Exa’s Duncan, who adds that all of these individual variances accumulate and will shift up or down the forces that engineering teams are trying to measure.

Sometimes the car itself is also a model at about 60 percent scale (injecting even more approximations into the process), but there is testing where the actual car is used, however even then considerations must be made to how vehicles are attached to the wind tunnel. For example, the car may be attached to a central sting that goes up to the roof (generally used in a scale model) or it can be secured at the struts.

Traditionally, the entire process was based on physical testing where all design was done in a wind tunnel. Now each car company has its own unique workflow and CFD testing can be used in all stages of the process and wind tunnels can also be used throughout. Most automotive companies are using a mix of both; a typical workflow may be heavy in CFD in the beginning and then wind tunnel testing comes in due to compliance and regulations dictated by the EPA (in the US) and the NEDC (in Europe). Even in this area, changes are coming, say Exa, referencing a recent Jaguar model that was completely developed using Exa CFD software and then only the final validation was done in wind tunnel. Discounting the artificial restrictions set down by the Formula One management body, wind tunnel testing is a more limited resource than CFD, but it is still essential for meeting regulatory requirements around fuel emissions. (There is some overlap with safety regulations, but according to Exa, that tends to be its own domain: crash testing and crash simulation.)

“The real value of simulation for aerodynamics is about designing the vehicle so it performs well. What we’re trying to do is make the world a better place — improve fuel economy, reduce emissions, and still make it possible for car companies to sell those cars because their ability to sell is based on the aesthetics and performance of the car and all the other things that consumers are looking for. They’re not necessarily going pay for fuel economy as much as some other things but we want the car companies to be able to deliver a competitive vehicle that performs well in every way, is still beautiful, still has the interior space, still has the safety, and meets all the regulations. We want to give car companies a way to be able to be best in class in aerodynamics and then they can say that this car has better fuel economy than all of its competitors,” said Duncan.

“Where aerodynamics simulation fits in is it allows you to design for fuel economy and emissions. Because you are using a digital test environment you are able to do that in collaboration with the other work that the car company is doing, which is designing the car for style and checking that it meets all of their other engineering requirements.”

“When it comes to engineering simulations, aerodynamics is the one that is most closely coupled to the aesthetics of the styling of the car,” Duncan shared. “Brand identity and what determines if a car sells has so much more to do with styling.

“They say that form follows function, but that’s not really true in the automotive market.

“And for race cars it’s a different story, it’s about winning the next race. For that the simulation has to be accurate and it has to give them a better design solution.

“In automotive racing, you have these engineers that are competitive and then they have to find the best answer to the design problem as fast as possible, which means they need accurate data but they also need something more like what you would call insight simulation can give you that insight because it offers so much more data — visual data and and practical data that helps them make a design decision.

“That’s one of the key limitations to the wind tunnel; you can’t get insight into why something works or doesn’t work.”

Duncan also noted that doing something on a computer allows engineers to optimize the testing process in a way that just isn’t possible with physical testing.

“The workflow is complex and involves adding and removing parts on a car, positioning them differently, combining them in different ways,” he explained. “When you remove material or reposition a part, if you then decide to put it back, if you’re trying to do three different things and find the combinations means you have to keep putting part A back to test part B and C in their positions. That can be challenging unless you have a very well-designed test, designed for efficient modularity. All these steps also involve a lot of time delays and the result is a number on the screen. Automated optimization is on a computer is much more straight-forward.”

Duncan reserved his heartiest endorsement of CFD for F1 design for Exa’s PowerFLOW software, a fully-transient time-accurate solver. PowerFLOW’s transient property means that the software can model how turbulence changes with time. According to Exa, the software provides unmatched simulation accuracy of ±1 count (equivalent to a Cd of 0.001), as opposed to an estimated ±30 counts for the rival steady-state codes. With Lattice Boltzmann physics, fluid is treated as mesoscopic particles (not molecules). “We’re using a very different approach to the physics,” he added, “based on IP and advanced research. You can look up Lattice Boltzmann in a text book, but you can’t look up what’s in our solver.”

“Some of the older CFD codes based on steady-state solvers are providing only one time-averaged flow result and it’s not able to tell you that flow is changing in time, so it’s not getting buffeting and oscillations or the changes that are happening due to a realistic flow effect. A flow around a car is not actually steady, so they are making this giant assumption by using steady-state, one-shot simulations. So they are not able to get as much benefit out of it. This built-in inaccuracy has traditionally been accepted as a tradeoff made to get the answer faster.”

“PowerFLOW doesn’t have a steady-state mode; it only runs in what we call real-mode,” said Duncan. “You see these small vortices and wakes and structures in the airflow that give you that look and feel that the flow is real and also is much more accurate because of that. It’s more similar to the wind tunnel and the track in that sense that the flow is realistically modeled because we have a significant competitive advantage in how we model airflow.

Most CFD codes use Navier-Stokes equations, but Lattice-Boltzman achieves very realistic flow simulation that works for these complex flows that flow around the F1 car. There are other codes based on LBM, inluding XFlow released by Next Limit Technologies, but Exa claims its PowerFLOW software has much more development behind it.

The tradeoff of this additional accuracy is that PowerFLOW is more computationally expensive to run a fully time accurate solver. “It’s so much more accurate that it’s hard to say that that’s actually a problem because if you run a steady-state solver and you get the wrong answer it might cost you more because you’re trying to work around the limitation and you might test something and find that it doesn’t work and then you have expensive delays and rebuilds because the information it gave you wasn’t very helpful. So in the end, you might say that a more accurate solver is priceless.”

In line with these benefits, Exa is not known for being a low-cost solution, Duncan said, but he pointed to Exa’s IP, higher levels of expertise and service and support. “We’re not just providing software in a box, we’re providing the whole engineering solution, so I think the customer is getting a lot more.”

Every simulation typically runs on a compute cluster of 200-400 cores for about one-to-two days, with turnaround time being the critical parameter. “The code scales very well, better than most other CFD solvers,” Duncan said, explaining that because the LB is particle-based there’s a lot of independent operations that make it very condusive to scaling up to thousands of processors. “Most customers are running the code on commodity machines with the individual core performance being the most important parameter along with good networking since overall performance is limited by the data bandwidth between nodes. We try to keep up on the latest Intel racks and interconnect fabrics, InfiniBand and the like,” said Duncan.

The F1 regulations currently enforce limitations on both wind tunnel testing and CFD work with an inverse relationship between the two. Testing is limited no matter how you do it, although workarounds have been rumored to exist. Given this limitation there is an argument to be made that having a more accurate test would make the most use out of the limited time available. As for those who point to poor results with teams who have used a pure-play CFD approach, Duncan suggested that this could be do to those traditional steady-state codes that make accuracy tradeoffs for the sake of speed. Looked at another way, the faster run times mean more runs can be completed within the mandated envelope, but there’s a catch-22 here. If the rules cap CPU hours, teams can do a lot more tests if they have a less accurate method, but a less accurate method is less valuable.

Clarifying an earlier point that he’d made, Duncan noted that he wouldn’t advise that just any CFD solution would be sufficient to completely replace wind tunnels, but an accurate simulation method could replace wind tunnels completely. “There are so many ways that you can get better data from CFD than wind tunnels in terms of predicting the performance of the car on the road. You can make the case that a team would benefit from spending their entire allocation on a more accurate solution.”

One gets the sense that these limitations while intentioned to enhance the competitiveness are hampering the sport and reducing potential innovations that could set the stage for technology transfer.

Duncan responded that automotive companies are very competitive right now to improve aerodynamics. With the CAFE requirements from the Obama administration, car makers are in a tight spot because they have to perform better and still make a car that people want to buy. They can’t give up performance or any other attributes of the car in order to meet these fuel economy mandates. The solution, said Duncan, has to be innovation.

“The only way to get a more aerodynamic car and still meet all these other constraints and make a beautiful car is a truly innovative design process,” he continued. “So that’s where certainly in motor sports and especially Formula One things that they try are always leading-edge. They have to do something else or they are not going to win. The strides that they would make would advance aerodynamics and fuel economy for the rest of the world.”

PowerFLOW is available via annual license or pay-as-you-go on-demand, with the option to run on secure, hosted, high-performance systems as part of its ExaCLOUD offering.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from its predecessors, including the red-hot H100 and A100 GPUs. Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. While Nvidia may not spring to mind when thinking of the quant Read more…

2024 Winter Classic: Meet the HPE Mentors

March 18, 2024

The latest installment of the 2024 Winter Classic Studio Update Show features our interview with the HPE mentor team who introduced our student teams to the joys (and potential sorrows) of the HPL (LINPACK) and accompany Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the field was normalized for boys in 1969 when the Apollo 11 missi Read more…

Apple Buys DarwinAI Deepening its AI Push According to Report

March 14, 2024

Apple has purchased Canadian AI startup DarwinAI according to a Bloomberg report today. Apparently the deal was done early this year but still hasn’t been publicly announced according to the report. Apple is preparing Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimization algorithms to iteratively refine their parameters until Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the fi Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimizat Read more…

PASQAL Issues Roadmap to 10,000 Qubits in 2026 and Fault Tolerance in 2028

March 13, 2024

Paris-based PASQAL, a developer of neutral atom-based quantum computers, yesterday issued a roadmap for delivering systems with 10,000 physical qubits in 2026 a Read more…

India Is an AI Powerhouse Waiting to Happen, but Challenges Await

March 12, 2024

The Indian government is pushing full speed ahead to make the country an attractive technology base, especially in the hot fields of AI and semiconductors, but Read more…

Charles Tahan Exits National Quantum Coordination Office

March 12, 2024

(March 1, 2024) My first official day at the White House Office of Science and Technology Policy (OSTP) was June 15, 2020, during the depths of the COVID-19 loc Read more…

AI Bias In the Spotlight On International Women’s Day

March 11, 2024

What impact does AI bias have on women and girls? What can people do to increase female participation in the AI field? These are some of the questions the tech Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Analyst Panel Says Take the Quantum Computing Plunge Now…

November 27, 2023

Should you start exploring quantum computing? Yes, said a panel of analysts convened at Tabor Communications HPC and AI on Wall Street conference earlier this y Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Training of 1-Trillion Parameter Scientific AI Begins

November 13, 2023

A US national lab has started training a massive AI brain that could ultimately become the must-have computing resource for scientific researchers. Argonne N Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire