The U.S. Department of Energy's Office of Science has allocated 400,000 processor hours of supercomputing time at its National Energy Research Scientific Computing (NERSC) Center to the U.S. Army Corps of Engineers New Orleans District to run a series of simulations of hurricane protection projects within coastal Louisiana.

Energy Secretary Samuel Bodman announced the allocation of NERSC resources earlier this week during his keynote address at the Energy Leadership Forum in Robinsonville, MS.

“The Department of Energy is so much more than gasoline and coal, we're Americans who saw our neighbors devastated by the hurricanes — and we're finding ways to help,” said Secretary Bodman. “In addition to strengthening our nation's energy infrastructure and improving our department's response to natural disasters, our goal is to help those affected in every way that we can.”

The Army Corps of Engineers has been asked by the Federal Emergency Management Agency (FEMA) to run a series of simulations estimating hurricane-induced storm surge elevations as part of FEMA's Map Modernization Program to update Flood Insurance Rate Maps in other areas. The data collected from the simulations will provide valuable flood elevation data that will be used by FEMA to develop new flood hazard information. Additionally, FEMA has asked the Corps' New Orleans District to speed up development of new flood insurance studies in areas where they are working on them.

“Not only is it critical that we help those communities which were devastated by Hurricanes Katrina and Rita, but it is just as important that we provide whatever resources we can to help ensure that rebuilding efforts are carried out with the goal of providing as much protection as possible against the effects of future storms,” said Director of DOE's Office of Science, Raymond Orbach. “Utilizing the computing resources at NERSC for this project is an ideal match. The center has consistently been held up as a model for other computing centers and has a well-earned reputation for providing highly reliable systems, fast turnaround on critical projects and dedicated support for users. We are glad we can be of service to this important project.”

The goal is to complete the calculations by the end of February. To help ensure this deadline is met, NERSC will provide dedicated technical staff expertise to the project. NERSC, which has been the DOE Office of Science's flagship center for unclassified supercomputing for more than 30 years, will make its three supercomputing systems available for the project.

Because of coastal inundation induced by Hurricanes Katrina and Rita, the New Orleans District has begun to rebuild and enhance the existing flood control system, as well as design a new system. This design will offer a higher level of protection to the city of New Orleans and coastal Louisiana while at the same time encompassing the state's coastal ecosystem. A critical piece of information needed for such a design is the resulting total water level (surge, tide, wave setup, and wave runup) induced by a hurricane.

However, accurately modeling such large-scale problems, which include many complex factors, is a very demanding computing project and is ideally suited to a large supercomputing center. Running the 400,000 hours of simulations on a single-processor PC would take about 46 years. Running the code on a small supercomputer, with 128 processors, would take about 130 days. But by tapping NERSC's supercomputers, which include a 6,080-processor IBM supercomputer, an 888-processor IBM cluster computer and a 640-processor Linux Networx cluster, the simulations are expected to be completed within the one-month deadline.

Given that the levee system must extend across the entire State of Louisiana, a very large number of simulations must be conducted in order to determine how to best design a level high enough to provide the targeted protection level. To obtain this information, the Corps of Engineers New Orleans District is performing three important hurricane protection projects within coastal Louisiana.

The first project is the FEMA Coastal Storm Surge Study. FEMA has asked the District to accelerate the computer modeling of these storm surge analyses across the Gulf coast as the results will be critical to the rebuilding of New Orleans. The studies are performed using a high grid resolution physics-based modeling software system called the “Advanced CIRCulation” (ADCIRC) model, which was developed for the Corps of Engineers by Dr. Joannes Westerink at the University of Notre Dame, Department of Civil Engineering and Geologic Sciences and by Dr. Rick Luettich at the University of North Carolina at Chapel Hill, Institute of Marine Sciences.

The second and third projects are the Donaldsonville and Morganza to the Gulf of Mexico Projects. The Morganza project alone is an approximately $740 million project for building a 72-mile levee protection system spanning the area between the cities of Larose and Houma. The Donaldsonville project is studying levee alternatives to protect against surging water caused by tropical storms and will provide protection starting at the New Orleans West Bank Levees to Larose.

The Donaldsonville and Morganza projects will form critical storm surge protection from the New Orleans area west towards the Atchafalaya Basin River Levee system. Additionally, the ADCIRC work on these projects is laying the foundation for large-scale Gulf wide coast storm surge computations.

The ADCIRC mesh for Louisiana is over 600,000 grid points and contains more than 1.1 million elements, and typically runs simultaneously on 128 or more processors. Corps of Engineers computer experts predict the combined computing projects will require more than 350,000 hours of computer processing time.

Together with the Corps' Engineer Research and Development Center, the District will use the ADCIRC model for estimating hurricane-induced storm surge elevations for existing and design conditions. They will also use the STWAVE application, which predicts wave conditions in coastal areas, for predicting wave setup and wave runup elevations.