The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing
October 29, 2009
With great power come great I/O issues
Despite the muscle of today's premier supercomputing systems, scaling and I/O often prevent leading software packages from taking full advantage of the latest hardware's potential power. These ancillary tasks take precious time and resources away from studying the fundamental science that impacts our everyday lives.
To address these issues, a team of researchers from Oak Ridge National Laboratory (ORNL), Georgia Tech, and Rutgers University developed the ADaptable I/O file System (ADIOS). ADIOS is an I/O middleware package that has shown great promise with leading fusion codes, scaling up to 140,000 cores for XGC-1, GTC, and GTS, and in astrophysics with the CHIMERA code. Recently, ADIOS made its mark in the field of combustion.
Researchers at Sandia National Laboratory (Ray Grout, Jackie Chen, and Chun Sang Yoo) and Andrea Gruber of SINTEF, the largest independent research organization in Scandinavia, are using the leading combustion code S3D to perform the first direct numerical simulations of reacting jets in cross flow. These transverse jets are a class of flows used in practical applications in which high mixing rates are desirable -- for example, in fuel injection nozzles in stationary gas turbines for power generation or in aero-gas turbines. The gas turbine industry is faced with numerous R&D challenges in adapting conventional hydrocarbon burner designs to operate safely and cleanly with hydrogen-rich syngas.
Though the S3D team was able to fully scale the code to ORNL's entire Cray XT5 supercomputer, known as Jaguar, the scaling of the I/O proved to be difficult. Enter ADIOS.
Besides its ability to scale, ADIOS's BP file format is resilient to failures in the compute nodes and the file system, an attribute that hinted to researchers that S3D's analysis routines would pair well with the ADIOS ecosystem. After consultations with ORNL's Qing Liu and Scott Klasky (a leading ADIOS developer) and Georgia Tech's Jay Lofstead, Chen's post-doc Ray Grout decided to integrate ADIOS as an alternative I/O mechanism for S3D. ADIOS quickly surpassed the networking limitations imposed by the previous I/O stack.
"I appreciate the clean yet capable interface and that from my perspective, it just works," said Grout. "It would have been fairly difficult for us to get this work finished in time for our targeted paper deadline using our previous I/O solution, but now I'm confident that we'll make it thanks to ADIOS and the help of its team."
S3D's newfound scalability and flexible architecture has led a member of the S3D team, ORNL's Ramanan Sankaran, to explore creating a custom I/O transport method to perform special manipulations to the I/O and the data. With this new transport method, no source code changes would be required.
Thanks to the seamless integration of ADIOS into S3D, the middleware's 1.0 release may have already achieved the team's goal of satisfying 90 percent of ADIOS users "out of the box," said Klasky, adding, "It's really exciting to be able to help the top scientists such as Ray and Jackie obtain results more efficiently. These results make a difference in the world and will hopefully make the world a better place to live."
The ADIOS team thanks all of the scientists who have helped make this work possible, including Hasan Abbasi, Julian Cummings, Divya Dinkar, Ciprian Docan, Stephane Ethier, Garth Gibson, Steven Hodson, Scott Klasky, Zhihong Lin, Qing Liu, Jay Lofstead, Xiaosong Ma, Ron Oldfield, Manish Parashar, Norbert Podhorszki, Milo Polte, Alex Romosan, Ramanan Sankaran, Karsten Schwan, Arie Shoshani, Mladen Vouk, Matthew Wolf, Yong Xiao, Weikuan Yu, and Fang Zhang.
For more information, contact Scott Klasky at klasky@ornl.gov.
(Digg, Technorati, more)
PGI Accelerator™ Fortran 95/03 and C99 compilers for x64+NVIDIA
Accelerate applications on x64+GPU platforms by adding OpenMP-like compiler directives to existing Fortran and C programs. Available now for Linux, MacOS and Windows. Download a free 15 day trial.
Platform HPC Workgroup Manager
Platform HPC Workgroup Manager integrates all the cluster productivity tools you need to deploy, run and manage your HPC environment.
Mar 17 | The Register | But what about the tier ones? Read more...
Mar 17 | Cadalyst Magazine | A new generation of workstations is changing the nature of technical computing. Read more...
Mar 17 | Linux Magazine | Latest iteration of Sun Grid Engine able to tap into Cloud. Read more...
Mar 16 | Bio-IT World | Biotech firm builds genetic models from patient data. Read more...
Mar 15 | The Register | EMC's grand vision for unified global storage. Read more...
Jan 12 | | In-depth look at vSMP Foundation server virtualization technology, technical implementation, use cases and capabilities. The technical whitepaper provides an architectural overview and details on the three vSMP Foundation products: vSMP Foundation for SMP, vSMP Foundation for Cluster and vSMP Foundation for Cloud.
Jan 18 | | This white paper discusses Gore’s copper cable assemblies, and how they continue to exceed the standards for providing reliable, cost-effective solutions for high-performance computer applications.
Join this online panel discussion for live Q&A with leading industry experts, analysts, and end-users to discuss the latest innovations, best practices, barriers to implementation, and measurable benefits of server virtualization with a particular focus on today's real world solutions.
Learn about scalable fault-tolerant architectures and examples of energy efficient and scalable supercomputing clusters using dual QDR InfiniBand to combine capacity computing with network failover capabilities with the help of programming languages such as MPI and a robust Linux cluster management package.
LIVE@SCO9: The IBM team discusses new innovations in hardware, software and services that help clients better understand their workloads and get insight from their R&D efforts. Technology demonstrations include the soon-to-be-released Power7 HPC processor, the DCS990 system with 2.4 petabytes of storage, the xCAT management tool, secure HPC cloud computing and more. Winners of two HPCwire Readers' and Editors’ Choice Awards! Take the IBM virtual tour at SC09 or more information go online to: http://www-03.ibm.com/systems/deepcomputing/sc09.html