SCIENCE AND ENGINEERING NEWS
Oak Ridge, TN — Today’s scientific researchers are generating staggering volumes of data, with data sources ranging from computational simulations of global and regional climate, to digital instrumentation of physical experiments and satellite imagery. Add projects such as human genome mapping, with massive demands for rapid user access, and it’s easy to see why strategies for optimizing data storage and retrieval are vital for research laboratories.
The Oak Ridge National Laboratory (ORNL) and the National Energy Research Scientific Computing Center (NERSC) are tackling the storage challenge head-on with Probe, a newly established distributed testbed for storage-intensive applications which combines the high-speed networking of the latest ESnet technology with the R&D 100 award winning High Performance Storage System (HPSS). Probe will have significant installations at both ORNL and NERSC, providing access to researchers around the country.
The Probe testbed, funded by the Department of Energy’s Office of Science, is available for researchers from the scientific community to perform comparative evaluations of the latest technologies in storage hardware and software. By linking the two testbed systems together over the network, researchers will be able to evaluate the effects of network latency in remote storage access and develop new protocols for effectively using distributed storage systems. The testbed will also provide a platform for the developers of new storage and networking hardware and software to test their devices in high-demand facilities.
Marvin Frazier, Director, Life Sciences Division, DOE’s Office of Biological and Environmental Research, noted that “with the flood of genome data coming from human, model organism and microbial sequencing projects, better and faster tools that facilitate comparative genome analyses will be absolutely critical. New physical systems and novel design configurations will be crucial to enabling these research efforts to fulfill their potential.”
Researchers can modify or augment the configuration of Probe as needed, for instance, to perform comparative evaluations of equipment from various vendors or to test the throughput of a proposed configuration. Probe will be used to study strategies for exploiting wide-area, high-bandwidth networks connecting terascale data archives across the country. With a variety of network technologies installed, Probe can be used to explore new methods for high-speed transfers from storage to remote visualization systems.
“Extracting scientific understanding from petabytes of data will be a critical challenge for the Department of Energy and the US scientific community in the next decade,” said Dan Hitchcock, Acting Division Driector for Mathematical Information and Computational Sciences Division, DOE’s Office of Science. “The Probe testbed will enable important experiments in the technology needed to address this challenge. We believe that having this experimental testbed available to the data storage and managment research community will accelerate progress in this important area.”
HPSS, a software system providing gigabyte/second throughput and petabyte-scale capacity, is a development of IBM Global Services, Government Industry and five Department of Energy installations (the Oak Ridge, Los Alamos, Lawrence Livermore and Sandia National Laboratories and NERSC). HPSS is in production use at many of the leading research institutions, supercomputer centers, and universities around the world.
For more information on the Probe facility, visit the web site at http://www.ccs.ornl.gov/PROBE.html
============================================================