The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing
April 29, 2008
Indiana University Announces Lustre over WAN
SONOMA, Calif., April 29 -- University Information Technology Services (UITS) at Indiana University announced today that it has dedicated over 350 terabytes (TB) of new storage platforms to support collaborative research projects mounting the Lustre file system across the TeraGrid network and other national high speed networks. IU's Data Capacitor Project Lead Stephen Simms made the announcement during a panel discussion at the Sun Microsystems-sponsored 2008 Lustre User Group being held this week in Sonoma, California.
The ability to use Lustre over a wide area network (WAN) is a significant advancement in the ongoing struggle to meet user demand for easier and faster access to stored research data generated by high performance computers.
"A large impediment to maintaining a wide area file system that spans multiple organizations is the need for a unified user identification scheme across all clients," said Simms, who also serves as IU's TeraGrid site lead.
Through small modifications to the Lustre code base, Indiana University -- with help from DataDirect Networks -- has recently developed a lightweight scheme for mapping user IDs (UIDs) across multiple distributed clients, ensuring that file ownership permissions will remain consistent across all clients.
This development is a critical step toward widespread use of Lustre as a wide area file system, and will provide greater security and ease of use. Early Lustre WAN projects required strict synchronization of UIDs between client systems; IU's UID mapping scheme eliminates the need for this strict synchronization. Implementation of this new code will grant appropriate access to only those client UIDs that have been registered with the Lustre administrator.
Indiana University has made several notable achievements related to the use of Lustre over a WAN in the past year. In June, IU announced that its Data Capacitor -- a 535 TB Lustre file system designed to store and manipulate large data sets -- demonstrated in its opening weeks of production a single client transfer rate of 977 MB per second across the TeraGrid network. In November, a team led by Indiana University was awarded first place in the annual Bandwidth Challenge at the SC07 Conference in Reno, Nevada; using the Data Capacitor, the IU team achieved a peak transfer rate of 18.21 Gigabits/second out of a possible 20 Gigabits/second. This performance was nearly twice the peak rate of the nearest competitor.
"Indiana University has had amazing success using Lustre in the last several months," said Simms. "We've shown how the Data Capacitor allows researchers to access, manage, and manipulate massive data sets quickly and easily from remote locations. We are very excited to extend that work and offer a new resource dedicated to wide area collaborations."
The Data Capacitor project is supported in part by the National Science Foundation under NSF Award Number CNS0521433 (Craig Stewart, PI; Stephen Simms, Co-PI and project manager; Caty Pilachowski, Randall Bramley and Beth Plale, Co-PIs). IU's involvement in the TeraGrid is supported in part by NSF grants ACI-0338618l, OCI-0451237, OCI-0535258, and OCI-0504075.
About UITS
University Information Technology Services at IU, with offices on the Bloomington and Indianapolis campuses, develops and maintains a modern information technology environment throughout the university in support of IU's vision for excellence in research, teaching, outreach, and lifelong learning. UITS provides tools and services to support the academic and administrative work of the university, including a high-speed campus network with wireless access, central web hosting, a rich selection of free and low-cost software for personal use, tools and support for instruction and research, and supercomputers for data analysis and visualization.
-----
Source: Indiana University
Sponsored Links
Interview: Appro CEO Shares HPC Vision
Appro CEO Daniel Kim provides a glimpse into Appro's vision and opportunities for its supercomputer and high-performance cluster solutions.
RSS Feeds
Feeds by Topic
- Applications
- Developer Tools
- Interconnects
- Middleware
- Networks
- Processors
- Storage
- Systems
- Visualization
Feeds by Industry
Feeds by Content Type
Feature Articles
Computed Tomography Software Taps Into NVIDIA GPUs
Minnesota-based North Star Imaging, a firm that specializes in industrial X-rays for nondestructive testing and analysis, is employing NVIDIA GPUs to accelerate 3D renderings in their CT (computed tomography) software. Julien Noel, the company's CT product manager, says the exceptional computational power afforded by CUDA and Tesla hardware is increasing customer productivity and transforming their workflow.
Read More...
The Next Big Thing in Humanities, Arts and Social Science Computing: 18thConnect
For the humanities scholar who may have only recently mastered library and archival finding aids beyond the archaic card catalog, the possibility of retrieving source materials at the flash of a keystroke (well maybe a few...) is very heady stuff.
Read More...
HPC Clouds -- Alto Cirrus or Cumulonimbus
The "cloud" model of exporting user workload and services to remote, distributed and virtual environments is emerging as a powerful computing paradigm. Yet, one domain that challenges this model in its characteristics and needs is high performance computing.
Read More...
Top Headlines
Dawning 6000 to Use Chinese-Made Loongson Processor
Nov 28 | People's Daily Online | Currently under development, the Dawning 6000 HPC system will be based on the Chinese-made "Loongson" microprocessor. Read more...
Can Supercomputers Help Save the Economy?
Nov 27 | Computerworld | The use of supercomputers to increase the industrial might of the U.S. has amounted to little more than an asterisk from a financial standpoint in both the federal budget and the economy as a whole. Read more...
IBM to Establish 'Collaboratory' in Dublin
Nov 26 | Science Business | IBM is getting ready to set up a supercomputing research “collaboratory” in Dublin, Ireland. Read more...
Texan Prof Sees Big Future for Graphene Storage
Nov 25 | The Register | A Rice University professor believes that his proposed graphene arrays could be many times denser and faster than existing storage tech, and they'd be more reliable too. Read more...
Super Micro Computer: A One-Man, or at Least One-Family, Powerhouse
Nov 24 | The New York Times | Server maker Super Micro Computer lives by two principles: give customers what they want, and do it as fast as humanly possible. Read more...
Multimedia
Video White Paper: Architecting a Better Network Storage Solution
BlueArc's Titan architecture represents an evolutionary step in file servers by creating a hardware-based file system that can scale bandwidth, IOPS, and overall data capacity well beyond conventional software-based devices. With its ability to virtualize a massive storage pool of up to four usable petabytes of tiered storage, Titan can scale with growing data requirements, offering a competitive advantage for businesses, researchers, or other enterprises seeking to better manage data growth while still ensuring optimal performance.
Newsletters
Stay informed! Subscribe to HPCwire email Newsletters.
Get updates and insights on the High Productivity Computing industry delivered driectly to your inbox.
HPC Job Bank
-
Sr. Systems Developer/Engineer
Ohio State University -
HPC System Administration Specialist
Argonne National Laboratory - Visit the HPCwire Job Bank
Featured Events
-
February 24-27, 2009
TradeTech Architecture 2009
London England
-
April 13-16, 2009
Symposium 2009: LEAN, CLEAN, and GREEN
New York , NY
USA