Visit additional Tabor Communication Publications
March 07, 2013
For researchers plowing through some of the toughest scientific and large-scale computing challenges, system performance is only part of the problem. With the influx of big, diverse datasets, the role of high performance storage is becoming increasingly important.
|NetApp E5500 storage system|
To address these issues, NetApp and SGI made simultaneous announcements yesterday of new storage systems specifically targeted at HPC and big data applications. NetApp launched its latest offering in the E-Series line of controllers yesterday, the NetApp E5500. The E-Series is the product line from Engenio, which NetApp bought from LSI in early 2011.
SGI, an OEM partner of NetApp, has been incorporating the E-Series products into its RAID systems for years. In fact, the company has already adopted NetApp's new E5500 in its InfiniteStorage 5600 RAID system, also released yesterday, and it has made at least one pre-sale to an Australian cosmological research center.
NetApp says the E5500 has the highest speed, scale, cost-efficiency and scalability on the market, along with a very small footprint. To back up these claims, SGI showed off some audited, peer-reviewed SPC-2 benchmark results for its RAID systems using the controller. As for speed, the IS5600 achieved a per-spindle throughput of over 70 MB/second, some 2.5 times higher than the second-fastest product (from Sun).
"It's all about performance for this set of customers," says Bill Peterson, senior manager for Hadoop and big data analytics at NetApp. "Our customers have traditionally built all their systems around highly structured database schemas. Then you go to something like Hadoop, for example, and there is no schema. They want performance. At the end of the day, whether it's for big data or HPC, they're trying to gain insight, and that insight is always required quickly."
To reach its target audience, NetApp also offers preconfigured solutions optimized for applications such as high performance Lustre, full-motion video, media content management and seismic processing. With the IS5600, SGI also offers dedicated, application-specific support for parallel file systems, including Lustre, Hadoop and CXFS.
The E5500 "reaffirms NetApp's commitment to high-performance storage, which is important for organizations seeking to balance HPC and big data scalability and performance with the reliability of an enterprise-class storage provider," says Addison Snell, CEO of Intersect360 Research. "This release also underscores the ongoing maturation of the Lustre community, which is essential to bringing parallel file system technology to a wider range of users."
Speed isn't everything. SGI also shows off SPC-2 benchmark tests that give its RAID product the best cost-efficiency on the market, at about $15 per MBps performance. That's about half the cost per MBps of the next most efficient system, from Fujitsu.
SGI is also targeting its IS5600 RAID platform at HPC systems. Both the IS5600 and its previous generation IS5500 (based on NetApp's E5400 storage system) can scale to 384 drives and 1.2 petabytes, but the IS5600 offers 2.5 times more I/O. The combination of NetApp's storage technology and SGI's RAID configuration "really blazes a new trail, especially for price/performance," says Floyd Christofferson, director of storage product marketing at SGI. "You can't beat it for HPC."
Christofferson is also impressed with NetApp's SANtricity Dynamic Disk Pools, or DDE, which provides improved fault-tolerance for RAID. DDE uses a proprietary algorithm to distribute data, parity information and spare capacity across a pool of drives. Unlike RAID 6, which can generally only tolerate one drive failure before having to take the RAID offline to replace faulty drives, with DDE many drives can fail and the system can recover very quickly without being taken offline.
"You can keep failing disks all day long" says Christofferson. "As long as you have more than 11 drives in there, you're good to go. It will come back to performance within minutes, as opposed to hours or days" for RAID 6. "This could be a viable successor to RAID 6," he says.
SGI won't be the only company debuting systems with NetApp's new devices. IBM, RAID Inc., Teradata and others are all NetApp OEM partners.
But Christofferson says SGI already has "several customers in the funnel," for its new IS5600, which is expected to start shipping this April. One firm order is for a research facility in Australia, conducting very high-end research on fields such as general relativity; galaxies, cosmology, dark matter and energy; and even "transient Radio Phenomena caused by Extraterrestrial Life." That facility will be deploying a 100PB system from SGI this summer. It will use 27 IS5600 systems with 6 PB of storage and 94 PB of virtual storage.
It takes a lot of fast storage to read messages from ET calling long distance.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
May 15, 2013 |
Supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) have worked on important computational problems such as collapse of the atomic state, the optimization of chemical catalysts, and now modeling popping bubbles.
May 10, 2013 |
Program provides cash awards up to $10,000 for the best open-source end-user applications deployed on 100G network.
May 09, 2013 |
The Japanese government has revealed its plans to best its previous K Computer efforts with what they hope will be the first exascale system...
May 08, 2013 |
For engineers looking to leverage high-performance computing, the accessibility of a cloud-based approach is a powerful draw, but there are costs that may not be readily apparent.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
In this demonstration of SGI DMF ZeroWatt disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.
The Cray CS300-AC cluster supercomputer offers energy efficient, air-cooled design based on modular, industry-standard platforms featuring the latest processor and network technologies and a wide range of datacenter cooling requirements.