HPCwire: SAN Solutions for HPC Storage

The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing

HPCwire >> Features

April 20, 2007

SAN Solutions for HPC Storage

by Tim Lustig, Senior Manager of Technical Marketing, QLogic Corp.

Page: 1 of 4
1 | 2 | 3 | 4 All »

The rise of multi-server clusters has changed the landscape of computing dramatically and permanently. Not only are clusters surpassing yesterday's behemoth supercomputers, they are also triggering a concurrent revolution in storage architecture. Today's high-performance computing (HPC) environment not only breaks the boundaries that stymied Teraflop progress for decades, but does so at far lower cost than supercomputers.

Storage needs are skyrocketing as content and applications grow more complex and sophisticated. Nowadays, an organization needn't be working at supercomputer levels to have massive storage requirements -- which explains in part the growing exodus from direct attached storage to the "shared" storage area network (SAN) approach. Until recently, the innovative SAN infrastructure was almost entirely overlooked by administrators who specialized in optimizing transaction performance. But increasingly, these same administrators are recognizing the ability of SANs to centrally manage and scale their ever increasing storage needs.

The SAN Is Rising

As most administrators know, using multiple file servers to support a giant cluster increases the cost, complexity and management of the environment. Simply adding more storage is difficult and disruptive when dealing with data-intensive applications. As new volumes and mount points are added, capacity and bandwidth must be balanced and re-balanced across multiple servers. While clusters can generate trillions of calculations per second, managing that huge data volume is an issue. These and other challenges are why, until recently, advances in processing performance easily outpaced scalable storage. And why the unique abilities of SANs are making inroads into supercomputing.

For over a decade, SANs have gone through the typical product life cycle. It has gone from early adopter Fortune 100 companies making the first purchases in an effort to find a way to consolidate storage and save datacenter costs up to today, where now small- to medium-sized businesses can afford to own and operate a SAN. Along the way, cost and complexity was removed from the implementation and management in order to accelerate adoption.

A similar effect has happened to HPC with the use of general-purpose CPUs and open systems software (Linux, clustering and file systems), both of which have provided for the removal of cost, complexity and size of the compute engine. Both technologies are at the crest of a new wave of adoption in each other's markets.

The right SAN in an HPC environment can support the huge capacity and aggregate throughput of the cluster or grid, plus perform dynamic load balancing and data redistribution, spreading data across the storage architecture and providing a single point of management and namespace for a file system. A relatively low-cost SAN investment can easily surpass attached storage in terms of utilization, flexibility, scalability and performance.

Nevertheless, to fulfill its promise, networked storage must be simple, transparent and inexpensive to implement, so that its costs/benefits prevail in the real world. Finding the optimal storage strategy is a challenge that administrators must first recognize, then move forward to develop the architecture, find the right vendor, implement the solution economically -- and manage it all. In the past, this transition may have seemed risky and daunting to administrators, who elected to stay with their old solutions. By now, however, SANs have the momentum from the commercial marketplace to leverage its adoption into the scientific marketplace.

Early Adoption of SANs in HPC

For studios producing animated films or special effects, success in terms of deadlines, cost, audience appeal, revenue and future viability now depends heavily on SANs. Achieving smooth transitions, fine detail of motion, texturing, shadowing and tonal qualities requires compute power and unprecedented amounts of data, which must travel among workstations at lightning speed throughout post-production, editing and other processes. CGI (computer-generated imagery) data transfer and editing needs a high-bandwidth, low-latency system -- and that's where the SAN shines.

Page: 1 of 4
1 | 2 | 3 | 4 All »

Article Tools

Share Options

(Digg, Technorati, more)

Subscribe

Subscribe to HPCwire

Discussion

There are 0 discussion items posted.

Join the Discussion

You must be a registered user in order to comment on HPCwire stories.

Members enjoy:

	Access to Special Content, including Podcasts, Webcasts, and Whitepapers and more!
	Ability to take an active role in the discussions that are shaping the HPC technology discipline!
	Special offers available only to HPCwire members!

Become a Registered User Today!

Registered Users Log in Here to Comment

Email Address:	Password (case sensitive)

Remember me	Forgot Password?

RSS Feeds

Feeds by Topic

Feeds by Industry

Feeds by Content Type

Top Headlines

Intel Partners See 'Easy' Upgrade Path With Xeon 5600 Chips

Mar 18 | ChannelWeb | Westmere parts already showing up in HPC machines. Read more...

AMD: OEMs primed for Opteron 6100s

Mar 17 | The Register | But what about the tier ones? Read more...

Arrival of the Desktop Supercomputer

Mar 17 | Cadalyst Magazine | A new generation of workstations is changing the nature of technical computing. Read more...

Scheduling HPC In The Cloud

Mar 17 | Linux Magazine | Latest iteration of Sun Grid Engine able to tap into Cloud. Read more...

Tailoring Medicine with Supercomputers

Mar 16 | Bio-IT World | Biotech firm builds genetic models from patient data. Read more...

Read more headlines...

Featured Whitepapers

Virtualization for Aggregation And The vSMP Architecture™

Jan 12 | | In-depth look at vSMP Foundation server virtualization technology, technical implementation, use cases and capabilities. The technical whitepaper provides an architectural overview and details on the three vSMP Foundation products: vSMP Foundation for SMP, vSMP Foundation for Cluster and vSMP Foundation for Cloud.

Copper Cable Technologies for High Performance Computing

Jan 18 | | This white paper discusses Gore’s copper cable assemblies, and how they continue to exceed the standards for providing reliable, cost-effective solutions for high-performance computer applications.

View the White Paper Library

Multimedia

Webcast: Virtualized Data Center Roundtable

Join this online panel discussion for live Q&A with leading industry experts, analysts, and end-users to discuss the latest innovations, best practices, barriers to implementation, and measurable benefits of server virtualization with a particular focus on today's real world solutions.

Webcast: Watch SC09 Birds of a Feather Video: Scalable Fault-Tolerant HPC Supercomputers

Learn about scalable fault-tolerant architectures and examples of energy efficient and scalable supercomputing clusters using dual QDR InfiniBand to combine capacity computing with network failover capabilities with the help of programming languages such as MPI and a robust Linux cluster management package.

Webcast: High Performance Computing for a Smarter Planet

LIVE@SCO9: The IBM team discusses new innovations in hardware, software and services that help clients better understand their workloads and get insight from their R&D efforts. Technology demonstrations include the soon-to-be-released Power7 HPC processor, the DCS990 system with 2.4 petabytes of storage, the xCAT management tool, secure HPC cloud computing and more. Winners of two HPCwire Readers' and Editors’ Choice Awards! Take the IBM virtual tour at SC09 or more information go online to: http://www-03.ibm.com/systems/deepcomputing/sc09.html