HPCwire: Florida State Gives Virtual SMPs a Spin

Since 1986 - Covering the Fastest Computers in the World and the People Who Run Them

HPCwire >> Features

March 10, 2010

Florida State Gives Virtual SMPs a Spin

by Michael Feldman, HPCwire Editor

Page: 1 of 2
1 | 2 All »

The prospects for virtual SMP technology got another boost last month when Florida State University (FSU) announced it had installed a new HPC system from 3Leaf Systems. The servers are being housed at the university's HPC facility and will be used across a range of scientific disciplines. It represents the first announced shipment of 3Leaf's virtual SMP offering.

A relative newcomer to the HPC space, 3Leaf launched its initial server product, called the "Dynamic Data Center Server" (DDC-Server) in November 2009. Using special hardware and software, it enables a set of x86 server nodes to be aggregated into one or more virtual SMP machines. The 3Leaf server shipping today is based on dual-socket Opteron motherboards supplied by Supermicro.

The idea is to provide a platform for applications that require high core count and/or high memory environments, but without the expense of a hard-wired SMP system. And since the aggregation is all done virtually, users can rejigger the SMP configuration on the fly to match different application needs. It's basically having a cluster and eating it too.

In general, the 3Leaf technology mirrors that of competitor ScaleMP. Unlike ScaleMP though, which does all its virtualization magic in software, 3Leaf has devised a custom ASIC that manages cache coherency at the hardware level. Also unlike ScaleMP, the 3Leaf product is integrated with AMD Opteron-based motherboards. Currently, ScaleMP only supports Intel Xeon-based servers.

This actually turned out to be an important consideration for the university. Since FSU is primarily an AMD shop, sticking with Opteron hardware meant ruling out the ScaleMP solution. The other attraction to 3Leaf was the better promised performance for their ASIC-based cache coherency mechanism.

The new system at FSU is based on six-core Istanbul CPUs, and consists of 12 dual-socket servers connected over QDR InfiniBand. The system sports 576 GB of memory and 6 TB of disk storage. Of the 144 total cores, 138 are available to user apps, with the remainder dedicated to the operating system. Although FSU is not saying how much they paid for the servers, according to 3Leaf, the list price for a system this size is $206 thousand.

On top of the hardware is the resource allocation software. 3Leaf offers different variations of resource management, depending on how you want to slice and dice the servers. The software chosen by FSU was DDC-Pool, which can construct virtual SMP machines at the granularity of the cluster node. Also, with DDC-Pool, reconfiguration requires a system reboot. Obviously, this type of setup implies that it will be used for applications that run for a relatively long time before a reallocation of CPUs and memory is needed.

Since the new system is part of the university's shared HPC facility, it's intended to support any FSU researcher. Currently the facility supports about 450. Jim Wilgenbusch, who heads the shared HPC facilty at university, estimates around 25 percent of those users share a need for SMP-type systems with larger memory or larger numbers of CPU cores. To satisfy the demand, they've set up a "SMP queue" into which anyone can request a specific number of processors and memory footprint. As needed, the system's CPUs and RAM are allocated to users as a virtual system and then released back into the resource pool when the application completes.

Wilgenbusch thinks the main application areas that will make good use of the SMP technology are astrophysics, earth and atmospheric science, and imaging applications. FSU recently received a large NIH grant for a cryo-electron microscope, which generates reams of 3-D images for post-processing. Wilgenbusch says that's just the type of application well-suited to the 3Leaf platform, since it tends to need a much larger memory footprint than can be had on a single physical node.

Theoretically, one application could hog all 138 cores, but most of the user requests they're getting are for allocations of 44 to 128 cores, with approximately 44 GB per core. As in most shared systems, the bigger the request, the longer the wait for resources.

Page: 1 of 2
1 | 2 All »

Article Tools

Share Options

(Digg, Technorati, more)

Subscribe

Subscribe to HPCwire

Discussion

There are 0 discussion items posted.

RSS Feeds

Feeds by Topic

Feeds by Industry

Feeds by Content Type

Around the Web

Picking the Right Processor

Sep 03 | Should engineers take advantage of GPU computing? Read more...

HP, Hynix Start Memristor on Path to Commercialization

Sep 02 | Could see first products in three years. Read more...

TED Talks for the IT Crowd

Sep 01 | A hand-picked selection of video presentations from the TED conference -- because the next big thing has to start somewhere. Read more...

LHC Compute Grid Teaches Some Valuble Lessons

Aug 30 | CERN project adapts its computation and storage strategy as hardware gets cheaper and better. Read more...

Godson CPUs Groomed for Supercomputing Duty

Aug 26 | Chinese-made chip adds vector SIMD unit; delivers 128 gigaflops in 40 watts. Read more...

Read more headlines...

Featured Whitepapers

Effective Backup and Restore

Jul 29 | | Panasas storage solutions deliver high throughput with many concurrent backup IO streams to standard backup applications such as Veritas NetBackup™ or EMC® NetWorker™. Download this whitepaper to understand the essential elements for effective backup and restore: the tape subsystem, networking, file system workload and administrative policy.

GPU Cluster Realities Whitepaper from Platform Computing

Jul 28 | | As compelling economics and performance drive GPUs into HPC clusters, developers are scrambling to catch up. Download this whitepaper from Platform Computing to understand how to capture the benefits of exciting new GPU capabilities.

View the White Paper Library

Multimedia

Webcast: Are you drowning in data?

In this webinar you will hear about the current storage challenges facing the HPC community, how Panasas storage solutions provide exceptional performance, scalability, and manageability, and how you can achieve the lowest total Cost of Ownership with a system that installs and configures in 15 minutes.

Webcast: Virtualized Data Center Roundtable

Join this online panel discussion for live Q&A with leading industry experts, analysts, and end-users to discuss the latest innovations, best practices, barriers to implementation, and measurable benefits of server virtualization with a particular focus on today's real world solutions.

Webcast: Watch SC09 Birds of a Feather Video: Scalable Fault-Tolerant HPC Supercomputers

Learn about scalable fault-tolerant architectures and examples of energy efficient and scalable supercomputing clusters using dual QDR InfiniBand to combine capacity computing with network failover capabilities with the help of programming languages such as MPI and a robust Linux cluster management package.