HPCwire: Myricom Demonstrates Low-Latency 10-Gigabit Ethernet

The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing

HPCwire >> Off the Wire

June 30, 2006

Myricom Demonstrates Low-Latency 10-Gigabit Ethernet

At the International Supercomputer Conference in Germany this week, Myricom is introducing new message-passing software for its Myri-10G products, delivering 10-Gigabit Ethernet latencies formerly associated only with specialty high-performance computing interconnects. Myricom extended its Myrinet Express (MX) software, already widely used in HPC clusters interconnected with Myrinet, to also operate over 10-Gigabit Ethernet. "MX over Ethernet" works with Myricom's dual-protocol Myri-10G network-interface cards (NICs) and conventional 10-Gigabit Ethernet switches to achieve latencies 5 to 10 times lower than with TCP/IP over Ethernet. MX/Ethernet delivers 2.4 us latency at the application level, 1.2 GB/s (9.6 Gigabit/s) throughput, and very low host-CPU utilization. These performance metrics are nearly on par with those achieved by MX over Myrinet. MX/Ethernet extends the power of HPC beyond clusters and grids to enterprise applications already employing Ethernet switching.

Creating a new standard in open Ethernet networking, MX/Ethernet is "plug-and-play," transparent to Ethernet switch makers, less expensive than proprietary HPC solutions, and applicable both to HPC and to enterprises. Besides latencies 5 to 10 times lower than TCP/IP over Ethernet, MX/Ethernet provides host-CPU utilization in a range below 1 percent, e.g., 1 us of host CPU time to transfer one MB in 833 us.

"Networking experts have known for ages that there is no technical reason why Ethernet can't perform better," says Dr. Chuck Seitz, founder and CEO of Myricom. "What we're doing with MX/Ethernet is creating an open solution that brings the best of HPC to Ethernet with no compromises in terms of standards, interoperability, or software compatibility. One can think of this convergence as 'HPC for everyone.'"

Myricom's Myri-10G solutions introduced a convergence at 10-Gigabit/s data rates of Myrinet, one of the most successful specialty network for HPC applications, and mainstream Ethernet. Myri-10G was designed to create an open, flexible, new standard for HPC interconnects and high-end enterprise networking. Dual-protocol Myri-10G Network Interface Cards (NICs) initially achieved optimal performance running MX software using the Myrinet protocol through Myri-10G switches. MX's kernel-bypass techniques achieve low latency and low host-CPU overhead by allowing application programs to communicate directly with firmware in the programmable Myri-10G NICs. Now, the availability of MX/Ethernet extends MX's techniques to standard 10-Gigabit Ethernet switching. OEMs and cluster integrators can achieve HPC performance with purely open technology. Myricom will make the MX/Ethernet protocols fully open and accessible, just as with earlier Myrinet protocols and source code.

MX/Ethernet with Myri-10G NICs uses 10-Gigabit Ethernet as a layer-2 network with an MX EtherType to identify MX packets. The EtherType, a part of the Ethernet standards since the earliest days, identifies the protocol of an Ethernet frame. For example, there are EtherTypes for the Internet Protocol (IP), Address Resolution Protocol (ARP), and AppleTalk, and all of these protocols can be carried concurrently on the same Ethernet network. Ethernet switches normally ignore the EtherType. Myri-10G NICs can carry TCP/IP and other traffic along with MX traffic, but achieve the best performance when they circumvent TCP/IP.

Early MX/Ethernet trials were conducted using a Fulcrum Microsystems FM2224, 24-port, 10-Gigabit switch and a Fujitsu XG700, 12-port, 10-Gigabit Ethernet switch. Both of these layer-2 switches are relatively low-latency, and delivered user-level latency and host-CPU utilization comparable with pure Myrinet configurations. These solutions will be limited to smaller clusters that can be served with a single 10-Gigabit Ethernet switch, because of performance losses in building larger networks by connecting multiple Ethernet switches. Inasmuch as there are no high-port-count, low-latency, full-bisection, 10-Gigabit Ethernet switches on the market today, MX/Myrinet with 10-Gigabit Myrinet switches will continue to be preferred for large clusters because of the economy and scalability of Myrinet switching.

"MX/Ethernet provides strong new evidence that 10-Gigabit Ethernet will become the interconnect technology of choice for HPC clusters, initially for small clusters, but, as 10-Gigabit Ethernet switch technology advances, for larger clusters as well," says Seitz. "Gigabit Ethernet dominates HPC clustering and the TOP500 Supercomputer list today, but is too slow for high-end cluster hosts with multiple, multi-core processors. Both in HPC and in enterprise networking, the MX/Ethernet innovation will enable 10-Gigabit Ethernet to gain market share rapidly over specialty networks. I'd like to think Bob Metcalfe would be proud."

Article Tools

Share Options

(Digg, Technorati, more)

Subscribe

Subscribe to HPCwire

Discussion

There are 0 discussion items posted.

Join the Discussion

You must be a registered user in order to comment on HPCwire stories.

Members enjoy:

	Access to Special Content, including Podcasts, Webcasts, and Whitepapers and more!
	Ability to take an active role in the discussions that are shaping the HPC technology discipline!
	Special offers available only to HPCwire members!

Become a Registered User Today!

Registered Users Log in Here to Comment

Email Address:	Password (case sensitive)

Remember me	Forgot Password?

RSS Feeds

Feeds by Topic

Feeds by Industry

Feeds by Content Type

Feature Articles

The Week in Review

C-DAC announces plans for a petaflop system; IBM researchers are working on vertical integration techniques to extend Moore's Law another 15 years. We recap those stories and more in our weekly wrapup.
Read More...

Moscow State University Supercomputer Has Petaflop Aspirations

The Moscow State University supercomputer, Lomonosov, has been selected for a high-performance makeover, with the goal of tripling its processing power to achieve petaflop-level performance in 2010. T-Platforms, who developed and manufactured the supercomputer, is the odds-on favorite to lead the project.
Read More...

Intel Ups Performance Ante with Westmere Server Chips

Right on schedule, Intel has launched its Xeon 5600 processors, codenamed "Westmere EP." The 5600 represents the 32nm sequel to the Xeon 5500 (Nehalem EP) for dual-socket servers. Intel is touting better performance and energy efficiency, along with new security features, as the big selling points of the new Xeons.
Read More...

Read more HPCwire features...

Top Headlines

Intel Partners See 'Easy' Upgrade Path With Xeon 5600 Chips

Mar 18 | ChannelWeb | Westmere parts already showing up in HPC machines. Read more...

AMD: OEMs primed for Opteron 6100s

Mar 17 | The Register | But what about the tier ones? Read more...

Arrival of the Desktop Supercomputer

Mar 17 | Cadalyst Magazine | A new generation of workstations is changing the nature of technical computing. Read more...

Scheduling HPC In The Cloud

Mar 17 | Linux Magazine | Latest iteration of Sun Grid Engine able to tap into Cloud. Read more...

Tailoring Medicine with Supercomputers

Mar 16 | Bio-IT World | Biotech firm builds genetic models from patient data. Read more...

Read more headlines...

Featured Whitepapers

Virtualization for Aggregation And The vSMP Architecture™

Jan 12 | | In-depth look at vSMP Foundation server virtualization technology, technical implementation, use cases and capabilities. The technical whitepaper provides an architectural overview and details on the three vSMP Foundation products: vSMP Foundation for SMP, vSMP Foundation for Cluster and vSMP Foundation for Cloud.

Copper Cable Technologies for High Performance Computing

Jan 18 | | This white paper discusses Gore’s copper cable assemblies, and how they continue to exceed the standards for providing reliable, cost-effective solutions for high-performance computer applications.

View the White Paper Library

Multimedia

Webcast: Virtualized Data Center Roundtable

Join this online panel discussion for live Q&A with leading industry experts, analysts, and end-users to discuss the latest innovations, best practices, barriers to implementation, and measurable benefits of server virtualization with a particular focus on today's real world solutions.

Webcast: Watch SC09 Birds of a Feather Video: Scalable Fault-Tolerant HPC Supercomputers

Learn about scalable fault-tolerant architectures and examples of energy efficient and scalable supercomputing clusters using dual QDR InfiniBand to combine capacity computing with network failover capabilities with the help of programming languages such as MPI and a robust Linux cluster management package.

Webcast: High Performance Computing for a Smarter Planet

LIVE@SCO9: The IBM team discusses new innovations in hardware, software and services that help clients better understand their workloads and get insight from their R&D efforts. Technology demonstrations include the soon-to-be-released Power7 HPC processor, the DCS990 system with 2.4 petabytes of storage, the xCAT management tool, secure HPC cloud computing and more. Winners of two HPCwire Readers' and Editors’ Choice Awards! Take the IBM virtual tour at SC09 or more information go online to: http://www-03.ibm.com/systems/deepcomputing/sc09.html