Silicon Valley Symposium Heats Up Discussion of High Performance Interconnects

By Michael Feldman

August 3, 2011

IEEE’s Hot Interconnects symposium that kicks off later this month should be a real treat for the HPC crowd. The event focuses exclusively on cutting-edge developments in the interconnect arena, everything from the latest commodity networking technologies to the K supercomputer’s “Tofu” custom network. For the most part, the two-day program, which takes place at Intel’s Silicon Valley headquarters in Santa Clara on August 24-25, is geared toward developers and researchers in the field, with a day of tutorials on August 26.

The sessions reflect the dynamics between custom interconnects, designed for top-end supercomputers, and commodity technologies like Ethernet and InfiniBand. We asked the technical chairs of the event — IBM researcher Fabrizio Petrini, along with University of Illinois’ Torsten Hoefler, and Myricom’s Patrick Geoffray — how they perceive this dichotomy and how it will play out in the future as the industry moves toward cloud computing, and in the HPC world, toward exascale supercomputing. Here’s what they had to say.

HPCwire: Despite the commercial advantages of industry-standard interconnects like Ethernet and InfiniBand, there always seems to be a demand for proprietary ones like Cray’s Gemini and Fujitsu’s Tofu. What do you see as the principal drivers for these custom interconnects?

Proprietary interconnects will be needed to satisfy the special demands of the extreme-scale community. Currently, none of the available general-purpose interconnects is designed to serve such a large number (>10.000) of endpoints in a single LAN. Thus, such proprietary networks will drive innovation at large scale and may guide standardization. However, such innovation often requires substantial investments, often at national scales — for example in the Tofu or PERCS networks.

Commodity interconnects, such as Ethernet and InfiniBand serve different markets — much larger, but at a much smaller scale of single LAN segments. The standardization bodies typically move slowly and are constrained by compatibility issues. Thus, quick innovation in those areas is harder. Mellanox’ approach to large-scale is hybrid. They introduce custom features to commodity interconnects, for example, CORE-Direct, to serve separate communities.

HPCwire: How do you see this trend developing? Will standard interconnects eventually push out proprietary ones or do you think there will always be a role for the proprietary technologies?

At the extreme scale, there will always be custom interconnects. For exactly the same reason that there will be custom cars in Formula 1. But for the general high-performance computing area, commodity interconnects are already dominating the market. The arguments here are simply that there is no need to extreme scalability or absolutely highest performance. We can expect that some features trickle down from the custom interconnect area to commodity interconnects. For example, collective communication offload may be such a feature that could soon be available to a wider audience.

HPCwire: With the advent of cloud computing, Sun Microsystems’ vision of “the network is the computer” seems to be coming true in a very fundamental way. Will cloud computing impact the interconnect landscape?

Yes and no. Depending on what strategy wins for datacenter design: the high-performance datacenter or the commodity Google-style server farm. Both approaches have advantages and disadvantages. But for large-scale computing of any kind, for example, MapReduce, one needs a strong interconnect. However, due to price and fierce competition in this area, I expect commodity interconnects to rule this field. This is why our program, especially the tutorial, also focusses on latest developments for commodity interconnects.

HPCwire: What role will standard interconnects like Ethernet and InfiniBand play in exascale systems?

That is an interesting question. Standard interconnects will probably play only a side role. However, extensions to InfiniBand could be worthwhile to consider. We will have a presentation by Mellanox’ Eitan Zahavi who will present an InfiniBand-centric vision towards exascale.

HPCwire: In general, do you see interconnect technologies becoming more diverse or less diverse in the future? Is it possible that there will eventually just be various flavors of Ethernet with different sets of capabilities to meet a range of market niches?

That may very well become true. Right now, we’re still in two-lanes in the commodity market with InfiniBand and Ethernet. But convergence towards Ethernet is often advertised. InfiniBand is still slightly ahead of the Ethernet development in bandwidth and latency but clearly behind in market share. So it may be a three-tier market with Ethernet at the low-mid end, InfiniBand at the mid-high end and specialized interconnects at the high-end.

HPCwire: The upcoming Hot Interconnects symposium will feature talks on a range of high-performance interconnects, both standard and proprietary. Could you give us a quick rundown on what you expect to be some of the more unique presentations there?

Our program is covering the whole spectrum of HPC interconnects. The technical  session on “High-Performance Interconnect Architectures” will have talks on the networks running on the world’s most powerful machines, System K and Tianhe-1A. We will also have interesting presentations on how commodity  interconnects, InfiniBand (Mellanox) and Ethernet (Gnodal), plan to scale  to very large configurations. We will hear about interesting features, such as collective communication offload and support for the PGAS programming model.

The keynotes on “The IBM Blue Gene/Q Interconnection Network and Message Unit”, “The Computer that is the Network: A Future History of Big Systems”, and “Will interconnect help or limit the future of computing?” will provide  a comprehensive vision of where HPC networking is going.

We will also feature two intriguing panels on data-center and cloud networking, with representatives from the big players: Cisco, HP, Mellanox, Intel, Arista, Avaya, IBM, Microsoft, and Facebook.  In addition, the conference will provide great tutorials on OpenFlow, Infiniband,  and large-scale data center and cloud  networking.

We have never had such dynamic developments to hear about and debate!

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Inspur Establishes Artificial Intelligence (AI) Department

Google Showcases 2017 AI Research Highlights

January 23, 2018

Looking for a good snapshot of the state of AI research? Cloud giant Google recently reviewed its 2017 AI research and application highlights in a two-part blog. While hardly comprehensive, it’s a worthwhile, fast read Read more…

By John Russell

UCSD, AIST Forge Tighter Alliance with AI-Focused MOU

January 18, 2018

The rich history of collaboration between UC San Diego and AIST in Japan is getting richer. The organizations entered into a five-year memorandum of understanding on January 10. The MOU represents the continuation of a 1 Read more…

By Tiffany Trader

New Blueprint for Converging HPC, Big Data

January 18, 2018

After five annual workshops on Big Data and Extreme-Scale Computing (BDEC), a group of international HPC heavyweights including Jack Dongarra (University of Tennessee), Satoshi Matsuoka (Tokyo Institute of Technology), Read more…

By John Russell

HPE Extreme Performance Solutions

HPE and NREL Take Steps to Create a Sustainable, Energy-Efficient Data Center with an H2 Fuel Cell

As enterprises attempt to manage rising volumes of data, unplanned data center outages are becoming more common and more expensive. As the cost of downtime rises, enterprises lose out on productivity and valuable competitive advantage without access to their critical data. Read more…

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown and Spectre security updates on the performance of popular H Read more…

By Tiffany Trader

UCSD, AIST Forge Tighter Alliance with AI-Focused MOU

January 18, 2018

The rich history of collaboration between UC San Diego and AIST in Japan is getting richer. The organizations entered into a five-year memorandum of understandi Read more…

By Tiffany Trader

New Blueprint for Converging HPC, Big Data

January 18, 2018

After five annual workshops on Big Data and Extreme-Scale Computing (BDEC), a group of international HPC heavyweights including Jack Dongarra (University of Te Read more…

By John Russell

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown Read more…

By Tiffany Trader

Fostering Lustre Advancement Through Development and Contributions

January 17, 2018

Six months after organizational changes at Intel's High Performance Data (HPDD) division, most in the Lustre community have shed any initial apprehension aroun Read more…

By Carlos Aoki Thomaz

When the Chips Are Down

January 11, 2018

In the last article, "The High Stakes Semiconductor Game that Drives HPC Diversity," I alluded to the challenges facing the semiconductor industry and how that may impact the evolution of HPC systems over the next few years. I thought I’d lift the covers a little and look at some of the commercial challenges that impact the component technology we use in HPC. Read more…

By Dairsie Latimer

How Meltdown and Spectre Patches Will Affect HPC Workloads

January 10, 2018

There have been claims that the fixes for the Meltdown and Spectre security vulnerabilities, named the KPTI (aka KAISER) patches, are going to affect applicatio Read more…

By Rosemary Francis

Momentum Builds for US Exascale

January 9, 2018

2018 looks to be a great year for the U.S. exascale program. The last several months of 2017 revealed a number of important developments that help put the U.S. Read more…

By Alex R. Larzelere

ANL’s Rick Stevens on CANDLE, ARM, Quantum, and More

January 8, 2018

Late last year HPCwire caught up with Rick Stevens, associate laboratory director for computing, environment and life Sciences at Argonne National Laboratory, f Read more…

By John Russell

Inventor Claims to Have Solved Floating Point Error Problem

January 17, 2018

"The decades-old floating point error problem has been solved," proclaims a press release from inventor Alan Jorgensen. The computer scientist has filed for and Read more…

By Tiffany Trader

US Coalesces Plans for First Exascale Supercomputer: Aurora in 2021

September 27, 2017

At the Advanced Scientific Computing Advisory Committee (ASCAC) meeting, in Arlington, Va., yesterday (Sept. 26), it was revealed that the "Aurora" supercompute Read more…

By Tiffany Trader

Japan Unveils Quantum Neural Network

November 22, 2017

The U.S. and China are leading the race toward productive quantum computing, but it's early enough that ultimate leadership is still something of an open questi Read more…

By Tiffany Trader

AMD Showcases Growing Portfolio of EPYC and Radeon-based Systems at SC17

November 13, 2017

AMD’s charge back into HPC and the datacenter is on full display at SC17. Having launched the EPYC processor line in June along with its MI25 GPU the focus he Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

IBM Begins Power9 Rollout with Backing from DOE, Google

December 6, 2017

After over a year of buildup, IBM is unveiling its first Power9 system based on the same architecture as the Department of Energy CORAL supercomputers, Summit a Read more…

By Tiffany Trader

Fast Forward: Five HPC Predictions for 2018

December 21, 2017

What’s on your list of high (and low) lights for 2017? Volta 100’s arrival on the heels of the P100? Appearance, albeit late in the year, of IBM’s Power9? Read more…

By John Russell

Chip Flaws ‘Meltdown’ and ‘Spectre’ Loom Large

January 4, 2018

The HPC and wider tech community have been abuzz this week over the discovery of critical design flaws that impact virtually all contemporary microprocessors. T Read more…

By Tiffany Trader

Leading Solution Providers

Perspective: What Really Happened at SC17?

November 22, 2017

SC is over. Now comes the myriad of follow-ups. Inboxes are filled with templated emails from vendors and other exhibitors hoping to win a place in the post-SC thinking of booth visitors. Attendees of tutorials, workshops and other technical sessions will be inundated with requests for feedback. Read more…

By Andrew Jones

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown Read more…

By Tiffany Trader

Tensors Come of Age: Why the AI Revolution Will Help HPC

November 13, 2017

Thirty years ago, parallel computing was coming of age. A bitter battle began between stalwart vector computing supporters and advocates of various approaches to parallel computing. IBM skeptic Alan Karp, reacting to announcements of nCUBE’s 1024-microprocessor system and Thinking Machines’ 65,536-element array, made a public $100 wager that no one could get a parallel speedup of over 200 on real HPC workloads. Read more…

By John Gustafson & Lenore Mullin

How Meltdown and Spectre Patches Will Affect HPC Workloads

January 10, 2018

There have been claims that the fixes for the Meltdown and Spectre security vulnerabilities, named the KPTI (aka KAISER) patches, are going to affect applicatio Read more…

By Rosemary Francis

Delays, Smoke, Records & Markets – A Candid Conversation with Cray CEO Peter Ungaro

October 5, 2017

Earlier this month, Tom Tabor, publisher of HPCwire and I had a very personal conversation with Cray CEO Peter Ungaro. Cray has been on something of a Cinderell Read more…

By Tiffany Trader & Tom Tabor

Flipping the Flops and Reading the Top500 Tea Leaves

November 13, 2017

The 50th edition of the Top500 list, the biannual publication of the world’s fastest supercomputers based on public Linpack benchmarking results, was released Read more…

By Tiffany Trader

GlobalFoundries, Ayar Labs Team Up to Commercialize Optical I/O

December 4, 2017

GlobalFoundries (GF) and Ayar Labs, a startup focused on using light, instead of electricity, to transfer data between chips, today announced they've entered in Read more…

By Tiffany Trader

HPC Chips – A Veritable Smorgasbord?

October 10, 2017

For the first time since AMD's ill-fated launch of Bulldozer the answer to the question, 'Which CPU will be in my next HPC system?' doesn't have to be 'Whichever variety of Intel Xeon E5 they are selling when we procure'. Read more…

By Dairsie Latimer

  • arrow
  • Click Here for More Headlines
  • arrow
Share This