Visit additional Tabor Communication Publications
October 14, 2008
Woven Systems has expanded its Ethernet product lineup with a new 10 Gigabit Ethernet (GbE) top-of-rack switch. The 24-port TRX 200 offers 10 gigabits per second (Gbps) wirespeed performance on each port and InfiniBand-like latencies. The TRX 200 joins Woven's other two offerings, the 48-port Gigabit Ethernet TRX 100 top-of-rack switch and the 144-port 10 GbE "Fabric Switch" for the network core.
Like its Woven brethren, the TRX 200 is aimed at HPC and Web services -- two markets where bandwidth and latency requirements exceed that of the standard enterprise setup. Leading-edge interconnect performance has been the norm in HPC environments for some time. But with the advent of the Web services industry, a whole new market is developing for high bandwidth, low latency infrastructure. In this area, search engines or any application that performs Web page indexing must often operate with soft real-time constraints, so node-to-node latencies must be kept to a minimum.
Bandwidth can always be overprovisioned with extra switches, but that doesn't help the latency picture. Woven has specifically designed its products for InfiniBand-like latencies. Instead of store-and-forward switching used in standard Ethernet gear, Woven employs cut-through switching. The company claims latencies of 1.6µs for its flagship EFX 1000 switch.
Woven's big story with the new TRX 200 top-of-rack switch is its pricing. At less than $500 per wirespeed 10 Gbps port ($11,995 for a single unit), Woven is pushing back against Arastra, its closest competitor in high-performance 10 GbE switching. When Arastra launched its line of Ethernet gear last year, it quoted $400 per port. But it's not clear if that pricing applies across its entire product line. The new TRX against Arastra's 24-port 7124S would be the real apples-to-apples comparison, since both products claim to offer bi-directional wirespeed performance (480 Gbps aggregate) plus low latency.
The closest Cisco gear is probably the 4900M, which is a top-of-rack switch for users transitioning from GbE to 10 GbE. But at a maximum aggregate throughput of just 320 Gbps, and latencies in the 2.6µs range (according to eWeek testing), the Cisco switch is really not in the same performance ballpark as the Woven and Arastra offerings. Also, with a price that starts at $22,000, the 4900M is at least twice as expensive as its upstart competition.*
The roadblocks remaining for the Woven offerings, and for 10 GbE switches in general, are price (compared to standard Gigabit Ethernet) and performance (compared to InfiniBand). But if you are an Ethernet vendor, time may be on your side.
Many in the industry are predicting that by 2010 10 GbE will move onto the server motherboard en masse, reducing the cost of connection from about $300 or $400 down to around $22 dollars. (The real cost to the buyer is even a bit less than that since motherboard manufacturers will be replacing the older GbE interfaces.) In that same year, the total cost of a 10 GbE connection will be just twice that of a GbE connection. In 2002, the 2x cost differential proved to be an inflection point for the transition from Fast Ethernet to GbE. "That will usher in a much bigger ramp for 10 GigE servers and thus the beginning of a large transformation of the datacenter," predicts Woven VP of marketing Joe Ammirato.
If history does repeat itself, one of the first places we're likely to see the GbE to 10 GbE transition is on the TOP500 list. Even today, 57 percent of the top "supercomputers" are based on GbE. It must be said, though, that in most of these cases, the interconnect is not the bottleneck for system performance, or if it is, it's a tolerable one. For loosely-coupled, embarrassingly-parallel applications, node-to-node communications are only needed intermittently, so larger latencies and lower bandwidth are not as much of an issue.
For more tightly-coupled HPC applications, DDR InfiniBand is now the interconnect of choice. When 10 GbE goes mainstream, the choice becomes more difficult. Joe Ammirato says both performance and cost are catching up to InfiniBand, even without the benefit of native 10 GbE on the motherboard. When that happens, the interconnect interface becomes essentially free for Ethernet fabrics compared to InfiniBand, which will still require a $300 adapter.
DDR and QDR InfiniBand will still have the raw performance advantage, offering perhaps a half or a third the latency of the best Ethernet solutions and more than twice the bandwidth (QDR is 40 Gbps, but because the on-board PCIe interface limits how fast data can be moved, only about 25 Gbps is realized). Masum Mir, Woven's senior product manager, admits that InfiniBand will remain viable, but the presence of affordable 10 GbE solutions will compete at the high end. Especially with larger clusters and more variable traffic data traffic patterns, Mir sees Ethernet solutions like theirs -- with dynamic congestion avoidance and lossless fabric support -- as the more flexible choice.
Certainly for end users looking for a longer ROI horizon, Ethernet will look less risky. The battle cry of all Ethernet vendors continues to be that Ethernet will prevail. This may be less true for HPC users, who have come to view InfiniBand as a more mainstream technology with each passing year. And with much of the discussion about 10 GbE still in the future tense, companies like Woven Systems will be required to push the technology uphill for the next couple of years.
*Update: A more accurate comparison may be with Cisco's new Nexus 5020, a 40-port 10 GbE switch that offers wirespeed performance and a switch latency of 3.2µs. The 5020 can be expanded to up to 52 10 GbE ports to yield an aggregate throughput of 1 Tbps. At around $900 per port it's twice as expensive as the Woven or Arastra gear, but the Cisco box also comes with support for Fibre Channel over Ethernet and Cisco Data Center Ethernet.
Jun 19, 2013 |
Supercomputer architectures have evolved considerably over the last 20 years, particularly in the number of processors that are linked together. One aspect of HPC architecture that hasn't changed is the MPI programming model.
Jun 18, 2013 |
The world's largest supercomputers, like Tianhe-2, are great at traditional, compute-intensive HPC workloads, such as simulating atomic decay or modeling tornados. But data-intensive applications--such as mining big data sets for connections--is a different sort of workload, and runs best on a different sort of computer.
Jun 18, 2013 |
Researchers are finding innovative uses for Gordon, the 285 teraflop supercomputer housed at the San Diego Supercomputer Center (SDSC) that has a unique Flash-based storage system. Since going online, researchers have put the incredibly fast I/O to use on a wide variety of workloads, ranging from chemistry to political science.
Jun 17, 2013 |
The advent of low-power mobile processors and cloud delivery models is changing the economics of computing. But just as an economy car is good at different things than a full size truck, an HPC workload still has certain computing demands that neither the fastest smartphone nor the most elastic cloud cluster can fulfill.
Jun 14, 2013 |
For all the progress we've made in IT over the last 50 years, there's one area of life that has steadfastly eluded the grasp of computers: understanding human language. Now, researchers at the Texas Advanced Computing Center (TACC) are utilizing a Hadoop cluster on its Longhorn supercomputer to move the state of the art of language processing a little bit further.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?
Join our webinar to learn how IT managers can migrate to a more resilient, flexible and scalable solution that grows with the data center. Mellanox VMS is future-proof, efficient and brings significant CAPEX and OPEX savings. The VMS is available today.