Since 1986 - Covering the Fastest Computers
in the World and the People Who Run Them
Visit additional Tabor Communication Publications
Blog: From the Editor
From the Editor | Main Blog Index
February 03, 2010
Facebook Dreams of Terabit Ethernet
Facebook engineer Donn Lee probably feels a little bit like Amity Police Chief Martin Brody did in the first Jaws movie. After seeing the monster shark for the first time, Brody tells Captain Quint with a deadpan delivery: "You're gonna need a bigger boat."
Substitute social networking demand for the shark and the network for the boat, and you basically have Facebook's own horror story. The iconic social networking site is trying to cope with bandwidth requirements that double every year as it tries to support its 300 million (and growing!) user base.
In an article this week in Computerworld, Facebook's Lee explains the company's dilemma. According to him, their applications already require 100 Gigabit Ethernet and in the not-too-distant future will need 1 Terabit Ethernet. That means a single Facebook datacenter will need 64 Terabit pipes in the backbone. This would necessitate thousands of 10 GbE ports and more than 100 of the largest switches available -- not really a feasible solution. In a nutshell, bandwidth-hungry social networking is tied to an Ethernet anchor.
OK, so Facebook is not a traditional HPC app, but it sure acts like one. From the article:
Facebook is different from many enterprises in that it throws many servers at a single application rather than dividing up each server into multiple virtual machines. That means it faces a special challenge of knitting the many servers together. But its bandwidth challenge is rooted in fundamental advances in technology. All server motherboards come with Gigabit Ethernet built in, and today's multicore processors can easily fill those pipes.
Sound familiar? HPC apps can at least can take advantage of 40 Gbps InfiniBand for server-to-server communication today. But ultimately everything must go through an Ethernet pipe once you exit the LAN (ignoring the few InfiniBand-based WAN solutions).
In the Ethernet realm, 10 GbE is the fattest pipe available today and those products are just hitting the market en masse. The IEEE 802.3ba proposal, which specifies 100 GbE for the WAN backbone and 40 GbE for server-to-server communication, has been making its way through the IEEE standards process. 802.3ba is expected to be ratified later this year, and will presumably be followed by vendor offerings that support it.
Unfortunately, there doesn't seem to be any happy ending to this story. Given the sluggish speed at which the Ethernet industry moves, all of this seems like too little too late. Lee says the lack of bandwidth constrains innovation at Facebook, and ultimately the customer experience. For the time being, it looks like the disparity between the pace of social networking and Ethernet technology will continue to widen.
Posted by Michael Feldman - February 03, 2010 @ 6:12 PM, Pacific Standard Time
Discussion
There is 1 discussion item posted.
Google needs 10 terabit ethernet
Submitted by zipdisk2003
on Apr 5, 2010 @ 9:57 PM EDT
in fact, in this year's OF, people such s google even said that their data center needs to hav 10 terabit ethernet. In addition, this blog htp://www.terabit-ethernet.com is nice to be specificlly on terabit ethernet
Post #1
|
Join the Discussion |
Sponsored Links
Join Michael Wolfe as he compares the advantages and costs of using both low-level models and the directive-based OpenACC model for programming accelerated heterogeneous systems. Registration is free.
Accelerate your science with Seneca
One of the first HPC providers installing a 4X NVIDIA Kepler K-20 cluster. Invites you to a free evaluation on Seneca’s NVIDIA K20 Kepler cluster, pre-loaded with AMBER, NAMD, LAMMPS
High-Performance Computing in Action
Businesses that want to be on the cutting edge of their industries are increasingly turning to high-performance computing (HPC) solutions to handle complex compute processes and speed up their rate of innovation. Download this Executive Brief to see how businesses in energy, life sciences and entertainment put HPC solutions to work in their operations.
- The Accelerated Genomics Cloud $25,000 Giveaway
- Receive Higher Application Performance with end-to-end 56GbE
- ClusterStor™ 1500 – High Performance Efficiency
- Acer and Altair simplify cluster computing
- PGI 2013: OpenMP 1.5X faster than GCC, new GNU-compatible OpenACC C++ compiler, CUDA 5 and Tesla K20 support, and more.
- White Paper: Building a Data-Intensive Cluster Supercomputer
- Visualize Historical Usage for Optimized Returns on HPC Investments
- Memory Debugging Challenges in Parallel and Distributed Applications
RSS Feeds
Feeds by Topic
- Applications
- Developer Tools
- Interconnects
- Middleware
- Networks
- Processors
- Storage
- Systems
- Visualization
Feeds by Industry
Feeds by Content Type
- Features
- Blogs
- Short Takes
- Off the Wire
- Whitepapers
- Multimedia
- HPC Soundbite Podcasts
- HPCwire Job Bank
Subscribe to All Content
Michael Feldman
Michael Feldman is the editor of HPCwire.
Recent Comments
No Recent Blog Comments
Feature Articles
Exascale Advocates Stand on Nuclear Stockpiles
In quieter times, sounding the bell of funding big science with big systems tends to resonate further than when ears are already burning with sour economic and national security news. For exascale's future, however, the time could be ripe to instill some sense of urgency....
Read more...
NSF Forges Further Beyond FLOPs
In a recent solicitation, the NSF laid out needs for furthering its scientific and engineering infrastructure with new tools to go beyond top performance, Having already delivered systems like Stampede and Blue Waters, they're turning an eye to solving data-intensive challenges. We spoke with the agency's Irene Qualters and Barry Schneider about..
Read more...
CERN, Google Drive Future of Global Science Initiatives
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
Read more...
Short Takes
NASA Builds 'Climate in a Box'
May 23, 2013 |
The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
Read more...
Building Supercomputers with Raspberries
May 22, 2013 |
At some point in the not-too-distant future, building powerful, miniature computing systems will be considered a hobby for high schoolers, just as robotics or even Lego-building are today. That could be made possible through recent advancements made with the Raspberry Pi computers.
Read more...
Running Computational Fluid Dynamics in the Cloud
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
Read more...
Computing the Physics of Bubbles
May 15, 2013 |
Supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) have worked on important computational problems such as collapse of the atomic state, the optimization of chemical catalysts, and now modeling popping bubbles.
Read more...
Sponsored Whitepapers
Best Practices in Big Data Storage
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
Progress in Parallel: the Bull Parallel Programming Center
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
Sponsored Multimedia
SGI DMF ZeroWatt Disk Solution
In this demonstration of SGI DMF ZeroWatt disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.
Cray CS300-AC Cluster Supercomputer Air Cooling Technology Video
The Cray CS300-AC cluster supercomputer offers energy efficient, air-cooled design based on modular, industry-standard platforms featuring the latest processor and network technologies and a wide range of datacenter cooling requirements.
Blogs by Topics
Blogs by Author
- James Reinders
- Gary Johnson
- Andrew Jones
- Tiffany Trader
- Addison Snell
- Nicole Hemsoth
- Charlie Catlett
- Sue Korn
- Michael Feldman
- Chris Willard
- Isaac Lopez
- Tom Tabor
- Cathy Davidson
- William Lu
- Michael Wolfe
- Dan Gatti
- Caroline Connor
HPC Blogroll
- Computing Research Policy Blog
- Interconnects
- Reed's Ruminations
- Scalability.org
- StorageMojo
- Sun HPC Watercooler
Featured Events
- June 13, 2013
- June 14, 2013
Think Beyond Plastics Conference
Berkeley, CA
United States
- June 16, 2013
- June 20, 2013
ISC'13
Leipzig,
Germany
.gif)
- June 17, 2013
- June 18, 2013
Forecast 2013
San Francisco, CA
United States
- June 20, 2013
- June 21, 2013
CIO North America
Marina Del Rey, CA
United States
- June 24, 2013
- June 25, 2013
The Trading Show Chicago
Chicago, IL
United States
- June 25, 2013
- June 26, 2013
TERATEC 2013 Forum
Paris,
France
