Visit additional Tabor Communication Publications
February 03, 2010
Facebook engineer Donn Lee probably feels a little bit like Amity Police Chief Martin Brody did in the first Jaws movie. After seeing the monster shark for the first time, Brody tells Captain Quint with a deadpan delivery: "You're gonna need a bigger boat."
Substitute social networking demand for the shark and the network for the boat, and you basically have Facebook's own horror story. The iconic social networking site is trying to cope with bandwidth requirements that double every year as it tries to support its 300 million (and growing!) user base.
In an article this week in Computerworld, Facebook's Lee explains the company's dilemma. According to him, their applications already require 100 Gigabit Ethernet and in the not-too-distant future will need 1 Terabit Ethernet. That means a single Facebook datacenter will need 64 Terabit pipes in the backbone. This would necessitate thousands of 10 GbE ports and more than 100 of the largest switches available -- not really a feasible solution. In a nutshell, bandwidth-hungry social networking is tied to an Ethernet anchor.
OK, so Facebook is not a traditional HPC app, but it sure acts like one. From the article:
Facebook is different from many enterprises in that it throws many servers at a single application rather than dividing up each server into multiple virtual machines. That means it faces a special challenge of knitting the many servers together. But its bandwidth challenge is rooted in fundamental advances in technology. All server motherboards come with Gigabit Ethernet built in, and today's multicore processors can easily fill those pipes.
Sound familiar? HPC apps can at least can take advantage of 40 Gbps InfiniBand for server-to-server communication today. But ultimately everything must go through an Ethernet pipe once you exit the LAN (ignoring the few InfiniBand-based WAN solutions).
In the Ethernet realm, 10 GbE is the fattest pipe available today and those products are just hitting the market en masse. The IEEE 802.3ba proposal, which specifies 100 GbE for the WAN backbone and 40 GbE for server-to-server communication, has been making its way through the IEEE standards process. 802.3ba is expected to be ratified later this year, and will presumably be followed by vendor offerings that support it.
Unfortunately, there doesn't seem to be any happy ending to this story. Given the sluggish speed at which the Ethernet industry moves, all of this seems like too little too late. Lee says the lack of bandwidth constrains innovation at Facebook, and ultimately the customer experience. For the time being, it looks like the disparity between the pace of social networking and Ethernet technology will continue to widen.
Posted by Michael Feldman - February 03, 2010 @ 6:12 PM, Pacific Standard Time
Michael Feldman is the editor of HPCwire.
No Recent Blog Comments
In a recent solicitation, the NSF laid out needs for furthering its scientific and engineering infrastructure with new tools to go beyond top performance, Having already delivered systems like Stampede and Blue Waters, they're turning an eye to solving data-intensive challenges. We spoke with the agency's Irene Qualters and Barry Schneider about..
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
The Xeon Phi coprocessor might be the new kid on the high performance block, but out of all first-rate kickers of the Intel tires, the Texas Advanced Computing Center (TACC) got the first real jab with its new top ten Stampede system.We talk with the center's Karl Schultz about the challenges of programming for Phi--but more specifically, the optimization...
May 22, 2013 |
At some point in the not-too-distant future, building powerful, miniature computing systems will be considered a hobby for high schoolers, just as robotics or even Lego-building are today. That could be made possible through recent advancements made with the Raspberry Pi computers.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
May 15, 2013 |
Supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) have worked on important computational problems such as collapse of the atomic state, the optimization of chemical catalysts, and now modeling popping bubbles.
May 10, 2013 |
Program provides cash awards up to $10,000 for the best open-source end-user applications deployed on 100G network.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
In this demonstration of SGI DMF ZeroWatt disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.
The Cray CS300-AC cluster supercomputer offers energy efficient, air-cooled design based on modular, industry-standard platforms featuring the latest processor and network technologies and a wide range of datacenter cooling requirements.