Visit additional Tabor Communication Publications
December 07, 2011
Dec. 5 -- In a large, loud computer equipment room at NASA's Goddard Space Flight Center, amidst the humming of fans and trilling of transistors, is a gadget about the size of a small paperback. Network engineers call them "pluggables." These devices can pump data into a fiber optic line at rates up to 100 gigabits per second (100 Gbps).
That's "gigabit" as in "a billion bits." It is 10,000 times faster than a typical broadband cable modem connection, which operates at a mere 10 million bits per second, or 10 Mbps. 100 Gbps is fast enough to transfer a 25 GB Blu-ray (HD) movie over the Internet in 2 seconds flat.
A data superhighway as speedy as this one doesn't come cheap. The pluggable across the hall costs nearly as much as a luxury sports car. It converts electronic signals into pulses of laser light that travel down fiber optic wires and zip out onto the Internet at near-light speed.
A team of Goddard network engineers borrowed two of the super-fast 100 Gbps pluggables and a router in preparation for a major technology demonstration in Seattle at the Supercomputing 2011 (SC11) conference, November 12-18. The demo gave the high-performance computing world a glimpse of how the Internet will be used in the future to conduct research involving extraordinarily large transfers of data.
Today's leading edge is known as petascale computing. That's "peta" as in "petabyte," or 1,000 terabytes. One terabyte is 1,000 gigabytes, and a gigabyte is 1,000 megabytes, so a petabyte is a LOT of data. A petabyte of storage could hold 40,000 Blu-ray movies.
Now computer scientists are even talking about the coming era of exascale computing: "exa" as in one quintillion bytes, or one billion gigabytes.
Connected to the MAX
Quite a bit of network "plumbing" is required to move data at this scale. Various pieces of equipment and fiber-optic connections had to be put in place to create a 100 Gbps network connection from Goddard to Seattle.
The everyday Internet can't transfer data anywhere near 100 Gbps. The Goddard team had to make special arrangements with various entities on the Internet to create a big enough data pipeline. This required connecting multiple high-speed, fiber-optic research and education networks.
In Goddard's local neighborhood, a key partner in the demo was the Mid-Atlantic Crossroads — affectionately known in the network world as "the MAX," which is based in College Park, Md. The MAX provides Goddard access to Internet2 and other high-speed networks, such as ESnet, SCinet, and Starlight.
Specialized equipment, some of it not yet released on the open market, is also required to switch and route data at such blisteringly high speeds. NASA's vendor partners loaned leading-edge network and server technologies worth around $2 million for the SC11 demonstrations. These companies include Acadia Optronics, Alcatel-Lucent, Brocade, Ciena, cPacket, Force10, Fujitsu, Intel, Juniper, and Supermicro.
After much pre-meeting preparation, network engineers Bill Fink, Paul Lang, and Jeff Martz, members of Goddard's High End Computer Networking (HECN) team, flew out to Seattle to demonstrate technology that could be used to transfer exceedingly large sets of data across a 100 Gbps network. Crates of computer workstations and networking equipment were shipped to the site. The team used the equipment to build a local network at the convention center, connecting the booths of NASA and its various demo partners.
The demos consisted of transferring data between a computer workstation at Goddard and the NASA booth at SC11, and also among the booths of NASA and its partner organizations at SC11: Ciena, the SCinet Network Operations Center, and the University of Illinois at Chicago Laboratory for Advanced Computing/Northwestern University International Center for Advanced Internet Research.
One of the toughest tricks in high-performance networking is a so-called "disk to disk" (DTD) transfer of large data sets. In a DTD transfer, the speed limit is determined by how fast the data moves between two hard drives.
Hard drives are a notorious bottleneck in high-performance networks. In a typical personal computer, the fastest hard drives can transfer data at about 130 megabytes per second, or roughly 1,000 Mbps.
But the Goddard team has developed innovative ways to set up the hardware, software, and networking paths to allow the fastest possible data flow. One important dimension of this is using solid-state memory drives, which are scaled up versions of the small thumb drives people use to transport files between computing devices.
The computer workstation at Goddard used for disk-to-disk networking experiments contains some 40 Vertex3 120-GB solid-state hard drives. Each drive can spew data at a maximum rate of 550 MB per second — fast enough to download hundreds of iTunes songs per second. You would need more than 100 conventional spinning hard drives to match the performance of 40 solid-state drives.
Last year, at Supercomputing 2010, the team demonstrated disk-to-disk transfers up to 30 Gbps. This year they doubled it to 60 Gbps, transferring data between the NASA booth in Seattle and the workstation at Goddard. The top-speed disk-to-disk demo this year transferred data between two booths in Seattle at 72 Gbps (9 GB per second). That is equivalent to a Blu-ray movie download every 3 seconds.
The NASA networking research has its immediate applications in high-performance computing for understanding and predicting climate change, as well as other computer modeling tasks, carried out at the NASA Center for Climate Simulation and other organizations. Ultimately, networking technology enhances our ability to use computers to solve pressing social problems.
Source: NASA Goddard Space Flight Center
In quieter times, sounding the bell of funding big science with big systems tends to resonate further than when ears are already burning with sour economic and national security news. For exascale's future, however, the time could be ripe to instill some sense of urgency....
In a recent solicitation, the NSF laid out needs for furthering its scientific and engineering infrastructure with new tools to go beyond top performance, Having already delivered systems like Stampede and Blue Waters, they're turning an eye to solving data-intensive challenges. We spoke with the agency's Irene Qualters and Barry Schneider about..
Large-scale, worldwide scientific initiatives rely on some cloud-based system to both coordinate efforts and manage computational efforts at peak times that cannot be contained within the combined in-house HPC resources. Last week at Google I/O, Brookhaven National Lab’s Sergey Panitkin discussed the role of the Google Compute Engine in providing computational support to ATLAS, a detector of high-energy particles at the Large Hadron Collider (LHC).
May 23, 2013 |
The study of climate change is one of those scientific problems where it is almost essential to model the entire Earth to attain accurate results and make worthwhile predictions. In an attempt to make climate science more accessible to smaller research facilities, NASA introduced what they call ‘Climate in a Box,’ a system they note acts as a desktop supercomputer.
May 22, 2013 |
At some point in the not-too-distant future, building powerful, miniature computing systems will be considered a hobby for high schoolers, just as robotics or even Lego-building are today. That could be made possible through recent advancements made with the Raspberry Pi computers.
May 16, 2013 |
When it comes to cloud, long distances mean unacceptably high latencies. Researchers from the University of Bonn in Germany examined those latency issues of doing CFD modeling in the cloud by utilizing a common CFD and its utilization in HPC instance types including both CPU and GPU cores of Amazon EC2.
May 15, 2013 |
Supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center (NERSC) have worked on important computational problems such as collapse of the atomic state, the optimization of chemical catalysts, and now modeling popping bubbles.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
In this demonstration of SGI DMF ZeroWatt disk solution, Dr. Eng Lim Goh, SGI CTO, discusses a function of SGI DMF software to reduce costs and power consumption in an exascale (Big Data) storage datacenter.
The Cray CS300-AC cluster supercomputer offers energy efficient, air-cooled design based on modular, industry-standard platforms featuring the latest processor and network technologies and a wide range of datacenter cooling requirements.