Visit additional Tabor Communication Publications
December 07, 2011
Dec. 5 -- In a large, loud computer equipment room at NASA's Goddard Space Flight Center, amidst the humming of fans and trilling of transistors, is a gadget about the size of a small paperback. Network engineers call them "pluggables." These devices can pump data into a fiber optic line at rates up to 100 gigabits per second (100 Gbps).
That's "gigabit" as in "a billion bits." It is 10,000 times faster than a typical broadband cable modem connection, which operates at a mere 10 million bits per second, or 10 Mbps. 100 Gbps is fast enough to transfer a 25 GB Blu-ray (HD) movie over the Internet in 2 seconds flat.
A data superhighway as speedy as this one doesn't come cheap. The pluggable across the hall costs nearly as much as a luxury sports car. It converts electronic signals into pulses of laser light that travel down fiber optic wires and zip out onto the Internet at near-light speed.
A team of Goddard network engineers borrowed two of the super-fast 100 Gbps pluggables and a router in preparation for a major technology demonstration in Seattle at the Supercomputing 2011 (SC11) conference, November 12-18. The demo gave the high-performance computing world a glimpse of how the Internet will be used in the future to conduct research involving extraordinarily large transfers of data.
Today's leading edge is known as petascale computing. That's "peta" as in "petabyte," or 1,000 terabytes. One terabyte is 1,000 gigabytes, and a gigabyte is 1,000 megabytes, so a petabyte is a LOT of data. A petabyte of storage could hold 40,000 Blu-ray movies.
Now computer scientists are even talking about the coming era of exascale computing: "exa" as in one quintillion bytes, or one billion gigabytes.
Connected to the MAX
Quite a bit of network "plumbing" is required to move data at this scale. Various pieces of equipment and fiber-optic connections had to be put in place to create a 100 Gbps network connection from Goddard to Seattle.
The everyday Internet can't transfer data anywhere near 100 Gbps. The Goddard team had to make special arrangements with various entities on the Internet to create a big enough data pipeline. This required connecting multiple high-speed, fiber-optic research and education networks.
In Goddard's local neighborhood, a key partner in the demo was the Mid-Atlantic Crossroads — affectionately known in the network world as "the MAX," which is based in College Park, Md. The MAX provides Goddard access to Internet2 and other high-speed networks, such as ESnet, SCinet, and Starlight.
Specialized equipment, some of it not yet released on the open market, is also required to switch and route data at such blisteringly high speeds. NASA's vendor partners loaned leading-edge network and server technologies worth around $2 million for the SC11 demonstrations. These companies include Acadia Optronics, Alcatel-Lucent, Brocade, Ciena, cPacket, Force10, Fujitsu, Intel, Juniper, and Supermicro.
After much pre-meeting preparation, network engineers Bill Fink, Paul Lang, and Jeff Martz, members of Goddard's High End Computer Networking (HECN) team, flew out to Seattle to demonstrate technology that could be used to transfer exceedingly large sets of data across a 100 Gbps network. Crates of computer workstations and networking equipment were shipped to the site. The team used the equipment to build a local network at the convention center, connecting the booths of NASA and its various demo partners.
The demos consisted of transferring data between a computer workstation at Goddard and the NASA booth at SC11, and also among the booths of NASA and its partner organizations at SC11: Ciena, the SCinet Network Operations Center, and the University of Illinois at Chicago Laboratory for Advanced Computing/Northwestern University International Center for Advanced Internet Research.
One of the toughest tricks in high-performance networking is a so-called "disk to disk" (DTD) transfer of large data sets. In a DTD transfer, the speed limit is determined by how fast the data moves between two hard drives.
Hard drives are a notorious bottleneck in high-performance networks. In a typical personal computer, the fastest hard drives can transfer data at about 130 megabytes per second, or roughly 1,000 Mbps.
But the Goddard team has developed innovative ways to set up the hardware, software, and networking paths to allow the fastest possible data flow. One important dimension of this is using solid-state memory drives, which are scaled up versions of the small thumb drives people use to transport files between computing devices.
The computer workstation at Goddard used for disk-to-disk networking experiments contains some 40 Vertex3 120-GB solid-state hard drives. Each drive can spew data at a maximum rate of 550 MB per second — fast enough to download hundreds of iTunes songs per second. You would need more than 100 conventional spinning hard drives to match the performance of 40 solid-state drives.
Last year, at Supercomputing 2010, the team demonstrated disk-to-disk transfers up to 30 Gbps. This year they doubled it to 60 Gbps, transferring data between the NASA booth in Seattle and the workstation at Goddard. The top-speed disk-to-disk demo this year transferred data between two booths in Seattle at 72 Gbps (9 GB per second). That is equivalent to a Blu-ray movie download every 3 seconds.
The NASA networking research has its immediate applications in high-performance computing for understanding and predicting climate change, as well as other computer modeling tasks, carried out at the NASA Center for Climate Simulation and other organizations. Ultimately, networking technology enhances our ability to use computers to solve pressing social problems.
Source: NASA Goddard Space Flight Center
Contributing commentator, Andrew Jones, offers a break in the news cycle with an assessment of what the national "size matters" contest means for the U.S. and other nations...
Today at the International Supercomputing Conference in Leipzing, Germany, Jack Dongarra presented on a proposed benchmark that could carry a bit more weight than its older Linpack companion. The high performance conjugate gradient (HPCG) concept takes into account new architectures for new applications, while shedding the floating point....
Not content to let the Tianhe-2 announcement ride alone, Intel rolled out a series of announcements around its Knights Corner and Xeon Phi products--all of which are aimed at adding some options and variety for a wider base of potential users across the HPC spectrum. Today at the International Supercomputing Conference, the company's Raj....
Jun 18, 2013 |
The world's largest supercomputers, like Tianhe-2, are great at traditional, compute-intensive HPC workloads, such as simulating atomic decay or modeling tornados. But data-intensive applications--such as mining big data sets for connections--is a different sort of workload, and runs best on a different sort of computer.
Jun 18, 2013 |
Researchers are finding innovative uses for Gordon, the 285 teraflop supercomputer housed at the San Diego Supercomputer Center (SDSC) that has a unique Flash-based storage system. Since going online, researchers have put the incredibly fast I/O to use on a wide variety of workloads, ranging from chemistry to political science.
Jun 17, 2013 |
The advent of low-power mobile processors and cloud delivery models is changing the economics of computing. But just as an economy car is good at different things than a full size truck, an HPC workload still has certain computing demands that neither the fastest smartphone nor the most elastic cloud cluster can fulfill.
Jun 14, 2013 |
For all the progress we've made in IT over the last 50 years, there's one area of life that has steadfastly eluded the grasp of computers: understanding human language. Now, researchers at the Texas Advanced Computing Center (TACC) are utilizing a Hadoop cluster on its Longhorn supercomputer to move the state of the art of language processing a little bit further.
Jun 13, 2013 |
Titan, the Cray XK7 at the Oak Ridge National Lab that debuted last fall as the fastest supercomputer in the world with 17.59 petaflops of sustained computing power, will rely on its previous LINPACK test for the upcoming edition of the Top 500 list.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?
Join our webinar to learn how IT managers can migrate to a more resilient, flexible and scalable solution that grows with the data center. Mellanox VMS is future-proof, efficient and brings significant CAPEX and OPEX savings. The VMS is available today.