NSF Rolls Out Innovative Cloud Testbeds

By Tiffany Trader

August 21, 2014

The National Science Foundation (NSF) announced funding for two cloud testbeds, named “Chameleon” and “CloudLab.” A total award of $20 million to be split evenly between the projects will enable the academic research community to create and experiment with novel cloud architectures and potentially transformative applications.

Along with developing next-generation cloud systems, the programs emphasize the importance of forward-looking applications related to medical devices, power grids, and transportation systems.

The new projects are part of the NSF CISE Research Infrastructure: Mid-Scale Infrastructure – NSFCloud program, a follow-on to NSFNet, which supported and facilitated cutting-edge networking infrastructure.

Both “Chameleon” and “CloudLab” are underway and will ramp up over the next two years.

“Chameleon” will provide a configurable large-scale environment for cloud research. The project will initially leverage the existing FutureGrid hardware at the University of Chicago and the Texas Advanced Computing Center before transitioning over to newer hardware, also co-located at the University of Chicago and The University of Texas at Austin. Other partners include Ohio State University, Northwestern University, and the University of Texas at San Antonio.

The testbed will consist of 650 multicore cloud nodes, 5 petabytes of storage, with 100 Gbps connection between the sites. The architecture will combine homogenous hardware to support large-scale experiments with heterogenous systems that allow experimentation with high-memory, large-disk, low-power, GPU and coprocessor units. Researchers will be able to configure and test different cloud architectures on a range of problems, including machine learning and adaptive operating systems to climate simulations and flood prediction.

Chameleon was conceived as a creative endeavor. Researchers are encouraged to mix-and-match hardware, software and networking components and test their performance. Access to “bare-metal” hardware is key to enabling this flexibility. According to the University of Chicago’s Computation Institute, “this system will allow researchers to develop and test at scale new high-performance and low-noise virtualization solutions that might make possible high-performance computing in the cloud – creating virtual supercomputers on demand for research.”

“Like its namesake, the Chameleon testbed will be able to adapt itself to a wide range of experimental needs, from bare metal reconfiguration to support for ready made clouds,” said Kate Keahey, a scientist at the Computation Institute and principal investigator for Chameleon. “Furthermore, users will be able to run those experiments on a large scale, critical for big data and big compute research. But we also want to go beyond the facility and create a community where researchers will be able to discuss new ideas, share solutions that others can build on or contribute traces and workloads representative of real life cloud usage.”

The other project, CloudLab, is envisioned as a large-scale distributed infrastructure comprised of three clusters of 5,000 cores each, based at Clemson University, the University of Wisconsin and the University of Utah. Each site has a different technology focus to enable researchers to evaluate novel cloud technologies in a realistic environment.

The first cluster to be hosted at Clemson University in South Carolina should be up and running in the fall of 2014. Designed in partnership with Dell, the focus is on high-performance computing and high-memory configurations. The second cluster is scheduled to come online this winter at the University of Wisconsin in Madison. It’s being built in collaboration with Cisco, and will focus on SDN capabilities. The third cluster, expected to go live in the spring of 2015 at the University of Utah in Salt Lake City, will incorporate a hybrid approach with low-power ARM64 processors being used in tandem with x86 processors. Designed by HP, the emphasis of this site is energy efficient computing.

The three centers will be interconnected via 100 gigabit-per-second connections on Internet2’s advanced platform to form a unique scientific infrastructure. Researchers will have control and visibility all the way down to bare metal. They’ll have access to “slices” of the testbed, which they can use to build their own clouds. Other partners include UMass Amherst, Raytheon BBN Technologies and US Ignite.

“CloudLab will enable the next generation of innovations in the entire cloud ecosystem, ranging all the way from hardware elements, to software infrastructures, to the applications that run on the clouds,” observes UW-Madison computer science processor and CloudLab co-principal investigator Aditya Akella. “In turn, these innovations will lead to exciting new services that are simply impossible to realize on clouds today, benefitting our economy and society at large.”

Today’s NSF investment comes three years after the DOE-funded Magellan cloud testbed folded in late 2011. Although the $32 million project failed to show a significant ROI for “scientific cloud computing,” there are a raft of new technologies, configurations and applications that warrant exploration.

Public clouds, like those offered by Amazon, Google and Microsoft, run on privately-owned datacenters. They enable utility-style computing, but they don’t allow direct access to the infrastructure. By providing an all-access playground, Chameleon and CloudLab are filling an unmet need. In the words of University of Massachusetts Amherst researcher Michael Zink [speaking about CloudLab], “there’s nothing like this available today. This is something that will be very beneficial to many, many researchers.”

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Live and in Color, Meet the European Student Cluster Teams

November 21, 2017

The SC17 Student Cluster Competition welcomed two teams from Europe, the German team of FAU/TUC and Team Poland, the pride of Warsaw. Let's get to know them better through the miracle of video..... Team FAU/TUC is a c Read more…

By Dan Olds

SC17 Student Cluster Kick Off – Guts, Glory, Grep

November 21, 2017

The SC17 Student Cluster Competition started with a well-orchestrated kick-off emceed by Stephen Harrell, the competition chair. It began with a welcome from SC17 chair Bernd Mohr, where he lauded the competition for Read more…

By Dan Olds

Activist Investor Starboard Buys 10.7% Stake in Mellanox; Sale Possible?

November 20, 2017

Starboard Value has reportedly taken a 10.7 percent stake in interconnect specialist Mellanox Technologies, and according to the Wall Street Journal, has urged the company “to improve its margins and stock and explore Read more…

By John Russell

HPE Extreme Performance Solutions

Harness Scalable Petabyte Storage with HPE Apollo 4510 and HPE StoreEver

As a growing number of connected devices challenges IT departments to rapidly collect, manage, and store troves of data, organizations must adopt a new generation of IT to help them operate quickly and intelligently. Read more…

Installation of Sierra Supercomputer Steams Along at LLNL

November 20, 2017

Sierra, the 125 petaflops (peak) machine based on IBM’s Power9 chip being built at Lawrence Livermore National Laboratory, sometimes takes a back seat to Summit, the ~200 petaflops system being built at Oak Ridge Natio Read more…

By John Russell

Live and in Color, Meet the European Student Cluster Teams

November 21, 2017

The SC17 Student Cluster Competition welcomed two teams from Europe, the German team of FAU/TUC and Team Poland, the pride of Warsaw. Let's get to know them bet Read more…

By Dan Olds

SC17 Student Cluster Kick Off – Guts, Glory, Grep

November 21, 2017

The SC17 Student Cluster Competition started with a well-orchestrated kick-off emceed by Stephen Harrell, the competition chair. It began with a welcome from Read more…

By Dan Olds

SC Bids Farewell to Denver, Heads to Dallas for 30th

November 17, 2017

After a jam-packed four-day expo and intensive six-day technical program, SC17 has wrapped up another successful event that brought together nearly 13,000 visit Read more…

By Tiffany Trader

SC17 Keynote – HPC Powers SKA Efforts to Peer Deep into the Cosmos

November 17, 2017

This week’s SC17 keynote – Life, the Universe and Computing: The Story of the SKA Telescope – was a powerful pitch for the potential of Big Science projects that also showcased the foundational role of high performance computing in modern science. It was also visually stunning. Read more…

By John Russell

How Cities Use HPC at the Edge to Get Smarter

November 17, 2017

Cities are sensoring up, collecting vast troves of data that they’re running through predictive models and using the insights to solve problems that, in some Read more…

By Doug Black

Student Cluster LINPACK Record Shattered! More LINs Packed Than Ever before!

November 16, 2017

Nanyang Technological University, the pride of Singapore, utterly destroyed the Student Cluster Competition LINPACK record by posting a score of 51.77 TFlop/s a Read more…

By Dan Olds

Hyperion Market Update: ‘Decent’ Growth Led by HPE; AI Transparency a Risk Issue

November 15, 2017

The HPC market update from Hyperion Research (formerly IDC) at the annual SC conference is a business and social “must,” and this year’s presentation at S Read more…

By Doug Black

Nvidia Focuses Its Cloud Containers on HPC Applications

November 14, 2017

Having migrated its top-of-the-line datacenter GPU to the largest cloud vendors, Nvidia is touting its Volta architecture for a range of scientific computing ta Read more…

By George Leopold

US Coalesces Plans for First Exascale Supercomputer: Aurora in 2021

September 27, 2017

At the Advanced Scientific Computing Advisory Committee (ASCAC) meeting, in Arlington, Va., yesterday (Sept. 26), it was revealed that the "Aurora" supercompute Read more…

By Tiffany Trader

NERSC Scales Scientific Deep Learning to 15 Petaflops

August 28, 2017

A collaborative effort between Intel, NERSC and Stanford has delivered the first 15-petaflops deep learning software running on HPC platforms and is, according Read more…

By Rob Farber

Oracle Layoffs Reportedly Hit SPARC and Solaris Hard

September 7, 2017

Oracle’s latest layoffs have many wondering if this is the end of the line for the SPARC processor and Solaris OS development. As reported by multiple sources Read more…

By John Russell

AMD Showcases Growing Portfolio of EPYC and Radeon-based Systems at SC17

November 13, 2017

AMD’s charge back into HPC and the datacenter is on full display at SC17. Having launched the EPYC processor line in June along with its MI25 GPU the focus he Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Google Releases Deeplearn.js to Further Democratize Machine Learning

August 17, 2017

Spreading the use of machine learning tools is one of the goals of Google’s PAIR (People + AI Research) initiative, which was introduced in early July. Last w Read more…

By John Russell

GlobalFoundries Puts Wind in AMD’s Sails with 12nm FinFET

September 24, 2017

From its annual tech conference last week (Sept. 20), where GlobalFoundries welcomed more than 600 semiconductor professionals (reaching the Santa Clara venue Read more…

By Tiffany Trader

Amazon Debuts New AMD-based GPU Instances for Graphics Acceleration

September 12, 2017

Last week Amazon Web Services (AWS) streaming service, AppStream 2.0, introduced a new GPU instance called Graphics Design intended to accelerate graphics. The Read more…

By John Russell

Leading Solution Providers

EU Funds 20 Million Euro ARM+FPGA Exascale Project

September 7, 2017

At the Barcelona Supercomputer Centre on Wednesday (Sept. 6), 16 partners gathered to launch the EuroEXA project, which invests €20 million over three-and-a-half years into exascale-focused research and development. Led by the Horizon 2020 program, EuroEXA picks up the banner of a triad of partner projects — ExaNeSt, EcoScale and ExaNoDe — building on their work... Read more…

By Tiffany Trader

Delays, Smoke, Records & Markets – A Candid Conversation with Cray CEO Peter Ungaro

October 5, 2017

Earlier this month, Tom Tabor, publisher of HPCwire and I had a very personal conversation with Cray CEO Peter Ungaro. Cray has been on something of a Cinderell Read more…

By Tiffany Trader & Tom Tabor

Cray Moves to Acquire the Seagate ClusterStor Line

July 28, 2017

This week Cray announced that it is picking up Seagate's ClusterStor HPC storage array business for an undisclosed sum. "In short we're effectively transitioning the bulk of the ClusterStor product line to Cray," said CEO Peter Ungaro. Read more…

By Tiffany Trader

Reinders: “AVX-512 May Be a Hidden Gem” in Intel Xeon Scalable Processors

June 29, 2017

Imagine if we could use vector processing on something other than just floating point problems.  Today, GPUs and CPUs work tirelessly to accelerate algorithms Read more…

By James Reinders

Intel Launches Software Tools to Ease FPGA Programming

September 5, 2017

Field Programmable Gate Arrays (FPGAs) have a reputation for being difficult to program, requiring expertise in specialty languages, like Verilog or VHDL. Easin Read more…

By Tiffany Trader

HPC Chips – A Veritable Smorgasbord?

October 10, 2017

For the first time since AMD's ill-fated launch of Bulldozer the answer to the question, 'Which CPU will be in my next HPC system?' doesn't have to be 'Whichever variety of Intel Xeon E5 they are selling when we procure'. Read more…

By Dairsie Latimer

Flipping the Flops and Reading the Top500 Tea Leaves

November 13, 2017

The 50th edition of the Top500 list, the biannual publication of the world’s fastest supercomputers based on public Linpack benchmarking results, was released Read more…

By Tiffany Trader

IBM Advances Web-based Quantum Programming

September 5, 2017

IBM Research is pairing its Jupyter-based Data Science Experience notebook environment with its cloud-based quantum computer, IBM Q, in hopes of encouraging a new class of entrepreneurial user to solve intractable problems that even exceed the capabilities of the best AI systems. Read more…

By Alex Woodie

Share This