The National Science Foundation (NSF) announced funding for two cloud testbeds, named “Chameleon” and “CloudLab.” A total award of $20 million to be split evenly between the projects will enable the academic research community to create and experiment with novel cloud architectures and potentially transformative applications.

Along with developing next-generation cloud systems, the programs emphasize the importance of forward-looking applications related to medical devices, power grids, and transportation systems.

The new projects are part of the NSF CISE Research Infrastructure: Mid-Scale Infrastructure – NSFCloud program, a follow-on to NSFNet, which supported and facilitated cutting-edge networking infrastructure.

Both “Chameleon” and “CloudLab” are underway and will ramp up over the next two years.

“Chameleon” will provide a configurable large-scale environment for cloud research. The project will initially leverage the existing FutureGrid hardware at the University of Chicago and the Texas Advanced Computing Center before transitioning over to newer hardware, also co-located at the University of Chicago and The University of Texas at Austin. Other partners include Ohio State University, Northwestern University, and the University of Texas at San Antonio.

The testbed will consist of 650 multicore cloud nodes, 5 petabytes of storage, with 100 Gbps connection between the sites. The architecture will combine homogenous hardware to support large-scale experiments with heterogenous systems that allow experimentation with high-memory, large-disk, low-power, GPU and coprocessor units. Researchers will be able to configure and test different cloud architectures on a range of problems, including machine learning and adaptive operating systems to climate simulations and flood prediction.

Chameleon was conceived as a creative endeavor. Researchers are encouraged to mix-and-match hardware, software and networking components and test their performance. Access to “bare-metal” hardware is key to enabling this flexibility. According to the University of Chicago’s Computation Institute, “this system will allow researchers to develop and test at scale new high-performance and low-noise virtualization solutions that might make possible high-performance computing in the cloud – creating virtual supercomputers on demand for research.”

“Like its namesake, the Chameleon testbed will be able to adapt itself to a wide range of experimental needs, from bare metal reconfiguration to support for ready made clouds,” said Kate Keahey, a scientist at the Computation Institute and principal investigator for Chameleon. “Furthermore, users will be able to run those experiments on a large scale, critical for big data and big compute research. But we also want to go beyond the facility and create a community where researchers will be able to discuss new ideas, share solutions that others can build on or contribute traces and workloads representative of real life cloud usage.”

The other project, CloudLab, is envisioned as a large-scale distributed infrastructure comprised of three clusters of 5,000 cores each, based at Clemson University, the University of Wisconsin and the University of Utah. Each site has a different technology focus to enable researchers to evaluate novel cloud technologies in a realistic environment.

The first cluster to be hosted at Clemson University in South Carolina should be up and running in the fall of 2014. Designed in partnership with Dell, the focus is on high-performance computing and high-memory configurations. The second cluster is scheduled to come online this winter at the University of Wisconsin in Madison. It’s being built in collaboration with Cisco, and will focus on SDN capabilities. The third cluster, expected to go live in the spring of 2015 at the University of Utah in Salt Lake City, will incorporate a hybrid approach with low-power ARM64 processors being used in tandem with x86 processors. Designed by HP, the emphasis of this site is energy efficient computing.

The three centers will be interconnected via 100 gigabit-per-second connections on Internet2’s advanced platform to form a unique scientific infrastructure. Researchers will have control and visibility all the way down to bare metal. They’ll have access to “slices” of the testbed, which they can use to build their own clouds. Other partners include UMass Amherst, Raytheon BBN Technologies and US Ignite.

“CloudLab will enable the next generation of innovations in the entire cloud ecosystem, ranging all the way from hardware elements, to software infrastructures, to the applications that run on the clouds,” observes UW-Madison computer science processor and CloudLab co-principal investigator Aditya Akella. “In turn, these innovations will lead to exciting new services that are simply impossible to realize on clouds today, benefitting our economy and society at large.”

Today’s NSF investment comes three years after the DOE-funded Magellan cloud testbed folded in late 2011. Although the $32 million project failed to show a significant ROI for “scientific cloud computing,” there are a raft of new technologies, configurations and applications that warrant exploration.

Public clouds, like those offered by Amazon, Google and Microsoft, run on privately-owned datacenters. They enable utility-style computing, but they don’t allow direct access to the infrastructure. By providing an all-access playground, Chameleon and CloudLab are filling an unmet need. In the words of University of Massachusetts Amherst researcher Michael Zink [speaking about CloudLab], “there’s nothing like this available today. This is something that will be very beneficial to many, many researchers.”