Nov. 1, 2017 — Since 2010, the OpenStack platform has been a key player in the cloud computing boom, providing open source software for providing and using virtual compute resources. Used by large scientific projects at NASA and CERN, corporations such as Walmart and Sony, and popular web apps such as Box, eBay, and PayPal, OpenStack has helped weave cloud computing into our daily lives. Yet OpenStack remains a work in progress, with contributions from thousands of programmers around the world making improvements and creating new components.

While OpenStack underlies much of the cloud computing present, the Chameleon project is concerned with cloud computing’s future, offering a testbed for researchers to try out experimental architectures and applications. But a recent intersection between the two projects led to an exciting outcome, as a component called Blazar revived by Chameleon developers was voted to become an official project by the OpenStack Technical Committee. When Queens, the next OpenStack release, comes out in early 2018, Blazar will be added as a core component — an impressive accomplishment for the relatively small Chameleon team.

“It is very rare that a small academic project can influence a major open source project in a major way,” said Kate Keahey, principal investigator of Chameleon, computer scientist at Argonne National Laboratory, and Computation Institute Senior Fellow. “The results of our work will now reach many millions of people using OpenStack, in addition to the thousands we are reaching via Chameleon. It is impressive, especially as our contributions to OpenStack were only a side-effect of our work on Chameleon.”

Since its launch in 2014, Chameleon has run on the OpenStack infrastructure, but the software did not provide all the functionality the team needed “out of the box.” One such feature was the ability to schedule the use of cloud computing resources in advance, required by Chameleon researchers who want to test large-scale applications on the platform and need to reserve uninterrupted time on the system.

Pierre Riteau, Chameleon Lead DevOps Engineer, found a potential solution in an experimental OpenStack component previously developed for another project, XLCloud. Originally named Climate, the scheduler was used for “leasing” resources in the future, and also for cloud providers to manage their energy usage — by knowing in advance how many nodes would be in use, cloud computing providers could budget appropriately for the amount of power that would be used. At some point, its name changed to Blazar, and while it had been gradually built out by the OpenStack community, it wasn’t quite ready for a large-scale system like Chameleon.

Over the last two years, University of Chicago’s tiny team of developers not only implemented the testbed but also brought Blazar from a largely unused and inactive project to production. In true open source software fashion, the developers made patches and additions to the core Blazar project for their own use, but also shared them back with the community so that others could use the improved code.

“Blazar was a good candidate because we didn’t change the goals of the project, but we made it reliable enough to be used on a production research testbed,” Riteau said. “By fall 2016, we had accumulated bug fixes from using Blazar in Chameleon and we wanted to contribute them back.”

At the 2016 OpenStack summit in Barcelona, Riteau presented the work and joined forces with large telecommunications corporations NEC and NTT, who saw the reservation system as a helpful component for their Promise project. Telecom operators need advance reservations to plan ahead for maintenance work or high-traffic events such as the Super Bowl, Riteau said. A team at the University of Cambridge is also exploring use of Blazar for prototyping activities related to the Square Kilometre Array (SKA) project.

In September, the OpenStack Technical Committee approved Blazar as an official component, marking it for inclusion in next year’s release. That will lead to more use of the reservation service as well as more contributors improving the software, which will in turn help support the Chameleon project.

“It’s a return on the investment for us, having the component improved and reaping benefits without having to do all the work ourselves,” Keahey said. “It’s a big deal, because it will increase our impact from thousands of Chameleon users to the millions of people who use OpenStack.”

---

Source: Rob Mitchum, The Computation Institute