The National Institute of Advanced Industrial Science and Technology (AIST) has developed a technology to enable users to spin up a virtual HPC cluster on top of any cloud-based infrastructure.
The impetus for the project was two-fold:
1. To allow a personalized high-performance computer to be created on-demand
2. To build a virutal cluster once and be able to run it anywhere
In high-performance computing, clustering tools connect many computers so they can run as a single computer, yet often the hardware configuration is not uniform. Virtual cloud-based computers that are not dependent on hardware configuration can also be bundled together, with the result being a virtual cluster. Yet as the AIST announcement explains, when the environment has to be re-created in another cloud, software must be reinstalled and settings reconfigured, which adds to the time and labor cost.
Using the “Build Once, Run Everywhere” concept, once the environment to run the application has been established it may be run on any cloud, private or public. Moreover, if computing power in one cloud is insufficient, a larger virtual cluster can be constructed on another cloud to fulfill computing requirements.
For this experiment, AIST verified that the technology did indeed operate on both its private cloud, AIST Super Green Cloud (ASGC), as well as the Amazon EC2 infrastructure.
“With this technology, users and application fields that could not use high-performance computing previously can now use high-performance computing,” notes AIST. “Thus, the developed technology is expected to contribute to the enhancement of industrial competitiveness.”
Also at issue when using virtualization or virtualization-based clouds for HPC workloads is the drop in performance caused by additional overhead. While cognizant that this hit is somewhat application-dependent, AIST researchers in one example, achieved 6.77 teraflops (LINPACK) for a non-virtualized cluster computer versus 6.40 TFLOPS for a virtualized setup, about a 5 percent performance decline, which they found reasonable.
The next step for AIST is to deploy this technology as an operational service on ASGC.
AIST is a publicly-funded research institution with headquarters in Tsukuba and Tokyo and satellite research centers located all over Japan. The group includes about 2,300 researchers as well as visiting scientists, post-doctoral fellows, and students from Japan and overseas.