Make way, Bright Computing is coming to the cloud. On Tuesday, the company announced that the upcoming version of its cluster management software, Bright Cluster Manager 6.0, will include support for cloud bursting through Amazon’s Elastic Compute Cloud (Amazon EC2). HPC in the Cloud spoke with Bright’s CEO Matthijs van Leeuwen to get the details.

The new solution supports two scenarios: one is for organizations that do not have their own on-site cluster and the other is for organizations that do. In the first case, the user needs a cluster but because of resource constraints, chooses to run their computations in the cloud. With the new solution, the customer can create a complete cluster in the cloud with access provided through the Bright GUI, from which the user can submit jobs. Bright Cluster Manager provides turn-key support to HPC workloads. It includes MPI libraries, management, and monitoring. All of the cluster goodness is now inside the Amazon cloud. Bright Computing believes it has a unique offering in that everything the customer needs is built-in. Other solutions, like Rocks, offer similar capability, but the cloud-bursting functionality requires the support of a third-party, such as RightScale.

In the second scenario, which van Leeuwen characterized as the more interesting of the two, the customer has an in-house cluster, but needs more capacity, instantly, on-demand. Using the new solution, the customer can add nodes, which although running inside Amazon EC2, appear, from user’s perspective, to be part of the same local cluster. The user can choose to submit jobs to the local queue, to the cloud queue, or to the mixed local and cloud queue. Of course, workloads selected for cloud computation must be easily parallelized, that is able to scale out, in a distributed fashion. Good examples include Monte Carlo or parameter studies, as well as protein docking to the same molecules (as in drug discovery), where you repeat the same process thousands and thousands of times, with little interaction between the individual computations.

Bright Computing is seeing the most interest for cloud bursting in the pharmaceutical industry, where a lot of drug companies are already making use of it, and also in oil and gas, as well as manufacturing. In fact, the CEO was enthusiastic in pointing out the solution’s suitability for the “missing middle” initiative. The US is seeing a big push to revive the US manufacturing industry, especially the small-to-medium-sized manufacturers who can use HPC to increase competitiveness. Cloud bursting is important for this segment because it lowers the barrier to entry for HPC by eliminating the upfront expense of an in-house cluster.

“The customer doesn’t need much Amazon or cloud expertise,” van Leeuwen stated. “Bright Computing Cluster does everything for him. It ensures that the nodes are running with the same software configuration, same versions of libraries and Linux kernels, MPI libraries, mathematical libraries, and the same configurations in terms of network settings, kernel parameters, and on the local cluster.”

Bright Cluster Manager 6.0 screen shot

As far as “burstability” goes, Bright Computing does face some competition, most notably from Platform ISF and Adaptive’s Moab, and then there’s the open-source Rocks cluster software, all of which are compatible with Cluster Compute Instances in Amazon EC2. Bright’s message, though, is that their solution is easier to use, with everything the customer needs in one interface.

In the words of the CEO: “All the customer needs to do is provide his Amazon account details, and then Bright Cluster Manager will do everything else.”

Martijn de Vries, CTO of Bright Computing, remarked in the official news release that “the integrated approach…is unique to Bright [and] eliminates multiple failure points typical of cluster management built with toolkits.”

The company has been working on the new capabilities since early summer. The project evolved in response to a growing cloud market, a reality that was confirmed by increased cloud interest from customers and prospects. Indeed, the company considers this a first step to lots more private cloud capability for Bright Cluster Manager. However, van Leeuwen pointed out that Bright Computing has been planning for this day since it started back in 2004 – not for cloud specifically, but for grid, cloud’s predecessor. The original Bright Cluster Manager was developed from scratch with plans to eventually scale into thousands of nodes.

“The next step with private cloud is to make sure we can handle virtual machines as easily as we can handle physical machines, but we’re confident we can do that quickly because we’ve been using virtual machines for internal use for a long time to simulate clusters,” the CEO stated.

To that end, the company hopes to announce more private cloud capabilities in 2012, but in terms of the HPC cloud space, they want you to know that the cloud bursting capability is a very big step. They believe that cloud-bursting is more relevant to HPC in the cloud than the private cloud because the latter is going to be oriented toward general enterprise use.

The company is planning to release 6.0 by the end of the year for early adopters, with general availability scheduled for January 2012. Bright Computing and Amazon will be demonstrating the new capabilities at SC11 in Seattle next week.