⟦Attend the IBM HPC, AI and LSF User Group Meeting at SC19 in Denver on November 19.⟧

At any moment, the atmosphere of the earth contains roughly 37.5 million billion gallons of water.¹ For practical purposes, this is an unlimited supply.

The problem has always been in accessing all this valuable resource when needed. One possible solution – cloud seeding – was proposed as far back as 1891, but the first practical demonstration didn’t occur until November 1946 over upstate New York. And judging by the growing amount of desert around our planet, this solution has not proved entirely adequate.

The compute services model referred to as the “public cloud” actually possesses some of the same attributes as its natural namesake. Resources are, in theory, unlimited – from a consumer’s perspective. But accessing these resources can be a headache.

Nonetheless, there’s plenty of motivation to develop solutions that enable consumers to easily, perhaps even automatically, take advantage of the application, compute, and storage services offered by public clouds. Business directives driving workloads to the cloud include operating with shrinking IT budgets and ending the purchase of new on-premise servers.

[Also read: HPC in the Cloud: Avoid These Common Pitfalls.]

The financial services industry offers a case in point; new IT infrastructure demands in this sector are often driven by increasing regulation. For example, satisfying the Fundamental Review of the Trading Book (FRTB) requirements leads to more frequent risk calculations on a wider range of data – more work for underlying infrastructure. And in more traditional high-performance computing (HPC) segments such as manufacturing or electronic design automation (EDA), increased IT requirements are driven by the push for higher fidelity results and shorter times to solutions using larger and multi-scale models and more complex simulations.

With compute requirements growing much faster than IT budgets, most organizations are already investigating how best to leverage the public cloud’s essentially limitless resources. In fact, the vast majority of enterprises are already drawing IT services from more than one public cloud provider.² The resulting “multicloud” environments are being enabled by rapid innovation within existing data management software to incorporate cloud connectivity and essentially transparent resource orchestration from either on- or off-premises sources. Enabling on-premises workloads to be sent to the cloud for processing, known as “cloud bursting,” can often be done automatically and controlled by policy, and the experience of the end user does not change. Most of the time the end user does not know on which resources, on-premises or cloud, their workloads are being run.

[Read also: The Perils of Becoming Trapped in the Cloud]

IBM Spectrum LSF, IBM Spectrum Symphony and IBM Spectrum Scale are fine examples of this type of innovation.

For years, HPC and grid computing users with large and small technical and on-premises computing systems have selected IBM Spectrum LSF and IBM Spectrum Symphony for policy-driven control and scheduling that translates into better application performance, better throughput, better utilization, and the ability to respond quickly to business demands while controlling costs.

Beyond reaching for the cloud, many HPC workloads are now living in the cloud. And Spectrum LSF and Spectrum Symphony can live there too. These IBM solutions can make these off-premises solutions appear and operate much the same as traditional locally handled workloads.

Spectrum LSF and Spectrum Symphony currently support dedicated, hybrid cloud and multi-cloud deployments with automated workload-aware cloud bursting to Amazon EC2, Microsoft Azure, IBM Cloud, Google Compute Cloud, and OpenStack.

Originally known as General Parallel File System (GPFS) back in the days of its HPC roots, IBM Spectrum Scale is a powerful data management solution that has expanded it HPC capabilities for enterprise use cases, and now leverages its native ability to manage many data streams from multiple sources to include the public cloud as well.

The days when nearly unlimited resources were locked away in the clouds are gone. In the 21^st century, HPC users can make it rain almost anytime they want.

---

Resources

¹ The Why Files: How much water is in the atmosphere, March 2017 https://whyfiles.org/2010/how-much-water-is-in-the-atmosphere/index.html

² IBM Institute for Business Value: Assembling Your Cloud Orchestra, October 2018 https://www.ibm.com/downloads/cas/EXLAL23W