Utility Computing and HPC

By Nicole Hemsoth

August 11, 2006

When peak work loads call for extra computing cycles, HPC managers and directors must find the resources they need without breaking budgets. A new computing model is emerging that allows HPC organizations to rent the processing power they need — whenever and wherever they need it. This model, called utility computing, is an effective solution for project-oriented engineering and research teams that have critical time, space, power, and capital constraints.

HP is a leader in developing and providing utility computing services across the globe, especially for HPC applications such as computer-aided engineering (CAE) and life and materials sciences. HPCwire sat down with Norman Lindsey, chief architect, HP Flexible Computing Business Unit, to get his perspective on utility computing in HPC and HP's latest offering in this market, HP Flexible Computing Services.

HPCwire: What challenges do HPC managers face with today's “fixed” computing centers and assets for HPC?

Lindsey: HPC managers are continually trying to improve the scalability of their systems to meet peak activity levels, build in greater flexibility to respond to change, and enhance services for users — and to do all of this at the lowest possible cost. HPC assets are expensive and prone to rapid obsolescence, and they are frequently allocated inappropriately or without the right level of utilization to make them cost-effective. When this happens, not only does the bottom line suffer, but user satisfaction with IT performance decreases as well.

HPCwire: Is utility computing the answer to today's HPC provisioning challenges? Does this type of model make sense?

Lindsey: As with all IT, HPC organizations realize that they need to improve the flexibility and cost-effectiveness of their technology. Utility computing solutions can be ideal for project-oriented engineering and research teams that have critical peak requirements, such as we often see in HPC organizations. With a pay-as-you-go, usage-based solution, HPC professionals working in CAE, life and materials sciences, weather research, and geosciences can 'scale up' on processing power when and where required.

Often compared to the model used by public energy utilities, this “IT as a service” model is relatively new to the industry. When you consider the evolution of energy utilities over the past century, a services model for IT makes sense. In the early twentieth century, large businesses had to build and operate their own power plants to generate the electricity required to run their businesses and compete in the marketplace. Over time, these businesses realized that it was smarter to offload the burden of managing large internal infrastructures to utility service providers, who could provide electricity when and where it was needed.
The same trend is occurring in IT today. HPC directors are realizing the value of a hybrid computing environment — owning a portion of fixed assets for critical jobs and augmenting these resources with utility computing for peak demand or as a long-term, flexible resource.

HPCwire: If I were an HPC manager and decided to move part of my infrastructure to utility computing, what are the key benefits I would see?

Lindsey: There are numerous compelling IT and business advantages to a utility computing model for HPC. The most obvious benefit is scalable computing power for highly variable workloads. You can scale up and down quickly, synchronizing computing resources with your requirements — in a way that fixed assets can't. Utility computing is designed to deliver as much power as your HPC users need, whenever they need it. No more missed deadlines because you didn't have enough computing resources.

Utility computing also enables smaller organizations to access capabilities far beyond what their budgets would normally allow; letting them “punch above their weight.” Smaller organizations can quickly address projects they would ordinarily not be able to deliver.

The flexibility utility computing provides also enables you to try out new technologies or techniques. You can use the utility computing resources as an IT 'sandbox.' For example, you can try running a certain application in parallel or on a different version of Linux, to experiment with performance improvements. If it doesn't work out, no worries. If it does, the procedures are available to reproduce the environment immediately.

Utility computing can also provide more cost-effective HPC and improved budget flexibility. Many high-performance compute clusters as well as software licenses are underutilized. Since these services are purchased as-needed on an expensed basis, this model provides you with flexibility and control over costs. In contrast, traditional HPC purchases require long budgeting cycles, tie up large amounts of capital, and lock you into a long-term depreciation cycle.

Another utility computing benefit many people forget about is that utility computing can save a lot of time and money compared to purchasing and provisioning a new HPC environment on your own. It takes a typical organization at least six weeks to purchase and provision a new HPC environment or an addition to their current infrastructure if done on their own. However, if the same level of resource is acquired through utility computing, it can be up and running in literally a few days. An organization like HP can set up the configuration faster because we have the expertise, and we are set up to do it. We do it every day.

HPCwire: There is so much hype about utility computing today, how do you choose a vendor? What should an HPC architect look for?

Lindsey: Not surprisingly, many claims are being made about this new model. When you are trying to decipher the facts, look for providers that have real-world experience in delivering this new class of solutions.

Make certain that the vendor you are considering is a recognized leader in delivering HPC solutions for analytics applications, for instance, where much of the activity is today. The ability to service the associated peak-demand requirements of real customers across a number of specialized industries is an indicator of competence in the emerging utility climate. In addition, look to see if the vendor has made significant investments in utility and grid technology research over a number of years. Look for leadership in industry venues such as the Global Grid Forum and the Enterprise Grid Alliance. Consider whether substantial capital commitment to a global service infrastructure is in place to enable delivery of the scalable, flexible capacity needed to meet both planned and unpredictable peaks in demand. Finally, make sure the security model is robust enough to keep your IP safe; safe from outsiders and protected from inadvertent disclosure by the vendor.

HPCwire: Why should someone choose HP if they are considering a move into utility computing?

Lindsey: Experience and industry involvement is what sets HP apart from others in the utility computing arena. HP is a leader in the utility market today and has done all of the things I listed in my previous answer for things to look for in a vendor. Long-term research at HP Labs has been at the forefront of utility computing technology for more than a decade, providing a solid foundation for HP's utility computing solutions. The utility approach to computing is core to HP's vision of an Adaptive Enterprise, where business and IT are synchronized to capitalize on change.

HP is the market leader in HPC and has been for three consecutive years according to IDC. Our utility computing solutions leverage this HPC knowledge and capability to deliver the best experience and performance in the industry. We are actively involved in setting up the flexible computing model for our customers today, and we continue to explore future scenarios. We are continually seeking solutions that meet tomorrow's utility-computing challenges in the critical areas of business model, governance, infrastructure, and utility trust and security.

In addition to experience, several other important factors differentiate HP from the competition: world-class, industry-standard products; extremely high levels of security; comprehensive application ISV partnerships, flexibility in pricing and offerings, and a strong focus on vertical HPC solutions.

HPCwire: In November of last year, HP announced their latest utility computing offering — HP Flexible Computing Services. Can you tell us about this service?

Lindsey: HP Flexible Computing Services bring together our industry-leading infrastructure components and application-vendor solutions to provide organizations with secure, reliable, and scalable utility computing services. The cost-per-unit pricing of HP Flexible Computing Services is ideal for customers with compute-intensive jobs that can outstrip the capacity found even in well-equipped HPC environments.

Of particular interest for HPC customers is our Application Provisioning Service (APS) for CAE, which is an industry first. This complete application utility service includes many leading CAE applications for structural, crash, and fluid analysis. Some of HP's partners in this program include Abaqus, Fluent, LSTC, and MSC.Software. These partners provide CAE software on the utility infrastructure with the goal of enabling customers to access and pay for the entire computing stack — the hardware, operating system, and application software — on a flexible payment basis.

In addition to the Application Provisioning Service, HP's Flexible Computing Services portfolio includes the basic Infrastructure Provisioning Service (IPS) that provides a highly available IT infrastructure utility service with enhanced security delivered out of HP's world-class data centers. The next level, IPS+, includes the basic service, plus HP installs and manages workload and grid management software and/or software development environments for application developers. The APS includes either of these services, combined with the installation of key vertical industry application software to a qualified configuration.

HP's flexible computing solution may include HP ProLiant servers, Integrity servers, HP StorageWorks arrays, ProCurve networking products, support from world-class data center facilities, proven HP management capabilities, and expertise in Linux, HPUX and Microsoft Windows. And to help customers get started in utility computing in a low-risk, low-cost way, HP offers “a try before you buy” option.

HPCwire: Could you explain this 'try before you buy' option in more detail?

Lindsey: Yes, I was referring to the HP Flexible Computing Club. HP is the first to offer a low-risk, low-cost way to determine whether utility computing is right for your business. This includes startup and consulting services and a 48-hour pilot project.

HPCwire: Many HPC directors hesitate to embrace the utility computing model because of security concerns. Security is a legitimate concern, correct?

Lindsey: Absolutely! When choosing a vendor, make certain that they can provide you with a level of security you can trust in all areas — physical security, network and data security, and data isolation.

HP systems are protected and secured physically and logically with detailed procedures and policies, using the best IT methodologies and corporate practices in existence. HP facilities have multiple layers of physical security, with strictly limited and audited biometrically controlled access to data center areas. System administration and management is performed by either the customer (HP turns over “root”) or trusted HP employees bound by HP's contractual obligations to our customers and by HP's internal Standards of Business Conduct. Customer workloads run in securely isolated network containers. Rigorous network security models protect against attacks from external and internal sources, and all customer applications and data are fully removed and scrubbed from memory and disk at the completion of an engagement.

HPCwire: What about reliability? How can an HPC manager be sure that resources will be available when they are needed?

Lindsey: HP Flexible Computing Services are delivered from HP's own world-class HPC facilities, in Tier 3 datacenters, staffed with skilled HPC professionals and featuring redundant power, environmental, and network capability to help maximize availability of the service. HP's service platform is designed for high system reliability under high duty-cycle loading. All platform components include availability features such as RAID 5/6 array storage, redundant trunked network links, redundant fans and power supplies, optimized cooling flow, and Integrated Lights-Out management. HP maintains multiple hot-spare compute nodes online at all times to help ensure fast restart of any lost jobs. Our utility platform is monitored and supported 24×7×365.

HPCwire: Do you have any HPC customers using the HP Flexible Computing Services?

Lindsey: Yes, HP has proven experience in a wide range of application areas. We are delivering utility solutions to such demanding clients as Abaqus (customer support, application QA and regression testing), and DreamWorks Animation (CG animation rendering for feature films).

—–

Norman Lindsey Norman serves as the Chief Architect for Flexible Computing Services at Hewlett-Packard, active in customer interactions and strategic planning. Norman's background includes more than twenty years experience in various HPC roles. As part of his HPC achievements at HP, Norman has led the CAE program to a top ranked market position and is currently leading HP's utility computing solutions.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion XL — were added to the benchmark suite as MLPerf continues Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing power it brings to artificial intelligence.  Nvidia's DGX Read more…

Call for Participation in Workshop on Potential NSF CISE Quantum Initiative

March 26, 2024

Editor’s Note: Next month there will be a workshop to discuss what a quantum initiative led by NSF’s Computer, Information Science and Engineering (CISE) directorate could entail. The details are posted below in a Ca Read more…

Waseda U. Researchers Reports New Quantum Algorithm for Speeding Optimization

March 25, 2024

Optimization problems cover a wide range of applications and are often cited as good candidates for quantum computing. However, the execution time for constrained combinatorial optimization applications on quantum device Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at the network layer threatens to make bigger and brawnier pro Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HBM3E memory as well as the the ability to train 1 trillion pa Read more…

MLPerf Inference 4.0 Results Showcase GenAI; Nvidia Still Dominates

March 28, 2024

There were no startling surprises in the latest MLPerf Inference benchmark (4.0) results released yesterday. Two new workloads — Llama 2 and Stable Diffusion Read more…

Q&A with Nvidia’s Chief of DGX Systems on the DGX-GB200 Rack-scale System

March 27, 2024

Pictures of Nvidia's new flagship mega-server, the DGX GB200, on the GTC show floor got favorable reactions on social media for the sheer amount of computing po Read more…

NVLink: Faster Interconnects and Switches to Help Relieve Data Bottlenecks

March 25, 2024

Nvidia’s new Blackwell architecture may have stolen the show this week at the GPU Technology Conference in San Jose, California. But an emerging bottleneck at Read more…

Who is David Blackwell?

March 22, 2024

During GTC24, co-founder and president of NVIDIA Jensen Huang unveiled the Blackwell GPU. This GPU itself is heavily optimized for AI work, boasting 192GB of HB Read more…

Nvidia Looks to Accelerate GenAI Adoption with NIM

March 19, 2024

Today at the GPU Technology Conference, Nvidia launched a new offering aimed at helping customers quickly deploy their generative AI applications in a secure, s Read more…

The Generative AI Future Is Now, Nvidia’s Huang Says

March 19, 2024

We are in the early days of a transformative shift in how business gets done thanks to the advent of generative AI, according to Nvidia CEO and cofounder Jensen Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, codenamed Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

Intel Won’t Have a Xeon Max Chip with New Emerald Rapids CPU

December 14, 2023

As expected, Intel officially announced its 5th generation Xeon server chips codenamed Emerald Rapids at an event in New York City, where the focus was really o Read more…

IBM Quantum Summit: Two New QPUs, Upgraded Qiskit, 10-year Roadmap and More

December 4, 2023

IBM kicks off its annual Quantum Summit today and will announce a broad range of advances including its much-anticipated 1121-qubit Condor QPU, a smaller 133-qu Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire