Like an ROI-driven enterprise, the University of Michigan achieves full cost recovery on its new Great Lakes supercomputer for scientific research.

On a university campus, talk of return on investment, cost recovery and business models might sound more like the language spoken in the business school than that of the research computing service. But that indeed is the language spoken by the leaders of the Advanced Research Computing Technology Services team at the University of Michigan.

While all academic and research centers work to contain costs, most aren’t challenged to recover the full costs of a system. Not so at the University of Michigan. The Advanced Research Computing team members operate the university’s new Great Lakes supercomputer as it might be operated in a corporate enterprise. The funding for the system and its operational expense are paid entirely by users, and investment decisions for campus-wide machines must factor in exceptional performance, usability and management characteristics, as well as 100-percent cost recovery.

This cost-recovery mission is spearheaded by Brock Palen, the director of Advanced Research Computing Technology Services. Palen and his team have to account for every cost associated with the 13,000-core Great Lakes supercomputer, from the hardware to data center maintenance, electricity, personnel, software environment and full depreciation.

Palen explains that ensuring cost recovery means making the smartest possible decisions based on a diverse set of workload profiles and generalizing value from those scattered points. As one might imagine, this is no easy task, especially with a system that serves 2,500 people running a rich set of  workloads, ranging from compute-intensive to data-intensive to I/O-bound. The Intel-Scalable-processor-based Great Lakes HPC Cluster from Dell Technologies is available to all researchers on campus for simulation, modeling, machine learning, data science, genomics and more. To meet the diverse needs of this large user community, Great Lakes provides a balanced combination of computing power, I/O performance, storage capability and server accelerators.

And this is where more good business practices enter the picture. Palen and his team work actively to understand what their customers need and want from them, just as a business would do.

“We had to do a lot of benchmarking to understand user behavior and what people wanted out of a system,” Palen says. “In that process we found we did not have a good sense of I/O demand. For instance, there are several life sciences jobs that put significant demands on the network and I/O, and even if they are not classic HPC, they still require a lot of the components of an HPC system from a high performance fabric and scratch file system.”

Palen and his team also realized that half of the workloads were not high core count jobs and that access to storage and high performance data movement were paramount. Nonetheless, they still needed to support the existing large-scale traditional HPC jobs, as well as support the growing use of machine learning across disciplines.

“Our business model on the legacy system encouraged users to run longer, thinner jobs,” Palen says. “People were more likely to get 20 cores for a month than 200 cores for a week even though the total amount of compute time was the same. We have fixed that with this system with payment by the hour consumed. We expect people to do one big job, finish it up, and get on with their science instead of going at the speed of the business model for the system.”

A cloud connection

In today’s world filled with private, public and hybrid cloud services, HPC shops are expected to function like multi-cloud service providers and cloud brokers. Many HPC operations now provide centralized compute resources with multiple storage systems and access to multiple internal and external clouds.

In addition to providing access to HPC clusters on campus, University of Michigan Advanced Research Computing Technology Services offers the campus research community access to multiple public clouds. These services include:

• Globus — a robust cloud-based file transfer service designed to move many large files
• Public cloud services — the university has service agreements with Amazon Web Services, Google Cloud Platform and Microsoft Azure
• GENI, the Global Environment for Network Innovations — a project that provides a virtual laboratory for networking and distributed systems research and education.

A few key takeaways

The University of Michigan is one of the top academic centers in the United States, with more than $1.5 billion in research expenditures in 2018. In its business-like approach to making leading-edge HPC and cloud services available to a huge research program, it is demonstrating how a university can run an HPC shop like a business that pays close attention to the needs of its customers while recovering all the costs associated with the services it delivers.

That’s always a good way to do business — in a public or private setting.

To learn more

For the full story, see the Dell Technologies case study, “Great Lakes Cluster.” Learn more about the University of Michigan Advanced Research Computing at https://arc.umich.edu/. And for a look at how the university charges for time on its new HPC cluster, see the Great Lakes Cluster Rates chart.