Researchers who use supercomputers for science typically don’t limit themselves to one system. They move their projects to whatever resources are available, often using many different systems simultaneously, in their lab, on their campus cluster, at advanced computing centers like TACC, and in the cloud.
It’s not a lack of loyalty, it’s just a fact of research life — opportunities are shifting and hungry scientists find what they need to get their research accomplished.
The systems researchers use aren’t necessarily the same though. They may contain different hardware with different architectures, and different compilers or libraries.
This opportunistic computing paradigm creates a lot of extra work for computational scientists and system administrators – adapting old codes to work on new systems or installing software packages multiple times. As operating systems evolve, supporting code that was developed on deprecated environments becomes a reproducibility challenge.
In recent years, a new solution has emerged. Generically called “containers,” it involves a form of isolation, where a researcher’s code is packaged together with all the software dependencies in such a way that it can run at many sites without requiring recompilations. By incorporating an application’s many dependencies into self-sustainable images, containers avoid a lot of problems.
Popularized by Docker in 2013, containers were quickly accepted in both the commercial and academic scientific computing arenas. TACC was an early adopter and began enabling containerized science in 2016 – first through Docker, and more recently through Singularity, which was released in 2015 by a team of researchers at Lawrence Berkeley National Laboratory, and is particularly well-suited to high-performance computers, which contain tens of thousands of tightly connected processors.
Among the users of containers on TACC systems are Thomas Hughes — professor of Aerospace Engineering and Engineering Mechanics at The University of Texas at Austin and a member of the National Academies of Science and Engineering — and David Kamensky — a former member of Hughes’ team, now an assistant professor at the University of California, San Diego. The pair use containers to develop predictive models of coronary artery flow and study turbulence.
“The reason we started using containers was to run the numerical PDE [partial differential equation] software FEniCS on Stampede2,” said Kamensky. “FEniCS is a complicated software with many dependencies, and it can be difficult to install.”
When they needed to perform an isogeometric analysis on top of FEniCS, they converted a Docker image maintained by the FEniCS project team into a Singularity image and ended up using more than 1,000 node hours on Stampede2.
Collaborating with John Evans at the University of Colorado, Boulder (CU Boulder) on a turbulence modeling study, they were able to easily switch from Stampede2 to CU Boulder’s cyberinfrastructure because of containerization.
“I don’t see it as practical for supercomputer centers to maintain and debug all the different software to meet every scientist’s needs,” Kamensky said. “With Singularity, they only need to maintain one piece of software.”
Link to full article: https://www.tacc.utexas.edu/-/supporting-portable-reproducible-computational-science