SC17: Singularity Preps Version 3.0, Nears 1M Containers Served Daily

By John Russell

November 1, 2017

Just a few months ago about half a million jobs were being run daily using Singularity containers, the LBNL-founded container platform intended for HPC. That was already a big number for the young open source project. Today, the number is closer to one million per day says Gregory Kurtzer, former HPC Systems Architect for Lawrence Berkeley National, the original driving force behind Singularity, and now the CEO of SingularityWare LCC.

It’s increasingly clear that the appetite for containers in scientific computing is every bit as strong as it is in enterprise computing where several technologies are jostling but Docker remains firmly the king. Containers, of course, aren’t new. (I’ve probably written exactly those words before) Like virtual machines (VM) before them, containers embody the idea of encapsulating compute environments to enhance application and workflow portability (and reproducibility) across varying compute infrastructures. Gather what you need into a single ‘box’, wrap it with enough standardized hooks to play nicely on other machines running Singularity, and game on. Build it once. Use (and share) it many times.

This separation of the software environment (and all of its dependencies) from the systems it runs on is an important paradigm change, says Kurtzer, especially in HPC where tight coupling of software to the specific underlying hardware has long been the guiding principle for squeezing out maximum performance. It still is. But for many jobs squeezing out optimum performance is less important than achieving portability and consistency of adequate performance.

HPCwire recently talked with Kurtzer about Singularity’s rapid growth, its technology roadmap, its staffing challenge, and Kurtzer’s plans to create a little excitement for Singularity at SC17. He is chary of spending on an SC booth given that SC newcomers often land in far corners with limited traffic. Instead Kurtzer is sponsoring a scavenger hunt involving many booths of Singularity stakeholders (preliminary list of participating booths at the end of the article). We’ll see how that works out.

In the meantime there’s lots to talk about. Singularity 2.4 was released last month with 3.0 expected early in 2018. (The Singularity FAQ page provides a good overview of features.) Importantly support from the big and small government and academic computing centers has been significant. Many of the familiar supercomputing centers, for example OCLF, TACC, NASA Ames, SDSC, Sandia, NIH, and LBNL, are using Singularity in production. Though not comprehensive the Singularity registry (voluntary) provides snapshot of Singularity user base.

It’s also noteworthy that now that Singularity has made substantial inroads into the traditional HPC community, Kurtzer is planning expansion into the enterprise computing arena. That seems in keeping with many traditional HPC technology suppliers who have pivoted to the enterprise in search of growth. What follows is a portion of our conversation plus a few slides from a recent presentation by Kurtzer to a national lab.

HPCwire: Seems like the use of Singularity is growing quickly, even in the relatively short time since we talked last (see, Singularity HPC Container Technology Moves Out of the Lab). Besides adoption what been happening with Singularity itself.

Gregory Kurtzer, SingularityWare CEO

Greg Kurtzer: We’ve been focusing a lot on the 2.4 release which I am hoping will actually be out maybe even tonight (it’s now out). There are a couple important new features in 2.4. The biggest is something we are calling instance support. Basically it’s the ability to run persistent namespaces such that you can come back and join it after the fact. It’s almost like a virtual machine; you can start up a Singularity container, run your jobs and everything you want in it, and when you want to leave it and exit, you can leave and exit and then you can come back to it later and it is still running.

The other big change is we have moved to compressed, immutable images. By default, previously everything was embedded in a read write format that emulates a file system. Now what we are doing is something that will be a little bit different because it is much more optimal in terms of space consumption on your hard drive because it is always compressed and actually even runs compressed. You can execute and use a compressed container without ever having to un-compress it.

Also, we now have ability to do things like persistent overlays so you can capture all the deltas in a container and then use that as a data container, a new concept we are playing with.

HPCwire: I understand that 3.0 is actually not far off. Can you give a preview? What’s on the technology roadmap.

Kurtzer: I can give you a little bit of a heads ups on what is coming in future versions. We’re coining a new term, or at least I think we are as I have not heard it used before us, called data containers which is really along the line of encapsulation of data. Currently we have encapsulations of environments, operating environments, etc. but there are also really good reasons to encapsulate your data. We are going to have additional abilities with security to further limit security exposure to host systems. That’s a big one. A lot of supercomputing centers won’t even have to make Singularity “setuid” root; they’ll basically just use additional Linux capabilities for increasing security privileges in order to run containers.

The big change we are going for [in 3.0] is introducing something called the Singularity image format, SIF for short. We are changing the major version name from 2.x to 3.x because we are changing the image format. Every time we have changed the image format we have changed the major number. SIF is basically a single file image because the whole context of Singularity is a single image; that’s really the main point of what a singularity container is, differentiating itself from other container systems or at least that is one of the main ones. We are spending a lot of effort defining what that single image looks like.

This SIF file will have the ability to have multiple data regions within the file. In a manner of speaking you can have multiple containers or multiple container layers, within a single file. Let’s say it’s running CentOS with MySQL and some sort of genomics application that queries the SQL database and then can do other things. You can have that in the base image and that base is immutable, it cannot change. Then when we have another data region which is writeable where we capture all of the modifications to that image in the writable layer of the SIF file. This is important because we can checksum and sign the multiple data regions independently while still allowing the data within the container to change or evolve.

Also we are bringing forth the idea of cryptographically signing of containers which is actually huge because the other container systems that are out there, even in enterprise, don’t have the ability to cryptographicly sign a container in its runtime form. Thanks to the SIF format we can have sections of that file which are signed and other sections which are using overlays. To give a use case, let’s stick with the genomics example; let’s say you have a database server running and you are querying and updating that genomics database and as time goes on that database is going to grow. If you think about this in a traditional way, that would break the signing, it would break the validation of your container. You wouldn’t be able to verify it anymore because you just changed your image. But because it is based on an immutable base and the data in that base layer can be signed independently within the image, we have the ability to have containers that evolve with time without breaking signatures.

HPCwire: What else is on the docket for 3.0 or beyond?

Kurtzer: The SIF image format includes image signing and image verification/validation, and we are also additional support for network namespaces. Right now we have network namespace isolation, but this new set of features will give us the ability for a container to come up as a virtual IP address. That obviously will require a privileged user; that can’t be done with a regular user because it will be changing network configurations. We’re also looking into CGroup support (control groups, Linux), and virtual booting of instances. The idea of virtual booting of those instances would basically allow you to run a Singularity container just like a VM. We have already done prototypes of this and it only takes it a fraction of a second to boot so it is very fast.

Container performance monitoring is also on the roadmap but I am not sure if we are going to get that into 3.0. It’s basically the ability to do performance profiling through a container. Our goal, really, is to make Singularity a feature-full, science enabling platform.

HPCwire: Maybe this is a good spot to review the Singularity user base. How is support from the major science computing centers? What does a typical user look like? Is it mostly the “long tail of science” type of users and organizations?

Kurtzer: Well, Titan (OCLF supercomputer center at Oak Ridge National Lab) among several other top 10 supercomputers on the Top500 are running Singularity. We have had very strong support from a wide range of major computing centers including older and new machines. As for who the user is, we have people that are not only on the long tail of science but also people on the cutting edge of their computational domain. This is because Singularity changes the software distribution and archival paradigm as the container can fit within a researcher’s existing data management solutions.

Singularity is already running on most of the large centers; what I want to focus on now is enabling the independent scientists as much as we can and to focus on this notion of enterprise HPC. I’ve been contacted by multiple vendors, tier 1 and tier 2, on something that they are calling “Enterprise HPC.” It’s basically enterprise sites who have little or no expertise in HPC but who are starting to run HPC-like workloads (machine learning and compute driven analytics). Singularity is being targeted by many organizations as the vector for basically dealing with these applications and these workloads because these enterprises don’t have the HPC expertise to be building or maintaining all of these workflows. The vendors are talking about building workflows – e.g., machine learning or analytics workflows – and saying ‘we want to distribute those as singularity containers.’ So singularity is going to be really big, I think, in this kind of hybrid mix between enterprise and HPC.

Many of these enterprise HPC jobs are not the tightly-coupled highly parallelized jobs we’ve come to see as commonplace on HPC. When someone says HPC it means something really specific to traditional HPC folks; it’s tightly coupled, we’ve got some sort of low latency interconnect, parallel file systems, designed to run high performance, highly scalable custom applications. But today, this has changed. HPC has come to mean pretty much any form of scientific computing and as a result, its breadth has grown in terms of what kind of applications we need to support. The traditional HPC architecture is not as applicable to a general wide use case scenario.

Singularity has lowered the barrier of entry to HPC considerably. People can create their own workflows, can leverage Docker, can and leverage other people’s containers via singularity hub to recapitulate people’s workflows and then further expand on this basis.

Even people who are doing more traditional HPC type jobs that are tightly coupled and whatnot are looking into containers to escape some of the dependency issues and some of the difficulties in creating those workflows and archive and/or distribute their software stacks.

HPCwire: Surely not all HPC applications are easily containerized.

Kurtzer: True. Fluent (CFD simulation, ANSYS) for example is extremely difficult to containerize. This is because to do a multi-node parallel processing job Fluent wants to run the MPI but the host resource manager should be controlling the MPI. We end up in this chicken and egg problem as the MPI within the container actually wants to be the MPI outside the container. Having vendor buy-in and support in how we properly containerize these applications is critical (hint, hint ANSYS, let’s talk.).

HPCwire: Let’s change gears and talk about the Singularity organization. How’s it going?

Kurtzer: It is fantastic. I’ve created a few companies previously, and several open source projects, including CentOS Linux, and the growth and commercial interest in Singularity has surpassed all of them! At this point, I am building my core team. I am looking for experienced developers, great minds, and people who want to change the face of computing. It is an extremely surreal experience, and I am looking for the most fantastic of people to join in this project. Funding is available so it just comes down to finding the right people.

To that point, if there are any readers out there interested in being part of this endeavor reach out to me, and let’s talk.

Preliminary List of Organizations Supporting Singularity at Their Booths

  1. Bright Computing
  2. Globus
  3. HPCwire
  4. MVAPICH2/Ohio State University
  5. Penguin Computing
  6. RedBarn Computing
  7. Rutgers
  8. SSERCA (Sunshine State Educational and Research Computing Alliance)
  9. Texas Tech University Booth
  10. University of Michigan / Michigan State

Slide Source: Kurtzer

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Supercomputer Research Reveals Star Cluster Born Outside Our Galaxy

July 11, 2020

The Milky Way is our galactic home, containing our solar system and continuing into a giant band of densely packed stars that stretches across clear night skies around the world – but, it turns out, not all of those st Read more…

By Oliver Peckham

Max Planck Society Begins Installation of Liquid-Cooled Supercomputer from Lenovo

July 9, 2020

Lenovo announced today that it is supplying a new high performance computer to the Max Planck Society, one of Germany's premier research organizations. Comprised of Intel Xeon processors and Nvidia A100 GPUs, and featuri Read more…

By Tiffany Trader

Xilinx Announces First Adaptive Computing Challenge

July 9, 2020

A new contest is challenging the computing world. Xilinx has announced the first Xilinx Adaptive Computing Challenge, a competition that will task developers and startups with finding creative workload acceleration solutions. Xilinx is running the Adaptive Computing Challenge in partnership with Hackster.io, a developing community... Read more…

By Staff report

Reviving Moore’s Law? LBNL Researchers See Promise in Heterostructure Oxides

July 9, 2020

The reality of Moore’s law’s decline is no longer doubted for good empirical reasons. That said, never say never. Recent work by Lawrence Berkeley National Laboratory researchers suggests heterostructure oxides may b Read more…

By John Russell

President’s Council Targets AI, Quantum, STEM; Recommends Spending Growth

July 9, 2020

Last week the President Council of Advisors on Science and Technology (PCAST) met (webinar) to review policy recommendations around three sub-committee reports: 1) Industries of the Future (IotF), chaired be Dario Gil (d Read more…

By John Russell

AWS Solution Channel

Best Practices for Running Computational Fluid Dynamics (CFD) Workloads on AWS

The scalable nature and variable demand of CFD workloads makes them well-suited for a cloud computing environment. Many of the AWS instance types, such as the compute family instance types, are designed to include support for this type of workload.  Read more…

Intel® HPC + AI Pavilion

Supercomputing the Pandemic: Scientific Community Tackles COVID-19 from Multiple Perspectives

Since their inception, supercomputers have taken on the biggest, most complex, and most data-intensive computing challenges—from confirming Einstein’s theories about gravitational waves to predicting the impacts of climate change. Read more…

Penguin Computing Brings Cascade Lake-AP to OCP Form Factor

July 7, 2020

Penguin Computing, a subsidiary of SMART Global Holdings, Inc., announced yesterday (July 6) a new Tundra server, Tundra AP, that is the first to implement the Intel Xeon Scalable 9200 series processors (codenamed Cascad Read more…

By Tiffany Trader

Max Planck Society Begins Installation of Liquid-Cooled Supercomputer from Lenovo

July 9, 2020

Lenovo announced today that it is supplying a new high performance computer to the Max Planck Society, one of Germany's premier research organizations. Comprise Read more…

By Tiffany Trader

President’s Council Targets AI, Quantum, STEM; Recommends Spending Growth

July 9, 2020

Last week the President Council of Advisors on Science and Technology (PCAST) met (webinar) to review policy recommendations around three sub-committee reports: Read more…

By John Russell

Google Cloud Debuts 16-GPU Ampere A100 Instances

July 7, 2020

On the heels of the Nvidia’s Ampere A100 GPU launch in May, Google Cloud is announcing alpha availability of the A100 “Accelerator Optimized” VM A2 instance family on Google Compute Engine. The instances are powered by the HGX A100 16-GPU platform, which combines two HGX A100 8-GPU baseboards using... Read more…

By Tiffany Trader

Q&A: HLRS’s Bastian Koller Tackles HPC and Industry in Germany and Europe

July 6, 2020

In this exclusive interview for HPCwire – sadly not face to face – Steve Conway, senior advisor for Hyperion Research, talks with Dr.-Ing Bastian Koller about the state of HPC and its collaboration with Industry in Europe. Koller is a familiar figure in HPC. He is the managing director at High Performance Computing Center Stuttgart (HLRS) and also serves... Read more…

By Steve Conway, Hyperion

OpenPOWER Reboot – New Director, New Silicon Partners, Leveraging Linux Foundation Connections

July 2, 2020

Earlier this week the OpenPOWER Foundation announced the contribution of IBM’s A21 Power processor core design to the open source community. Roughly this time Read more…

By John Russell

Hyperion Forecast – Headwinds in 2020 Won’t Stifle Cloud HPC Adoption or Arm’s Rise

June 30, 2020

The semiannual taking of HPC’s pulse by Hyperion Research – late fall at SC and early summer at ISC – is a much-watched indicator of things come. This yea Read more…

By John Russell

Racism and HPC: a Special Podcast

June 29, 2020

Promoting greater diversity in HPC is a much-discussed goal and ostensibly a long-sought goal in HPC. Yet it seems clear HPC is far from achieving this goal. Re Read more…

Top500 Trends: Movement on Top, but Record Low Turnover

June 25, 2020

The 55th installment of the Top500 list saw strong activity in the leadership segment with four new systems in the top ten and a crowning achievement from the f Read more…

By Tiffany Trader

Supercomputer Modeling Tests How COVID-19 Spreads in Grocery Stores

April 8, 2020

In the COVID-19 era, many people are treating simple activities like getting gas or groceries with caution as they try to heed social distancing mandates and protect their own health. Still, significant uncertainty surrounds the relative risk of different activities, and conflicting information is prevalent. A team of Finnish researchers set out to address some of these uncertainties by... Read more…

By Oliver Peckham

[email protected] Turns Its Massive Crowdsourced Computer Network Against COVID-19

March 16, 2020

For gamers, fighting against a global crisis is usually pure fantasy – but now, it’s looking more like a reality. As supercomputers around the world spin up Read more…

By Oliver Peckham

[email protected] Rallies a Legion of Computers Against the Coronavirus

March 24, 2020

Last week, we highlighted [email protected], a massive, crowdsourced computer network that has turned its resources against the coronavirus pandemic sweeping the globe – but [email protected] isn’t the only game in town. The internet is buzzing with crowdsourced computing... Read more…

By Oliver Peckham

Supercomputer Simulations Reveal the Fate of the Neanderthals

May 25, 2020

For hundreds of thousands of years, neanderthals roamed the planet, eventually (almost 50,000 years ago) giving way to homo sapiens, which quickly became the do Read more…

By Oliver Peckham

DoE Expands on Role of COVID-19 Supercomputing Consortium

March 25, 2020

After announcing the launch of the COVID-19 High Performance Computing Consortium on Sunday, the Department of Energy yesterday provided more details on its sco Read more…

By John Russell

Honeywell’s Big Bet on Trapped Ion Quantum Computing

April 7, 2020

Honeywell doesn’t spring to mind when thinking of quantum computing pioneers, but a decade ago the high-tech conglomerate better known for its control systems waded deliberately into the then calmer quantum computing (QC) waters. Fast forward to March when Honeywell announced plans to introduce an ion trap-based quantum computer whose ‘performance’ would... Read more…

By John Russell

Neocortex Will Be First-of-Its-Kind 800,000-Core AI Supercomputer

June 9, 2020

Pittsburgh Supercomputing Center (PSC - a joint research organization of Carnegie Mellon University and the University of Pittsburgh) has won a $5 million award Read more…

By Tiffany Trader

Global Supercomputing Is Mobilizing Against COVID-19

March 12, 2020

Tech has been taking some heavy losses from the coronavirus pandemic. Global supply chains have been disrupted, virtually every major tech conference taking place over the next few months has been canceled... Read more…

By Oliver Peckham

Leading Solution Providers

Contributors

10nm, 7nm, 5nm…. Should the Chip Nanometer Metric Be Replaced?

June 1, 2020

The biggest cool factor in server chips is the nanometer. AMD beating Intel to a CPU built on a 7nm process node* – with 5nm and 3nm on the way – has been i Read more…

By Doug Black

Nvidia’s Ampere A100 GPU: Up to 2.5X the HPC, 20X the AI

May 14, 2020

Nvidia's first Ampere-based graphics card, the A100 GPU, packs a whopping 54 billion transistors on 826mm2 of silicon, making it the world's largest seven-nanom Read more…

By Tiffany Trader

‘Billion Molecules Against COVID-19’ Challenge to Launch with Massive Supercomputing Support

April 22, 2020

Around the world, supercomputing centers have spun up and opened their doors for COVID-19 research in what may be the most unified supercomputing effort in hist Read more…

By Oliver Peckham

Australian Researchers Break All-Time Internet Speed Record

May 26, 2020

If you’ve been stuck at home for the last few months, you’ve probably become more attuned to the quality (or lack thereof) of your internet connection. Even Read more…

By Oliver Peckham

15 Slides on Programming Aurora and Exascale Systems

May 7, 2020

Sometime in 2021, Aurora, the first planned U.S. exascale system, is scheduled to be fired up at Argonne National Laboratory. Cray (now HPE) and Intel are the k Read more…

By John Russell

Summit Supercomputer is Already Making its Mark on Science

September 20, 2018

Summit, now the fastest supercomputer in the world, is quickly making its mark in science – five of the six finalists just announced for the prestigious 2018 Read more…

By John Russell

TACC Supercomputers Run Simulations Illuminating COVID-19, DNA Replication

March 19, 2020

As supercomputers around the world spin up to combat the coronavirus, the Texas Advanced Computing Center (TACC) is announcing results that may help to illumina Read more…

By Staff report

$100B Plan Submitted for Massive Remake and Expansion of NSF

May 27, 2020

Legislation to reshape, expand - and rename - the National Science Foundation has been submitted in both the U.S. House and Senate. The proposal, which seems to Read more…

By John Russell

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This