Katie Antypas
Services Department Head
NERSC

As Services department head for Lawrence Berkeley’s National Energy Research Scientific Computing Center (NERSC), Katie Antypas oversees Advanced Technologies, Data and Analytics Services, and User Services groups to increase the impact they have for the scientific community.

Antypas is also the project lead for the NERSC-8 system project that will deploy the center’s next-generation system in mid-2016 that will transition the broad DOE workload to a more energy efficient architecture. The system, named Cori (after Nobel laureate Gerty Cori), will be a Cray system featuring over 9300 Intel Knights Landing manycore processors.

From 2010 to 2013 Antypas engaged with scientists to help them best utilize NERSC resources in her role as group leader for User Services at NERSC. Prior to becoming the Group Leader of USG, she was a consultant in the group from 2006-2010 and was the co-implementation team lead on the Hopper project, NERSC’s first petaflop system.

HPCwire: Hi Katie. Congratulations on being selected as an HPCwire 2015 Person to Watch. You’re intimately involved with the roll out of Cori. Can you tell us a little bit about the system, its purpose, and your role with it in 2015?

Katie Antypas: The Cori system will be NERSC’s next supercomputer and will run applications that support the mission of the Department of Energy, Office of Science.  NERSC has over 5000 users across 700 projects so it is crucially important that the systems we deploy are productive and high performing for our diverse workload.  The computing industry, however, is facing a major challenge, how to increase performance without breaking power budgets on the path to exascale computing.

The Cori system will demonstrate a number of new technologies that we expect to see in future exascale systems, namely deeper memory hierarchies and a large number of ‘slower’ processor cores with longer vector units.  These features hold the key to achieving the highest performance on Cori, but to use them effectively many users will need to modify application codes.  And herein lies one of NERSC’s greatest challenges for the next couple of years, transitioning the broad DOE user community to more energy efficient manycore architectures while concurrently enabling users to make progress on their science goals.   To accomplish this, we launched the NERSC Exascale Science Application Program (NESAP), which will partner application teams with experts from Cray, Intel, and NERSC.  Twenty application teams that represent the broad DOE workload make up the core program.

NERSC is also working with library and tools teams as part of NESAP.  We plan to take the lessons learned from the NESAP program and create training programs and web tutorials for our broad user community.  As the project manager for the NERSC-8 Cori system project, I have the ultimate responsibility for the successful deployment, acceptance, and user satisfaction with the system, but no one person does this alone.  Deploying a new supercomputer is a massive multi-year effort that requires a team with expertise in architecture, system administration, file systems, applications, and facilities.  So far the NERSC-8 project has been very rewarding and a lot of fun.

HPCwire: What would you characterize as a distinguishing aspect of the Cori system? Also, what are the most significant hurdles have you have had to clear? Are there any in your field of view as you move towards the finish?

One distinguishing feature of the Cori system processor is the 16GB of high bandwidth on-package memory.  This on-package memory will have about 4-5 times the bandwidth of DRAM and is an important feature for our users because the majority of the NERSC workload is not CPU (flop) limited, but rather memory bound.

The challenge for our users will be how best to use the smaller amount of high bandwidth memory.  The on-package memory can be used as either a cache, which requires no user code changes, or allocated directly by the program.  Which mode will be the best for our users is something we are exploring as part of the NESAP project.

HPCwire: Given your vantage point, what trends do you see as the most significant in HPC as you work on Cori?

One trend I am personally very excited about is solid-state storage for HPC.  For many years the HPC community has struggled with parallel-I/O.  Trying to achieve high I/O performance over a wide range of I/O patterns and applications, running at various concurrencies when multiple users are simultaneously trying to access a disk based file system had been a real challenge for NERSC, and I think all HPC centers.

To satisfy users’ demand for I/O bandwidth we’ve been throwing more and more disks at the problem and asking users to perform I/O in ways that are unnatural to the application because it was the only way to get decent performance.  We are now seeing NVRAM flash being deployed into supercomputers because it has become cost competitive to provide I/O bandwidth using NVRAM flash, while still providing storage capacity with disk.

The Cori system will include a layer of NVRAM flash that sits between memory and disk, which is often called a ‘burst buffer’, and will serve to accelerate I/O performance.  We are currently working with Cray on the burst buffer software development.  With this software users will be able to reserve I/O bandwidth through the batch system like they do other resources.  I know the burst buffer will bring its own challenges, but for now I am optimistic.

HPCwire: On a more intimate level, what can you tell us about yourself – personal life, family, background, hobbies?

As someone who spends so much time indoors in front of a computer or sitting in meetings during the week, I really like to get outside into the fresh air on the weekends.  I like to run, hike, cycle and explore the beautiful West Coast.  I was born and raised in the Bay Area, but spent time in Boston and Chicago in as an undergraduate and grad school.  Some may call me a weather wimp, but I do think being able to spend quality time outdoors January through March (without massive amounts of layers and jackets) really does make me happier.

HPCwire: One last question – What can you share about yourself that you think your colleagues would be surprised to learn?

I’m a covert hot yoga fanatic (the classes where you sweat in a 105-degree room for 90 minutes doing weird poses.)    I’m not sure if it is exhaustion or dehydration, but I do always sleep much better after a class!

 

Katie Antypas NERSC	Buddy Bland Oak Ridge National Laboratory	Merle Giles NCSA
Eng Lim Goh SGI	Michael Heroux Computing Research Center	Ken King IBM
Bernd Mohr Jülich Supercomputing Centre	Jason Stowe Cycle Computing	Scott Tease Lenovo
Ryan Waite Cray	Charles Wuischpard Intel	Mike Yang Quanta