NERSC Invites Haswell-Based Cray XC40 Into Cori Fold

By Tiffany Trader

April 22, 2015

Nearly one year after the Department of Energy inked a $70 million contract for the exascale-relevant Cori system, news of a second, smaller system has come to light. In an interesting turn of events, the National Energy Research Scientific Computing Center (NERSC) revealed today that it will be acquiring a “Cori Phase 1” system, a 10-cabinet Cray XC40 machine outfitted with Intel “Haswell” parts to be installed this summer.

The new system is in addition to the original Cori contract – the outcome of the NERSC-8/Trinity procurement partnership – which is still on track for mid-2016 delivery. Named after American biochemist Gerty Cori, the next-gen Cray XC system with 9,300 self-hosted Intel Knights Landing (KNL) takes a manycore approach that is a distinct departure for the leading-edge of supercomputing.

The Phase 1 Cori will employ 16-core E5 v3 Xeons running at 2.3 Ghz, the follow-on to the Intel “Ivy Bridge” processors that drive Edison. The Cray XC40 will be the first NERSC system installed in the newly built Computational Research and Theory (CRT) facility at Lawrence Berkeley National Laboratory. Hopper, the Cray XE6 that was NERSC’s first petascale supercomputer, and another system, Carver, will not make the move to CRT; but will retire from the Oakland Scientific Facility (OSF) in downtown Oakland.

Speaking with HPCwire, Jay Srinivasan, NERSC’s Computational Systems Group lead, characterized the Haswell-based Cori as a familiar system that will bridge the gap between Hopper’s retirement and the KNL-based Cori in 2016. The upgraded Xeons mean that users can continue to run their applications without interruption in support of their efforts to develop new energy sources, improve energy efficiency, and understand climate change.

While NERSC was reticent to divulge total peak performance, Srinivasan reported that the Phase 1 system will offer its users roughly the same sustained application performance as Hopper, which touts a peak performance is 1.28 petaflops. The Lustre file system and dragonfly topology based on the Aries interconnect are identical to NERSC’s current Edison supercomputer (a Cray XC30). Each of the more than 1,400 Haswell compute nodes touts 128 gigabytes of memory, twice the per-node memory of Edison.

Beyond ensuring that there is no dip in compute supply for DOE users, other characteristics, from the processors’ enhanced instruction set to the additional memory bandwidth and accelerated application I/O, offer the opportunity “to explore new workflows and paths to computation,” said Srinivasan. Users that run data-intensive workloads on other NERSC systems now have the option to run on a Cray platform, he adds.

“They can start off by using the Haswell machine the same way they use Edison,” explained Srinivasan. “They can take their code and start running it on day one. Then there are other features that we have on there, the interactive nodes, the batch system policies, that allow for high-throughput and serial workflows and we will allow the compute nodes to interact with external databases, which a lot of data-intensive workflows need. They can then start bringing more data-intensive workflows into the machine, and start using it that way. They can have a firm foundation of familiarity and then start building on newer ways of using the system. Once we have Cori integrated, that will really provide a whole new computing paradigm, melding data-intensive computing with traditional computing.”

In its official press release, NERSC details a number of advanced features designed to benefit data-intensive applications, including:

* Large number of login/interactive nodes to support applications with advanced workflows.

* Immediate access queues for jobs requiring real-time data ingestion or analysis.

* High-throughput and serial queues can handle a large number of jobs for screening, uncertainty qualification, genomic data processing, image processing and similar parallel analysis.

* Network connectivity that allows compute nodes to interact with external databases and workflow controllers.

* The first half of an approximately 1.5 terabytes/sec NVRAM-based Burst Buffer for high bandwidth low-latency I/O.

* A Cray Lustre-based file system with over 28 petabytes of capacity and 700 gigabytes/second I/O bandwidth.

When work on Cori is finalized, 9,300 Knights Landing compute nodes and more than 1,900 Haswell nodes will be lashed together on the same high speed network, providing NERSC scientists with a twofold increase in application I/O acceleration. This speedup is owed to Cray’s DataWarp “Burst Buffer” technology, which uses NVRAM to move data more quickly between processor and disk. The Cori Phase 1 burst buffer will feature approximately 750 terabytes of capacity and approximately 750 gigabytes/second of I/O bandwidth. The completed Cori supercomputer will double these specs, boosting capacity to more than 1.5 petabytes and I/O bandwidth to more than 1.5 terabytes/second.

The staff at NERSC emphasized how the whole in this case is more than the sum of its parts, offering unique benefits to the large DOE user base.

“The line between big data and high performance computing is really very blurred, especially in computational science,” said Katie Antypas, head of NERSC’s User Services Department in a prepared statement. “The combined Cori system is the first system to be specifically designed to handle the full spectrum of computational needs of DOE researchers, as well as emerging needs in which data- and compute-intensive work are part of a single workflow. For example, a scientist will be able to run a simulation on the highly parallel Knights Landing nodes while simultaneously performing data analysis using the Burst Buffer on the Haswell nodes. This is a model that we expect to be important on exascale-era machines.”

NERSC and Cray also announced that they are working together on two ongoing R&D efforts aimed at enabling data-intensive science. One project seeks to maximize Cori’s data potential by enabling higher bandwidth network capability to the outside world, and the second project is focused on putting Linux container virtualization functionality on Cray compute nodes to allow custom software stack deployment.

The price of the new system was not disclosed, but it does fall under a separate contract from the original Cori win.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from its predecessors, including the red-hot H100 and A100 GPUs. Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. While Nvidia may not spring to mind when thinking of the quant Read more…

2024 Winter Classic: Meet the HPE Mentors

March 18, 2024

The latest installment of the 2024 Winter Classic Studio Update Show features our interview with the HPE mentor team who introduced our student teams to the joys (and potential sorrows) of the HPL (LINPACK) and accompany Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the field was normalized for boys in 1969 when the Apollo 11 missi Read more…

Apple Buys DarwinAI Deepening its AI Push According to Report

March 14, 2024

Apple has purchased Canadian AI startup DarwinAI according to a Bloomberg report today. Apparently the deal was done early this year but still hasn’t been publicly announced according to the report. Apple is preparing Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimization algorithms to iteratively refine their parameters until Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the fi Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimizat Read more…

PASQAL Issues Roadmap to 10,000 Qubits in 2026 and Fault Tolerance in 2028

March 13, 2024

Paris-based PASQAL, a developer of neutral atom-based quantum computers, yesterday issued a roadmap for delivering systems with 10,000 physical qubits in 2026 a Read more…

India Is an AI Powerhouse Waiting to Happen, but Challenges Await

March 12, 2024

The Indian government is pushing full speed ahead to make the country an attractive technology base, especially in the hot fields of AI and semiconductors, but Read more…

Charles Tahan Exits National Quantum Coordination Office

March 12, 2024

(March 1, 2024) My first official day at the White House Office of Science and Technology Policy (OSTP) was June 15, 2020, during the depths of the COVID-19 loc Read more…

AI Bias In the Spotlight On International Women’s Day

March 11, 2024

What impact does AI bias have on women and girls? What can people do to increase female participation in the AI field? These are some of the questions the tech Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Analyst Panel Says Take the Quantum Computing Plunge Now…

November 27, 2023

Should you start exploring quantum computing? Yes, said a panel of analysts convened at Tabor Communications HPC and AI on Wall Street conference earlier this y Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Leading Solution Providers

Contributors

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Training of 1-Trillion Parameter Scientific AI Begins

November 13, 2023

A US national lab has started training a massive AI brain that could ultimately become the must-have computing resource for scientific researchers. Argonne N Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire