What’s New in HPC Research: BaGuaLu, PolyCoder, Turbulent Flows & More

By Mariana Iriarte

March 29, 2022

In this regular feature, HPCwire highlights newly published research in the high-performance computing community and related domains. From parallel programming to exascale to quantum computing, the details are here.


Figure 5: (b): 2 structures and velocity contours suggest a dependence of the extent of the dead-zone on the Reynolds number.

Large eddy simulation of isothermal and non-isothermal turbulent flows in ventilated classrooms

A team of researchers from Argonne National Laboratory and the University of Illinois at Urbana-Champaign used the Oak Ridge National Laboratory’s Summit supercomputer to perform “large eddy simulations of a modified Nielsen test case, and a newly designed model classroom, with different arrangements of student units and ventilation pathways.” The effort was conducted to understand “the physics of isothermal and buoyant turbulent flows in enclosed ventilated spaces, and on the location and extent of dead zones.” In this paper, they were able to determine “the flow physics in the high Reynolds number jet region, resolve the regions of the flow near the solid wall surfaces, and simulate the effects of temperature gradients on the flow.”

Authors: Ramesh Balakrishnan, Rao Kotamarthi, and Paul Fischer

Hyperparameter optimization of data-driven AI models on HPC systems

As part of the European Center of Excellence in Exascale Computing “Research on AI- and Simulation-Based Engineering at Exascale” program, researchers from CERN, Switzerland, and from the National Institute of Chemical Physics and Biophysics, Estonia, test and evaluate advanced hyperparameter search algorithms, including the Random Search, Hyperband, and ASHA, for “accuracy and accuracy per compute resources spent.” They demonstrated that with the use of high performance computing resources,  hyperparameter optimization significantly increased the performance of machine-learned particle flow. In particular, “in the case of MLPF, the ASHA algorithm in combination with Bayesian optimization gives the largest performance increase per compute resources spent out of the investigated algorithm.” 

Authors: Eric Wulff, Maria Girone, and Joosep Pata

Figure 1: Existing language models of code, their sizes and availability ( open source vs.
not open-source ).

A systematic evaluation of large language models of code 

In this preprint paper, researchers from the School of Computer Science at Carnegie Mellon University present “a systematic evaluation of existing models of code – Codex, GPT-J, GPT-Neo, GPT-NeoX, and CodeParrot – across various programming languages.” In addition, they discovered the missing puzzle “piece in the form of a large open-source model trained exclusively on a multi-lingual corpus of code.” Thus releasing three new models, ranging from 160M to 2.7B parameters, dubbed PolyCoder, now available to the public in Github. In particular, PolyCoder “with 2.7B parameters based on the GPT-2 architecture, that was trained on 249GB of code across 12 programming languages on a single machine. In the C programming language, PolyCoder outperforms all models including Codex.” 

Authors: Frank F. Xu, Uri Alon, Graham Neubig, Vincent J. Hellendoorn

Distributed non-negative RESCAL with automatic model selection for Exascale data

A multi-institutional, international team introduced pyDRESCALk, a “distributed non-negative RESCAL algorithm for heterogeneous CPU/GPU architectures with automatic selection of the number of latent communities (model selection)” that is written in Python. In this paper, computer scientists demonstrated the results of  the scaling tests performed using the Grizzly and Kodiak supercomputers at Los Alamos National Laboratory, and showed “the algorithm’s scalability, and efficient node utilization on large-scale data.”  They demonstrated  “the correctness of pyDRESCALk with real-world and large synthetic tensors, and the efficacy showing near-linear scaling that concurs with the theoretical complexities. Finally, pyDRESCALk determines the number of latent communities in an 11-terabyte dense and 9-exabyte sparse synthetic tensor.” 

Authors: Manish Bhattarai, Namita Kharat, Erik Skau, Benjamin Nebgen, Hristo Djidjev, Sanjay Rajopadhye, James P. Smith, and Boian Alexandrov

Figure 1. Simplified computing process of the proposed
model.

BaGuaLu: targeting brain scale pretrained models with over 37 million Cores 

In this paper written by a team of international researchers from Tsinghua University, DAMO Academy, Alibaba Group, Zhejiang Lab, Beijing Academy of Artificial Intelligence, the authors present “BaGuaLu, the first work targeting training brain scale models on an entire exascale supercomputer, the New Generation Sunway Supercomputer.” In ancient Chinese mythology, BaGuaLu is known as a magic stove that creates effective medicine. Researchers found that by combining “hardware-specific intra-node optimization and hybrid parallel strategies, BaGuaLu enables decent performance and scalability on unprecedentedly large models.” Ultimately, they demonstrated: “BaGuaLu can train 14.5-trillion parameter models with a performance of over 1 EFLOPS using mixed-precision and has the capability to train 174- trillion-parameter models, which rivals the number of synapses in a human brain.”

Authors: Zixuan Ma , Jiaao He, Jiezhong Qiu, Huanqi Cao, Yuanwei Wang, Zhenbo Sun, Liyan Zheng, Haojie Wang, Shizhi Tang, Tianyu Zheng, Junyang Lin, Guanyu Feng, Zeqiang Huang, Jie Gao, Aohan Zeng, Jianwei Zhang, Runxin Zhong, Tianhui Shi, Sha Liu, Weimin Zheng, Jie Tang, Hongxia Yang, Xin Liu, Jidong Zhai, and Wenguang Chen

Design strategies and approximation methods for high-performance computing variability management

A group of researchers from Virginia Tech and Arizona State University focuses on input/output variability in a study that dives into performance variability management in high performance computing. In this case study, “comparisons are conducted for synthetic data analysis and HPC variability management application to explore the prediction accuracy of grid-based designs (GBDs) and, space-filling designs (SFDs) under different approximation methods.” They perform a “comprehensive investigation of design strategies and the prediction ability of approximation methods.” In addition, the researchers “use both synthetic data simulated from three test functions and the real data from the HPC setting.” The researchers showed that “overall the prediction error decreases as the design size increases. We find that no design outperforms all others uniformly. The Gaussian process (GP) with SFD, however, generates the best results under most scenarios”

Authors: Yueyao Wang, Li Xu, Yili Hong, Rong Pan, Tyler Chang, Thomas Lux, Jon Bernard, Layne Watson, and Kirk Cameron

The impact of hardware specifications on reaching quantum advantage in the fault tolerant regime

In this study, a team of international researchers at the University of Sussex and Universal Quantum, United Kingdom, and Qu & Co B.V., The Netherlands, “investigate how hardware specifications can impact the final run time and the required number of physical qubits to achieve a quantum advantage in the fault tolerant regime.” The team “utilize various space and time optimization strategies that have been previously considered within the field of error-correcting surface codes.” Specifically, they compare Game of Surface Code’s Units and AutoCCZ factories and “calculate the number of physical qubits required to break the 256-bit elliptic curve encryption of keys in the Bitcoin network within the small available time frame in which it would actually pose a threat to do so.” They were able to demonstrate that “it would require 317 x 106 physical qubits to break the encryption within one hour using the surface code, a code cycle time of 1 ls, a reaction time of 10 ls, and a physical gate error of 103.” To break the encryption in one day, it would take 13  x 106 physical qubits. 

Authors: Mark Webber, Vincent Elfving, Sebastian Weidt, and Winfried K. Hensinger


Do you know about research that should be included in next month’s list? If so, send us an email at [email protected]. We look forward to hearing from you.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket for an optional 8087 math coprocessor. The math coprocessor ma Read more…

IonQ Reports Advance on Path to Networked Quantum Computing

February 22, 2024

IonQ reported reaching a milestone in its efforts to use entangled photon-ion connectivity to scale its quantum computers. IonQ’s quantum computers are based on trapped ions which feature long coherence times and qubit Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Apple. Today the consumer electronics giant started rolling Read more…

GenAI Having Major Impact on Data Culture, Survey Says

February 21, 2024

While 2023 was the year of GenAI, the adoption rates for GenAI did not match expectations. Most organizations are continuing to invest in GenAI but are yet to derive any substantial value from it. However, the GenAI hyp Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performance Benchmarks – that builds on earlier work and is an eff Read more…

AWS Solution Channel

Shutterstock 2283618597

Deep-dive into Ansys Fluent performance on Ansys Gateway powered by AWS

Today, we’re going to deep-dive into the performance and associated cost of running computational fluid dynamics (CFD) simulations on AWS using Ansys Fluent through the Ansys Gateway powered by AWS (or just “Ansys Gateway” for the rest of this post). Read more…

Atom Computing Reports Advance in Scaling Up Neutral Atom Qubit Arrays

February 15, 2024

The scale-up challenge facing quantum computing (QC) is daunting and varied. It’s commonly held that 1 million qubits (or more) will be needed to deliver practical fault tolerant QC. It’s also a varied challenge beca Read more…

A Big Memory Nvidia GH200 Next to Your Desk: Closer Than You Think

February 22, 2024

Students of the microprocessor may recall that the original 8086/8088 processors did not have floating point units. The motherboard often had an extra socket fo Read more…

Apple Rolls out Post Quantum Security for iOS

February 21, 2024

Think implementing so-called Post Quantum Cryptography (PQC) isn't important because quantum computers able to decrypt current RSA codes don’t yet exist? Not Read more…

QED-C Issues New Quantum Benchmarking Paper

February 20, 2024

The Quantum Economic Development Consortium last week released a new paper on benchmarking – Quantum Algorithm Exploration using Application-Oriented Performa Read more…

The Pulse of HPC: Tracking 4.5 Million Heartbeats of 3D Coronary Flow

February 15, 2024

Working in Duke University's Randles Lab, Cyrus Tanade, a National Science Foundation graduate student fellow and Ph.D. candidate in biomedical engineering, is Read more…

It Doesn’t Get Much SWEETER: The Winter HPC Computing Festival in Corpus Christi

February 14, 2024

(Main Photo by Visit Corpus Christi CrowdRiff) Texas A&M University's High-Performance Research Computing (HPRC) team hosted the "SWEETER Winter Comput Read more…

Q-Roundup: Diraq’s War Chest, DARPA’s Bet on Topological Qubits, Citi/Classiq Explore Optimization, WEF’s Quantum Blueprint

February 13, 2024

Yesterday, Australian start-up Diraq added $15 million to its war chest (now $120 million) to build a fault tolerant computer based on quantum dots. Last week D Read more…

2024 Winter Classic: Razor Thin Margins in HPL/HPCG

February 12, 2024

The first task for the 11 teams in the 2024 Winter Classic student cluster competition was to run and optimize the LINPACK and HPCG benchmarks. As usual, the Read more…

2024 Winter Classic: We’re Back!

February 9, 2024

The fourth edition of the Winter Classic Invitational Student Cluster Competition is up and running. This year, we have 11 teams of eager students representin Read more…

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

Leading Solution Providers

Contributors

CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM

October 6, 1995

Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more…

SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING

October 6, 1995

Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more…

U.S. Will Relax Export Restrictions on Supercomputers

October 6, 1995

New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more…

Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996

October 6, 1995

Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more…

Cray Delivers J916 Compact Supercomputer to Solvay Chemical

October 6, 1995

Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more…

NEC Laboratory Reviews First Year of Cooperative Projects

October 6, 1995

Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more…

Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC

October 6, 1995

Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more…

New Study Says Parallel Processing Market Will Reach $14B in 1999

October 6, 1995

Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire