July 5, 2017 — On June 5, coding “sprinters”—teams of computational, theoretical, and domain scientists; software developers; and graduate and postdoctoral students—took their marks at the U.S. Department of Energy’s (DOE) Brookhaven National Laboratory, beginning the first of five days of nonstop programming from early morning until night. During this coding marathon, or “hackathon,” they learned how to program their scientific applications on devices for accelerated computing called graphics processing units (GPUs). Guiding them toward the finish line were GPU programming experts from national labs, universities, and technology companies who donated their time to serve as mentors. The goal by the end of the week was for the teams new to GPU programming to leave with their applications running on GPUs—or at least with the knowledge of how to do so—and for the teams who had come with their applications already accelerated on GPUs to leave with an optimized version.

The era of GPU-accelerated computing 

GPU-accelerated computing—the combined use of GPUs and central processing units (CPUs)—is increasingly being used as a way to run applications much faster. Computationally intensive portions of an application are offloaded from the CPU, which consists of a few cores optimized for serial processing (tasks execute one at a time in sequential order), to the GPU, which contains thousands of smaller, more efficient cores optimized for parallel processing (multiple tasks are processed simultaneously).

However, while GPUs potentially offer a very high memory bandwidth (rate at which data can be stored in and read from memory by a processor) and arithmetic performance for a wide range of applications, they are currently difficult to program. One of the challenges is that developers cannot simply take the existing code that runs on a CPU and have it automatically run on a GPU; they need to rewrite or adapt portions of the code. Another challenge is efficiently getting data onto the GPUs in the first place, as data transfer between the CPU and GPU can be quite slow. Though parallel programming standards such as OpenACC and GPU advances such as hardware and software for managing data transfer make these processes easier, GPU-accelerated computing is still a relatively new concept.

A hackathon with a history

Here’s where “Brookathon,” hosted by Brookhaven Lab’s Computational Science Initiative(CSI) and jointly organized with DOE’s Oak Ridge National Laboratory, Stony Brook University, and the University of Delaware, came in.

“The architecture of GPUs, which were originally designed to display graphics in video games, is quite different from that of CPUs,” said CSI computational scientist Meifeng Lin, who coordinated Brookathon with the help of an organizing committee and was a member of one of the teams participating in the event. “People are not used to programming GPUs as much as CPUs. The goal of hackathons like Brookathon is to lessen the learning curve, enabling the use of GPUs on next-generation high-performance-computing (HPC) systems for scientific applications.”

Brookathon is the latest in a series of GPU hackathons that first began in 2014 at Oak Ridge Leadership Computing Facility (OLCF)—a DOE Office of Science User Facility that is home to the nation’s most powerful science supercomputer, Titan, and other hybrid CPU-GPU systems. So far, OLCF’s Fernanda Foertter, a HPC user support specialist and programmer, has helped organize and host 10 hackathons across the United States and abroad, including Brookathon and one at the Jülich Supercomputing Centre in Germany earlier this year.

“Hackathons are intense team-based training events,” said Foertter. “The hope is that the teams go home and continue to work on their codes.”

The idea to host at Brookhaven started in May 2016, when Lin and Brookhaven colleagues attended their first GPU hackathon, hosted at the University of Delaware. There, they worked on a code for lattice quantum chromodynamics (QCD) simulations, which help physicists understand the interactions between particles called quarks and gluons. But in using the OpenACC programming standard, they realized it did not sufficiently support the C++ programming language that their code library was written in. Around this time, Brookhaven became a member of OpenACC so that CSI scientists could help shape the standard to include the features needed to support their codes on GPUs. Through the University of Delaware hackathon and weekly calls with OpenACC members, Lin came into contact with Foertter and Sunita Chandrasekaran, an assistant professor of computer science at the University of Delaware who organized that hackathon, both of whom were on board with bringing a hackathon to Brookhaven.

“Brookhaven had just gotten a computing cluster with GPUs, so the timing was great,” said Lin. “In CSI’s Computational Science Laboratory, where I work, we get a lot of requests from scientists around Brookhaven to get their codes to run on GPUs. Hackathons provide the intense hands-on mentoring that helps to make this happen.”

Teams from near and far

A total of 22 applications were submitted for a spot at Brookathon, half of which came from Brookhaven Lab or nearby Stony Brook University teams. According to Lin, Brookathon received the highest number of applications of any of the hackathons to date. Ultimately, a review committee of OpenACC members accepted applications from 10 teams, each of which brought a different application to accelerate on GPUs:

• Team AstroGPU from Stony Brook University: codes for simulating astrophysical fluid flows
• Team Grid Makers from Brookhaven, Fermilab, Boston University, and the University of Utah (Lin’s team): a multigrid solver for linear equations and a general data-parallel library (called Grid), both related to application development for lattice QCD under DOE’s Exascale Computing Project
• Team HackDpotato from Stony Brook University: a genetic algorithm for protein simulation
• Team Lightning Speed OCT (for optical coherence tomography) from Lehigh University: a program for real-time image processing and three-dimensional image display of biological tissues
• Team MUSIC (for MUScl for Ion Collision) from Brookhaven and Stony Brook University: a code for simulating the evolution of the quark-gluon plasma produced at Brookhaven’s Relativistic Heavy Ion Collider (RHIC)—a DOE Office of Science User Facility
• Team NEK/CEED from DOE’s Argonne National Laboratory, the University of Minnesota, and the University of Illinois Urbana-Champaign: fluid dynamics and electromagnetic codes (Nek5000 and NekCEM, respectively) for modeling small modular reactors (SMR) and graphene-based surface materials—related to two DOE Exascale Computing Projects, Center for Efficient Exascale Discretizations (CEED) and ExaSMR
• Team Stars from the STAR from Brookhaven, Central China Normal University, and Shanghai Institute of Applied Physics: an online cluster-finding algorithm for the energy-deposition clusters measured at Brookhaven’s Solenoidal Tracker at RHIC (STAR) detector, which searches for signatures of the quark-gluon plasma
• Team The Fastest Trigger of the East from the UK’s Rutherford Appleton Laboratory, Lancaster University, and Queen Mary University of London: software that reads out data in real time from 40,000 photosensors that collect light generated by neutrino particles, discards the useless majority of the data, and sends the useful bits to be written to disk for future analysis; the software will be used in a particle physics experiment in Japan (Hyper-Kamiokande)
• Team UD-AccSequencer from the University of Delaware: a code for an existing next-generation-sequencing tool for aligning thousands of DNA sequences (BarraCUDA)
• Team Uduh from the University of Delaware and the University of Houston: a code for molecular dynamics simulations, which scientists use to study the interactions between molecules

“The domain scientists—not necessarily computer science programmers—who come together for five days to migrate their scientific codes to GPUs are very excited to be here,” said Chandrasekaran. “From running into compiler and runtime errors during programming and reaching out to compiler developers for help to participating in daily scrum sessions to provide progress updates, the teams really have a hands-on experience in which they can accomplish a lot in a short amount of time.”

Read the full story at: https://www.bnl.gov/newsroom/news.php?a=212273

---

Source: BNL