Dec. 7 — The historic discovery of gravitational waves earlier this year has sent astronomers on the search for more. These scientists are turning to the Blue Waters supercomputer at NCSA in aid of their quest.

So it was fitting that the NANOGrav annual fall meeting be held at NCSA, allowing researchers to see and work with Blue Waters first hand. As part of their meeting, Eliu Huerta, chair of the local organizing committee of this event and leader of NCSA’s Gravity group, organized a hackathon with the goal of using the GPUs in Blue Waters to accelerate the pipelines NANOGrav uses to search for gravitational waves. Huerta obtained the needed computational resources for this event through a Blue Waters education allocation.

The end results: the two main bottlenecks in analysis were successfully removed and, for the first time, ipython notebooks ran on Blue Waters’ compute nodes in a stable configuration. In addition, the hackathon wetted the interest of Illinois participants, who expressed a keen interest in participating in future NANOGrav hackathons, and a desire to participate in searches of gravitational waves with pulsar timing arrays using Blue Waters.

The pipelines used for the hackathon were developed by Justin Ellis and Steve Taylor, postdoctoral researchers at the Jet Propulsion Laboratory (JPL). The duo prepared ipython notebooks to isolate the main bottlenecks in the analysis. By using ipython notebooks code developers could interactively quantify whether the use of new libraries or new pieces of code provided a significant speed up in the analysis.

Blue Waters team members Timothy Bouvet, Roland Haas, Sharif Islam, and Colin MacLean joined the hackers, contributing their expertise to provide access to the system and the stable ipython notebook platform needed to optimize NANOGrav’s pipelines.

A python library was implemented in the pipelines to take advantage of the properties of sparse matrices, which are common in pulsar timing analysis. This improvement accelerated the analysis by at least a factor of two. Another linear algebra bottleneck was related to the computation of matrix multiplications. This process is repeated according to the number of pulsars used in the analysis—currently about 50 times. openMP was implemented to further accelerate this analysis.

NANOGravers and undergraduates, graduates, and postdocs from the University of Illinois at Urbana-Champaign were instructors and code developers. They included instructors: Adam Brazier (Cornell), Justin Ellis (JPL), Nate Garver-Daniels (WVU), Roland Haas (NCSA), Eliu Huerta (NCSA), Steve Taylor (JPL), and code developers: Arian Azin (Illinois), Paul Baker (WVU), Patrick Dean Mullen (Illinois), Mark Dewing (Illinois), Daniel George (Dept. of Astronomy at Illinois and NCSA), Miguel Holgado, Michael Katolik, and Wei-Ting Liao, all with the Dept. of Astronomy at Illinois, and Kedar Phadke (Illinois).

---

Source: NCSA