At the Stephen Hawking Centre for Theoretical Cosmology (CTC) in Cambridge University, we are endeavouring to push back our understanding to the very beginnings of the Universe, tiny fractions of a second after the Big Bang. Cosmology has had many successes but it leaves many questions unanswered concerning how the Universe came into being and about the matter and energy that fills it. New high precision observations of the Universe offer us important clues because they allow us to see the underlying causes of the formation of cosmic structures, including galaxies, stars, planets, and ultimately ourselves. Such questions about the beginning of the Universe have always fascinated humanity and, from a scientific perspective, the Big Bang represents the ultimate frontier by probing the very highest energies possible. Cosmological experiments also advance other frontiers using the latest state-of-the-art technologies, and their analysis is extremely challenging computationally.

At CTC we develop cosmological theories and test them against observed data using COSMOS, the largest single-image shared-memory supercomputer in Europe. COSMOS is part of the national UK DiRAC HPC Facility. It has become a vital tool for a wide group of UK researchers at more than 12 different Universities working on a variety of projects, including the search for extra-terrestrial life.

Our most dramatic successes in cosmology have come from the cosmic microwave background (CMB) radiation—the relic radiation left over from the Big Bang. Observing the CMB is like looking at a snapshot of the early universe. The Planck satellite, launched in 2009, offers the best CMB maps of the Universe, and in 2013 yielded the highest precision measurements of the cosmic parameters to date. Analysing the Planck data to test our theories is a massive task that can only be done using supercomputers, like COSMOS.

Recent analysis of CMB observations confirm predictions that a period of enormously fast exponential expansion, which cosmologists call inflation, occurred in the early universe. During inflation, very small changes, or quantum fluctuations, were imprinted into the fabric of space-time. These later became the seeds for the development of all the structures we now see in the universe. Establishing the fundamental character of these fluctuations would offer vital clues about how the universe emerged out of inflation, one of the most important goals in fundamental science. So, supercomputers like COSMOS are critical to our understanding of the earliest times in the universe.

Many supercomputers are clusters of smaller compute systems networked together, but COSMOS is a single system, like a gigantic parallel-computing PC. The distinction is important when it comes to our work. A large part of what we do is software code development. COSMOS’ flexible shared-memory architecture is ideal for this purpose. It allows our researchers to focus on innovative codes first and develop efficient parallelism in their software while proving their theories. They can go from working on their laptop to COSMOS much more easily than programming for a large, distributed system, where the parallelism of the code becomes much more critical to get their applications to work.

Xeon Phi coprocessors provide 30 X performance boost

We use both proven production codes—many of them developed on COSMOS—and new codes which are being tested and written by consortium researchers. We are porting heavily utilized production codes, such as the WALLS code, onto the Intel Xeon Phi coprocessors added to COSMOS in 2012. This porting effort onto Intel coprocessors has sped up our results considerably—it now allows us to run the WALLS code alone 30 times faster—which means discoveries can happen sooner, and we can handle bigger problems. Intel support in this area is vital to our continuing research on COSMOS.

COSMOS has proved essential for our work with the CMB, particularly the Planck satellite maps of the entire sky. The magnitude of data is manageable, but the computational effort required to extract scientific information from it is formidable. For example, when we are looking at the statistics described by the three-point correlation function, we need to add up all the contributions from all the possible triangles we can draw within the 10 million pixels in a single Planck map. Naively, this is 10^21 sets of complex operations! These calculations need to be simulated and repeated many times to test and eliminate systematic experimental effects, so brute force methods are not possible. Just to calculate the three-point correlator data required 3 million core-hours. On a single core that would take over 300 years. We needed to be able to solve it within 300 days.

Using COSMOS, we have been able to rapidly develop and implement methods to calculate the complete three-point correlator for the first time, up to a given resolution. This is a statistic that looks at “triangles in the sky”, testing whether there is a connection between sets of three different points in the Universe. Porting the computational “hot spots” in our Planck pipeline to the Intel Xeon Phi coprocessors has greatly shortened turnaround times, which will ensure we substantially improve resolution in the future. We could not begin to analyse the Planck maps, let alone previous generations of experiments, without COSMOS.

Another major project being undertaken on COSMOS is the study of spectroscopic signatures of particular molecules in exoplanet atmospheres (the EXOMOL project). These signatures can help in the identification of exoplanets and whether or not there is extra-terrestrial life there.

With COSMOS and the Intel coprocessors, researchers and programmers are continually working on optimizing our vital codes for Intel Xeon Phi, while developing new ones. Our expectation is that all our cosmological field theory codes, like WALLS, will have similarly large speed-ups when optimized and ported to Xeon Phi. We are currently transferring a larger proportion of our CMB analysis to the coprocessors, work that will continue over the next 12 months, and we are excited by the prospect of pushing forward joint two-point and three-point correlator analysis for the first time.