HPCwire: Fixing the Holes

The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing

Leading HPC
Solution Providers

HPCwire >> Industry >> Academia & Research

September 07, 2007

Fixing the Holes

by Tim Palucka

By including black holes for the first time in a large-scale cosmological simulation, physicists uncover their function in regulating the growth of galaxies

Once thought to be rare, exotic bodies, black holes have turned out to be fundamental focal points in the architecture of the cosmos. Among the sprawling, intricate arrangement of matter in every galaxy, they are the invisible centerpiece. Their tremendous mass, which can be many billion times the mass of the Sun, gives them gravitational pull to swallow huge quantities of interstellar gas. As this gas swirls outside the lip of the drain, just before falling in, it heats to extreme temperatures and radiates energy as light waves, producing one of the brightest bulbs in the universe -- a quasar. Once over the edge of this lip, however, nothing escapes, not even light.

Despite their ubiquitous presence, black holes and the quasars they spawn have been until now absent from large-scale simulations of the universe -- too small to be resolved within the big picture of cosmic structure. Thanks to TeraGrid resources, however, in particular PSC's Cray XT3, Carnegie Mellon University theoretical cosmologist Tiziana Di Matteo and her colleagues were able to include black holes in a simulation encompassing a sizeable fraction of the universe.

"Ours is the first cosmological simulation to incorporate black-hole physics," says Di Matteo. "It involves more calculations than any prior similar modeling of the cosmos. The galaxies we see today look the way they do because of black holes. This result offers the best picture to date of how the cosmos formed and evolved."

Their findings, forthcoming in the Astrophysical Journal, report on evolution of structure over the 14 billion years of the universe's existence. Far from being only destroyers, gobbling up any matter within reach, black holes in this new picture are also regulators: Their mass is related to the size of the galaxy they reside in as well as to its total star mass.

The goal is a more fundamental understanding of the evolution of the universe. "What kind of quasars formed in what kind of galaxies at what time?" asks Di Matteo. "What is the progenitor of the most massive black hole today?" There is also a more immediate benefit -- directing astronomers where to aim the Next Generation Space Telescope -- the successor to the Hubble Space Telescope -- to observe the formation of the first galaxies and black holes.

A New Cosmic Recipe

For this huge simulation, Di Matteo started with the software called GADGET-2 developed by Volker Springel of the Max Planck Institute for Astrophysics. To account for black holes, Di Matteo added code to "seed" black holes at the centers of forming galaxies. Next, she added an equation to describe how black holes "accrete" or swallow gas, adding to their mass and gravitational pull. Finally, she included calculations for "feedback" -- heating of surrounding gas in the galaxy by quasar radiation produced at the lip of the black hole.

The researchers first applied this approach on a small system of two colliding galaxies with black holes at their centers. The success of this simulation, which revealed new behavior when black holes were included, led to a 2005 paper in Nature. It also prompted Di Matteo to move to a much bigger scale. The idea, says Di Matteo, was to simulate a large portion of the universe "at the same resolution and with the same spatial scale as those idealized calculations."

To obtain this high resolution, the researchers scattered 230 million hydrodynamic particles over a 33 megaparsec cube, a huge volume encompassing a million galaxies, a representative chunk of the universe. To track these particles, the simulation used 2,000 XT3 processors -- the whole system -- over four weeks of run time. They followed the evolution of superclusters of galaxies -- the largest structures in the universe -- while simultaneously resolving the growth of black holes at the centers of galaxies. The XT3 was key.

"The XT3 is ideal for this simulation because it has incredibly fast built-in communication," says Di Matteo. "If we didn't have the bandwidth to communicate large chunks of data among 2,000 processors, it would have been really tough. I don't think we could run this simulation anywhere else right now."

Repentant Overeaters

Analyzing the huge quantity of data from this simulation, about ten trillion bytes, was exciting. The major finding is the regulatory role black holes play in galaxies. It turns out that after gobbling gas for millennia, a black hole eventually begins pushing gas away, like a repentant overeater. "The black hole tries to swallow a lot of gas," says Di Matteo, "but this growth is kind of suicidal. As it tries to swallow more and more it will radiate so much energy that it will affect its surroundings and stop more gas from flowing in."

This first effect of this purging "feedback" process is that it shuts off the quasar associated with the black hole. Because gas is no longer falling in, gas at the lip of the black hole is no longer heated and eventually radiation stops. In effect, feedback blows out the candle of the quasar, which explains why - though virtually every galaxy had a quasar when the universe was young - now there are quasars in only about one in 10,000 galaxies.

While blowing out the quasar, the black hole also expels most of the galaxy's interstellar gas into inter-galactic space. As a result, since interstellar gas is the stuff that stars are made of, stars stop forming. This helps to explain the "red and dead" phenomenon with which astronomers have long struggled: the preponderance of old galaxies in which no new stars are forming. Old galaxies tend to appear red due to the overwhelming dominance of old stars. Stars should form readily if there is interstellar gas to feed the process, yielding young blue stars. Di Matteo's simulation shows that there is very little interstellar gas left, leaving old galaxies "red and dead."

Another conundrum was the common ratio found between the mass of the central black hole and the mass of its host galaxy. Precise measurement using the Hubble Space Telescope and other instruments has shown that the black hole has a mass approximately 1/1000th the mass of the stars in its galaxy. This same ratio emerged from Di Matteo's simulation, which helps to validate the model, but the reason for the ratio is not yet evident. Some mechanism related to a galaxy's critical mass might cause the black hole to stop swallowing when it does, but that mechanism remains a mystery. "Black holes clearly regulate how the galaxy forms in some way that is still debated," says Di Matteo, "and that is what we are still trying to understand."

Perhaps one of the most exciting end products of this huge calculation is a movie showing how the universe evolved over 14 billion years. The simulation begins before galaxies have formed, and the first frames (each frame is about half a million years) show a virtually uniform universe, with matter dispersed rather evenly, and just small perturbations in this background -- corresponding well with the picture provided by cosmic microwave background studies. The first black holes appear when the universe is 300 million years old -- just a child.

As the movie proceeds, matter in the universe clumps together in a filamentary fashion. "Empty regions become emptier and emptier," says Di Matteo. Dense regions become denser, and the universe starts to look like a spider web. By the end, supermassive black holes are lurking in the center of most galaxies, quasars shine bright and then blow out, and galaxies change colors from blue to red as they age.

Because of the extremely high resolution, Di Matteo can zoom in at any time and watch the growth of a particular black hole. "This is the most massive black hole that forms at an early time," she says, pointing to a specific black hole on a movie frame, "but it does not end up being the largest black hole today. It grows really fast then it stops. Other black holes start to grow later on and they grow at a faster rate at a much later stage, and so that's very exciting too."

Di Matteo and her colleagues don't know what other fascinating results await them in the huge mass of data they have begun to analyze. Perhaps they will learn where to point telescopes to see the remnants of the first quasars, or maybe they will see astronomical phenomena never dreamed of before. "The hugest challenge was to do this very large simulation," says Di Matteo. Analyzing the data will be a pleasure. "Both in terms of the resolution and the physics," she says, smiling, "nobody else has a simulation like this right now."

-----

Source: Reprinted with permission of Pittsburgh Supercomputing Center. For more information and graphics, visit http://www.psc.edu/science/2006/blackhole.html.

Article Tools

Share & Save Options

Discussion

There are 0 discussion items posted.

Join the Discussion

You must be a registered user in order to comment on HPCwire stories.

Members enjoy:

	Access to Special Content, including Podcasts, Webcasts, and Whitepapers and more!
	Ability to take an active role in the discussions that are shaping the HPC technology discipline!
	Special offers available only to HPCwire members!

Become a Registered User Today!

Registered Users Log in Here to Comment

Email Address:	Password (case sensitive)

Remember me	Forgot Password?

RSS Feeds

Feeds by Topic

Feeds by Industry

Feeds by Content Type

Feature Articles

The New Science of Visual Analytics

When Jim Thomas set out to find new ways to deal with the mountains of information our society generates, he didn't just create a new organization, he created a new science. In this article we'll take a look at how the National Visualization and Analytics Center is transforming the problem of finding needles in haystacks into an opportunity for a more secure future.
Read More...

The Week in Review

ORNL Jaguar doubles its performance; the SC08 Cluster Challenge is gearing up; the University of Central Florida uses Army dollars to purchase an IBM super; and IBM's RoadRunner prepares to break the petaflop barrier. John West recaps those stories and more in our weekly wrap-up.
Read More...

Revaluating FPGAs for 64-bit Floating-Point Calculations

We now have generally available 2.5 GHz quad-core Opterons and Virtex-5 LX330, SX95T and recently announced SX240T FPGAs. In addition to this, Xilinx is releasing a new version of their floating-point cores that reduces the amount of logic and DSP slices needed for building floating-point function units. Taken together it is time to revisit Opteron floating-point performance versus FPGA performance.
Read More...

Read more HPCwire features...

Top Headlines

Parallel Processing Calls for a Fortress Mentality

May 14 | InfoWorld | Sun Microsystems is taking the lessons learned from Java and applying them to the application development challenges of the high performance computing realm. Read more...

IBM Set to Test the Fastest Computer in the World

May 14 | Computerworld | IBM is assembling the final pieces of what they hope will soon become the world's most powerful supercomputer. Read more...

IBM Shifts Cell to 65 Nanometers

May 13 | EETimes | IBM Corp. has announced the next-generation version of its Cell processor, the first specifically geared for computer servers. Read more...

Record-Setting Simulations on Ranger Reconstruct the Reionization Era

May 12 | Texas Advanced Computing Center | In the coming months, Dr. Michael L. Norman of UCSD will use Ranger, the world's most powerful supercomputer for open-science research, to perform the largest cosmological simulation to date. Read more...

2 AMD Executives Out in Restructuring Amid Slump

May 12 | BusinessWeek | Two executives have left AMD, including the head of the slumping chip maker's microprocessor division, as the company tries to engineer a dramatic turnaround to fend off larger rival Intel Corp. Read more...

Read more headlines...

Multimedia

Podcast: Interview with Ben Bennett of ClearSpeed Technology

Today, HPC organizations are requiring substantially more floating point performance to solve real-world problems. In this podcast, Ben Bennett, ClearSpeed General Manager, discusses how acceleration technology can improve the overall performance of standard x86-based systems...