HPCwire: Berkeley Lab Scientists Build Software Framework for ATLAS Collaboration

Since 1986 - Covering the Fastest Computers
in the World and the People Who Run Them

Translation Disclaimer

Subscribe | Sign In

Visit additional Tabor Communication Publications

Leading HPC
Solution Providers

May 18, 2010

Berkeley Lab Scientists Build Software Framework for ATLAS Collaboration

May 17 -- Three thousand researchers in 37 countries are searching for the origins of mass, new dimensions of space and undiscovered forces of physics in the head-on collisions of high-energy protons at the Large Hadron Collider's ATLAS experiment. When ATLAS is turned-on, its detectors record about 400 collision events per second from a variety of perspectives, a rate equivalent to filling 27 compact disks per minute.

In order to sift out signs of new physics in this torrent of data, thousands of researchers must be able to process this information and collaborate on results in real time. To facilitate this distributed workflow, they are relying on a software framework called Athena, which was developed by an international team of scientists led by Paolo Calafiura of the Advanced Computing for Science Department in the Lawrence Berkeley National Laboratory's (Berkeley Lab) Computational Research Division.

"When developers plug their codes into the Athena framework, they get the most common functionality and communication among the different components of the experiment," says Calafiura, who is also the chief software architect for ATLAS.

He notes that the Athena software is essentially the "plumbing" for the international ATLAS collaboration. It allows scientists to focus on developing tools for analysis and actually analyzing data, instead of worrying about infrastructure issues like the compatibility of files. Researchers simply plug their codes into the framework, and Athena takes care of basic functions like coordinating the execution of applications and applying a common application-programming interface (API) so that collaborators can retrieve the same files.

"If you are a researcher that would like to computationally reconstruct the track left by high-energy muon particles while they traverse six different ATLAS detectors, you can use Athena StoreGate library to access the data coming from each detector and later to post the results of your reconstruction code for others to use," says Calafiura. "Once an object is posted to StoreGate, the library manages it according to preset policies and provides an API so that collaborators can easily share the data."

Athena is built on top of the Gaudi software framework that was originally developed for the LHCb experiment, which specifically looks at physics phenomena involving fundamental particles called b-mesons.

"When we began thinking about the software framework for ATLAS, we were very impressed by the Gaudi framework that was developed for LHCb. The architecture was very well designed, and it was simple to use. Rather than reinvent the wheel, we decided to work with what existed and plug in ATLAS-specific enhancements," says Calafiura. He notes that the Gaudi architecture is now a common kernel of software used by many experiments around the world and is co-developed by scientists mainly from ATLAS and LHCb.

Athena and Gaudi were developed in a close collaboration between computer scientists and physicists. Keith Jackson, Charles Leggett, and Wim Lavrijsen of the ACS department worked with Calafiura to develop the Athena framework. The team worked closely with David Quarrie, Mous Tatarkhanov and Yushu Yao, of the Berkeley Lab's Physics Division.

Berkeley Lab and LHC Computing

Beyond developing software for ATLAS, the Berkeley Lab will also contribute computing and network resources to CERN's LHC -- the world's largest particle accelerator. On March 30, the experiment achieved record-breaking seven trillion electron volt (7 TeV) proton collisions and opened a new realm of high-energy physics.

The LHC contains six major detector experiments, and experts predict that 15 petabytes of data will flow from the LHC per year for the next 10 to 15 years.That is enough information per year to create a tower of CDs more than twice as high as Mount Everest. Because CERN only has enough computational power to handle about 20 percent of this data, the workload is divided up and distributed to hundreds of universities and institutions around the globe via the LHC Computing Grid. The United States contributes 23 percent of the worldwide computing capacity for the ATLAS experiment and more than 30 percent of the computing power for another LHC experiment called CMS.

Raw data from all six LHC experiments are initially processed, archived and divided for distribution to 12 Tier-1 sites around the globe at CERN's onsite data center. In the US, the Fermi and Brookhaven National Laboratories are Tier-1 facilities that receive large portions of CMS and ATLAS data from CERN for another round of processing and archiving. This data is eventually carried to Tier-2 facilities, which primarily consist of universities and research institutions across the nation, for analysis. The Berkeley Lab's National Energy Research Scientific Computing Center (NERSC) is one of eight Tier-2 centers in the western US that will provide computing and storage resources for the LHC's ATLAS and ALICE experiments data analysis.

As each processing job completes, the sites will push data back up the tier system. In the United States, all data traveling between Tier-1 and Tier-2 DOE sites will be carried by the Energy Sciences Network (ESnet), which is managed by Berkeley Lab's Computational Research Division. ESnet is comprised of two networks -- an IP network that carries daily traffic to support lab operations, general science communication and science with relatively small data requirements, and a circuit-oriented Science Data Network (SDN) to transfer massive data sets like those from the LHC. Using the On-Demand Secure Circuit and Advanced Reservation System (OSCARS), researchers can reserve bandwidth on the SDN to guarantee that data is delivered to their collaborators with a certain time-frame.

For more information about Berkeley Lab contributions to LHC computing:

Global Reach: NERSC Helps Manage and Analyze LHC Data (PDF)
ESnet4 Helps Researchers Seeking the Origins of Matter

For more information about computing sciences at Berkeley Lab, visit www.lbl.gov/cs.

-----

Source: Lawrence Berkeley National Laboratory

Share Options

Subscribe

Subscribe to HPCwire

Discussion

There are 2 discussion items posted.

Submitted by bandmedia on May 2, 2011 @ 11:11 AM EDT

Printer Ink

Post #1

Submitted by bandmedia on May 2, 2011 @ 11:55 AM EDT

Lake District Accommodation

Post #2

Join the Discussion

Become a Registered User Today!

Registered Users Log in join the Discussion

RSS Feeds

Feeds by Topic

Feeds by Industry

Feeds by Content Type

Subscribe to All Content

All

May 23, 2012

May 22, 2012

May 21, 2012

May 18, 2012

May 17, 2012

May 16, 2012

May 15, 2012

May 14, 2012

May 11, 2012

Feature Articles

NVIDIA Works On CPU Co-Dependency Issues with Kepler GPU

NVIDIA is telling everyone that the GK110, its new Kepler GPU aimed at supercomputing, is all about improving performance per watt. But the other driving theme behind the new architecture is reducing the GPU's reliance on its CPU host. How well it accomplishes both these goals areas could determine the success of the new chip in high performance computing.
Read more...

OpenACC Starts to Gather Developer Mindshare

PGI, Cray, and CAPS enterprise are moving quickly to get their new OpenACC-supported compilers into the hands of GPGPU developers. At NVIDIA's GPU Technology Conference this week, there was plenty of discussion around the new HPC accelerator framework, and all three OpenACC compiler makers, as well as NVIDIA, were talking up the technology.
Read more...

NVIDIA Launches Kepler Into HPC

NVIDIA has introduced its first Kepler-generation GPU product for high performance computing, and revealed some of the inner working of the new architecture. The announcement took place at the kickoff of the company's GPU Technology Conference taking place this week in San Jose, California.
Read more...