TeraGrid ’09: OSG and TeraGrid Collaboration

By Elizabeth Leake

July 6, 2009

Paul AveryPaul Avery, a recognized leader in advanced grid and networking for science, delivered the first keynote address at the recent TeraGrid ’09 conference in Arlington, Va. A professor of physics at the University of Florida, Avery is co-principal investigator and founding member of the Open Science Grid (OSG). Avery talked about the history of OSG, some of the projects that leverage its resources, and OSG’s relationship with TeraGrid.

“The OSG is a multicultural consortium — half technology and half sociology,” said Avery.

While the majority of OSG users are from American institutions, 2,500 from 60 countries accessed OSG resources in the past year. From the current rate of usage, it is estimated that researchers will have used tens of petabytes of storage by 2012 and one hundred by 2013. By engaging with the Large Hadron Collider (LHC) at the European Organization for Nuclear Research, also known as CERN, researchers could need an exabyte of storage by the end of the next decade. Currently there are 5,000 physicists from hundreds of institutes who conduct research at CERN.

The OSG consortium includes science and research communities, national labs, international infrastructures, multidisciplinary (virtual) organizations (VO), technologists, HPC professionals, LHC/CERN experiments, regional and campus infrastructures, education communities as well as computer science and DOE-funded SciDAC projects.

OSG has a distributed facility of 85 resources with approximately 50,000 CPU cores of processing power and about 10 petabytes of disk storage. Its milestones and deliverables are a reflection of its science-driven charter. The leadership of OSG hails jointly from American national laboratory and university involvement where most science is conducted and collaborations are formed. OSG’s leadership is collaboratively focused with international and multidisciplinary interactions. The major stakeholders include the ATLAS and CMS experiments at the LHC and the Laser Interferometer Gravitational Wave Observatory (LIGO).

Jointly funded and overseen by the National Science Foundation (NSF) and the US Department of Energy (DOE), the 2006-2011 OSG project was funded with $30M — split roughly equally between NSF and DOE. There are 35 full time employees who are directly funded by the OSG project while many more are leveraged via OSG’s relationship with consortium members.

Avery offered a brief history of how OSG formed, starting about a decade ago with several NSF- and DOE-funded grid projects. The projects joined forces and in a bottom-up process began deploying test beds and a prototype grid that grew in size and complexity as resources were added from universities, campus grids and national laboratories. At the same time strong links were forged with CERN and the European grid projects to jointly form the Worldwide LHC Computing Grid (WLCG). He spoke of OSG’s developing partnerships with regional grids such as New York State Grid and SuraGrid and its collaborations with optical network organizations that provide critical links for high speed data movement.

OSG’s campus engagement model was developed by the Renaissance Computing Institute (RENCI) in North Carolina. By working with campus Cyberinfrastructure (CI) Days and through academic outreach organizations like EDUCAUSE, OSG is communicating its value to colleges and universities. “It seems to be working. Usage stats have doubled since 2008,” said Avery.

“Collaboration between TeraGrid and OSG is an integral component to achieving the nation’s vision for CI,” said Avery as he began to talk about the future of OSG. This vision was articulated in a “dear colleague” letter from the NSF Office of Cyberinfrastructure on June 9, 2009.

“As an overarching theme, OCI will promote the development of collaborative computational science — defined broadly to encompass research and development of comprehensive CI in all areas described below, as well as the application of CI to solve complex problems in science and engineering — as one of OCI’s primary missions. OCI will work to provide stewardship for computational science at NSF, in strong collaborations with other offices, directorates, and agencies.”

Explosive growth in processing and storage reflects the increasing sophistication and demands of 21st century research and engineering enterprises. However, more effective middleware and management tools are required for applications to scale so that they can efficiently use rapidly growing OSG and TeraGrid resources. OSG is moving to a bridging model to connect heterogeneous resources to one another and to the communities that utilize them in new and interesting ways. These include large digital libraries and public databases, computing clouds, massive distributed datasets, GPUs, visualization, instrument steering, collaborative environments, etc.

Avery noted that a closer OSG and TeraGrid relationship builds on several existing joint activities and substantial overlap in resources and services. Common interests such as communication, campus engagement, training, student workforce development, software licensing, security, science gateways, cloud development, virtualization and administrative tools offer additional opportunities for working together. Another mutually-beneficial goal is the development of a common software stack.

Many are aware of the fundamental differences between OSG and TeraGrid as they originated from distinctly different cultures. TeraGrid sprang from the NSF-funded PACI partnerships, while OSG grew out of a collaborative physics environment. OSG currently doesn’t have an allocation process, though future demand may cause it to examine TeraGrid’s mechanism. Both TeraGrid and OSG recognize the importance of monitoring trends to help them develop systems that will adapt to changing conditions and persist over time.

Even though OSG and TeraGrid have both contributed to the national CI which serves multiple communities, challenges remain in accommodating the expected growth and heterogeneity of the “third leg of science.” According to Avery, “A long-term OSG and TeraGrid partnership offers outstanding opportunities to advance a coherent national CI that is persistent over decades, is international in outlook, encourages federal agencies to adopt consistent strategies, and trains the workforce that will utilize it and continue its development.”

Avery’s presentation is available at http://www.teragrid.org/tg09/files/avery_teragrid.pdf.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Nvidia Debuts Turing Architecture, Focusing on Real-Time Ray Tracing

August 16, 2018

From the SIGGRAPH professional graphics conference in Vancouver this week, Nvidia CEO Jensen Huang unveiled Turing, the company's next-gen GPU platform that introduces new RT Cores to accelerate ray tracing and new Tenso Read more…

By Tiffany Trader

HPC Coding: The Power of L(o)osing Control

August 16, 2018

Exascale roadmaps, exascale projects and exascale lobbyists ask, on-again-off-again, for a fundamental rewrite of major code building blocks. Otherwise, so they claim, codes will not scale up. Naturally, some exascale pr Read more…

By Tobias Weinzierl

STAQ(ing) the Quantum Computing Deck

August 16, 2018

Quantum computers – at least for now – remain noisy. That’s another way of saying unreliable and in diverse ways that often depend on the specific quantum technology used. One idea is to mitigate noisiness and perh Read more…

By John Russell

HPE Extreme Performance Solutions

Introducing the First Integrated System Management Software for HPC Clusters from HPE

How do you manage your complex, growing cluster environments? Answer that big challenge with the new HPC cluster management solution: HPE Performance Cluster Manager. Read more…

IBM Accelerated Insights

Super Problem Solving

You might think that tackling the world’s toughest problems is a job only for superheroes, but at special places such as the Oak Ridge National Laboratory, supercomputers are the real heroes. Read more…

NREL ‘Eagle’ Supercomputer to Advance Energy Tech R&D

August 14, 2018

The U.S. Department of Energy (DOE) National Renewable Energy Laboratory (NREL) has contracted with Hewlett Packard Enterprise (HPE) for a new 8-petaflops (peak) supercomputer that will be used to advance early-stage R&a Read more…

By Tiffany Trader

STAQ(ing) the Quantum Computing Deck

August 16, 2018

Quantum computers – at least for now – remain noisy. That’s another way of saying unreliable and in diverse ways that often depend on the specific quantum Read more…

By John Russell

NREL ‘Eagle’ Supercomputer to Advance Energy Tech R&D

August 14, 2018

The U.S. Department of Energy (DOE) National Renewable Energy Laboratory (NREL) has contracted with Hewlett Packard Enterprise (HPE) for a new 8-petaflops (peak Read more…

By Tiffany Trader

CERN Project Sees Orders-of-Magnitude Speedup with AI Approach

August 14, 2018

An award-winning effort at CERN has demonstrated potential to significantly change how the physics based modeling and simulation communities view machine learni Read more…

By Rob Farber

Intel Announces Cooper Lake, Advances AI Strategy

August 9, 2018

Intel's chief datacenter exec Navin Shenoy kicked off the company's Data-Centric Innovation Summit Wednesday, the day-long program devoted to Intel's datacenter Read more…

By Tiffany Trader

SLATE Update: Making Math Libraries Exascale-ready

August 9, 2018

Practically-speaking, achieving exascale computing requires enabling HPC software to effectively use accelerators – mostly GPUs at present – and that remain Read more…

By John Russell

Summertime in Washington: Some Unexpected Advanced Computing News

August 8, 2018

Summertime in Washington DC is known for its heat and humidity. That is why most people get away to either the mountains or the seashore and things slow down. H Read more…

By Alex R. Larzelere

NSF Invests $15 Million in Quantum STAQ

August 7, 2018

Quantum computing development is in full ascent as global backers aim to transcend the limitations of classical computing by leveraging the magical-seeming prop Read more…

By Tiffany Trader

By the Numbers: Cray Would Like Exascale to Be the Icing on the Cake

August 1, 2018

On its earnings call held for investors yesterday, Cray gave an accounting for its latest quarterly financials, offered future guidance and provided an update o Read more…

By Tiffany Trader

Leading Solution Providers

SC17 Booth Video Tours Playlist

Altair @ SC17


AMD @ SC17


ASRock Rack @ SC17

ASRock Rack



DDN Storage @ SC17

DDN Storage

Huawei @ SC17


IBM @ SC17


IBM Power Systems @ SC17

IBM Power Systems

Intel @ SC17


Lenovo @ SC17


Mellanox Technologies @ SC17

Mellanox Technologies

Microsoft @ SC17


Penguin Computing @ SC17

Penguin Computing

Pure Storage @ SC17

Pure Storage

Supericro @ SC17


Tyan @ SC17


Univa @ SC17


  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This