May 13, 2022 — Scientific experimentation frequently generates enormous quantities of data. And, just as frequently, this data cannot be handled at the point where it is created and has to be moved to either large supercomputing clusters or the cloud. This process of moving data requires high-tech software packages that can transport large quantities of information seamlessly, reliably and quickly, all over the world.

Recently, a data moving competition was held at the SupercomputingAsia conference to test how well different data transfer applications moved data. The competition brought together experts from industry and academia to evaluate innovative solutions for transferring enormous quantities of data to servers around the world. Globus, a platform for research data management created by researchers at the University of Chicago and the U.S. Department of Energy’s (DOE) Argonne National Laboratory, received the Best Integrated Software Experience Award at the 2021 Data Mover Challenge.

“With state-of-the-art experimental facilities like Argonne’s Advanced Photon Source, which is a DOE Office of Science user facility, or CERN’s Large Hadron Collider, researchers need ways to quickly and accurately move data around,” said Argonne computer scientist Rajkumar Kettimuthu. ​“In many cases, it is important to be able to analyze data quickly after it is generated, and being able to move data plays an important role in that process.”

The Data Mover Competition formed an important stage to showcase Globus’s accomplishments, Kettimuthu explained. ​“Opportunities like this, even though they only last for a few days, provide a worthwhile testbed,” he said. ​“Although we aren’t able to test out all the features of Globus during the competition, it still allowed us to demonstrate how versatile a tool it is.”

“I am honored that Globus was selected among this group of competitors,” added Ian Foster, Globus co-inventor and director of Argonne’s Data Science and Learning division. ​“In today’s world the ability to rapidly and easily transfer large amounts of data, and to share it with partners and collaborators, is essential to progress in any research and business endeavor.”

According to Kettimuthu, Globus received the Best Integrated Software Experience Award because it is intuitive and user-friendly. ​“We hide a bunch of the complexities from the users,” he said. ​“The package automatically responds to failures and tunes for performance so that users don’t have to babysit their data transfers.”

Moving data between supercomputing clusters and the cloud can create challenges in data transfer, but it is an aspect that Globus handles remarkably well, Kettimuthu said. Globus is also good at integrity verification, in which the accuracy of the data is checked as the data is being transferred.

All in all, the recognition for Globus is one more piece of recognition that Argonne-developed computing technology can play a big role in enabling research and experimentation. ​“Without data movement and data sharing, modern science couldn’t exist,” Kettimuthu said. ​“Globus can be a useful tool for a wide range of sciences that rely on big data.”

Zhengchen Liu, Michael Link and Jack Kordas also contributed to the winning project.