SC13 Research Highlight: Large Graph Processing Without the Overhead

By Dr. Ling Liu and Kisung Lee

November 16, 2013

Many real world information networks consist of millions or billions of vertices representing heterogeneous entities and billions or trillions of edges representing heterogeneous types of relationships among entities.

Image Source: Max Delbrück Center for Molecular Medicine

For example, the crawled Web graph is estimated to have more than 20 billions of pages (vertices) with 160 billions hyperlinks (edges). Facebook user community exceeds 1 billion users (vertices) with more than 140 billion friendship relationships (edges) in 2012.  The billion triple challenges from the Semantic Web community have put forward large collection of RDF datasets with hundreds of millions of vertices and billions of edges.

As the size and variety of information networks continue to grow in many science and engineering domains, graph computations often exceed the processing capacity of conventional hardware, software systems and tools for a number of reasons. First, graph data often exhibits higher data correlations through both direct and indirect edges and such high correlation tends to generate large size of intermediate results during graph computations. When the size of intermediate results exceeds the available memory, the out of memory problem is unavoidable. Second, the graph datasets are growing in volume, variety and velocity. The bigger the size of the graphs gets, the worse the performance will be for most of the graph computations. One open challenge in this space is how to effectively partition a large graph to enable efficient parallel processing of complex graph operations.

One of the papers to be presented at the ACM/IEEE SC13 conference, titled “Efficient data partitioning model for heterogeneous graphs in the Cloud”, proposes a flexible graph partitioning framework, called VB-partitioner. This work is co-authored by the doctorate student Kisung Lee and Prof. Dr. Ling Liu from the school of Computer Science at Georgia Institute of Technology. To make parallel graph computations highly efficient, an important design goal of VB-Partitioner is to devise graph partitioning techniques that can effectively minimize the inter-partition communication overhead and maximize the intra-partition computation capacity (local processing).

Concretely, the first prototype of the VB-Partitioner focuses on efficient processing of graph queries, namely finding all the subgraphs matching a given subgraph pattern. VB-Partitioner partitions a large graph in three steps.

  • First, it constructs three types of Vertex Blocks (in-VBs, out-VBs and bi-VBs) to capture the general graph processing locality.  Each vertex block has an anchor vertex.
  • Second, it constructs three types of k-hop Extended Vertex Blocks (in-EVBs, out-EVBs and bi-EVBs) to distribute vertex blocks with better query locality. Each EVB has one anchor vertex. It achieves query locality by employing controlled edge replication. The setting of k is determined by the radius of frequent query graphs in order to ensure that most frequently requested queries can be processed in parallel at all partitions with minimized inter-partition communication overhead.
  • Third, it partitions a graph by grouping its vertex blocks and EVBs to maximize parallelism in graph processing while ensuring load balance, controlled edge replication and fast grouping.

Four techniques are considered and compared in the context of grouping and placement of VBs and EVBs to partitions: random grouping, hash-based grouping, min-cut based grouping and high degree vertex-based grouping.  As an integral part of the VB-Partitioner, a data partition-aware query partitioning model is also developed to handle the cases where the radius of a query is larger than k. The experimental results reported in the paper demonstrate the effectiveness of VB-Partitioner in terms of query processing efficiency, data loading time and partition distribution balance.

Graph computations can be broadly classified into two categories, graph queries that find matching subgraphs of a given pattern and iterative graph algorithms that find clusters, orderings, paths or correlation patterns. The former targets at subgraph matching problems over large static graphs and the later targets at graph inference kernels that traverse the graph by iteratively updating the weight of vertices or edges, such as PageRank, shortest path algorithms, spanning tree algorithms, topological sort, and so forth.

Although the first generation of the VB-Partitioner is tailored primarily for efficient parallel processing of graph queries, the ongoing work on VB-Partitioner includes exploring the feasibility and effectiveness of VB-Partitioner in the context of iterative graph algorithms. For example, to minimize inter-partition communications and maximize parallelism in graph computation, it is crucial to optimize the shared memory by minimizing parallel overhead of synchronization barriers. It is equally important to optimize the distributed memory by bounding message buffer sizes, bundling messages, overlapping communication with computation to amortize the overhead of barriers.

image1
Image Source: Giot et al., “A Protein Interaction Map of Drosophila melanogaster”, Science 302, 1722-1736, 2003.”

In addition to exploring parallel computation opportunities through graph partitioning using multi-threads, multi-cores, multiple workers, one can also exploit and combine with other performance optimization techniques to scale large graph analytics. Example techniques include

  • Compression to provide compact storage and in-memory processing,
  • Data placements on disk and in memory to balance computation with storage, and to maximize sequential access and minimize random access,
  • Indexing at vertex and/or edge level to utilize sequential access and minimize unnecessary random access,
  • Caching at vertex, edge or query level to gain performance for repeated access.

Please come hear more on Tuesday, November 19, 2013 10:30AM – 11:00AM (Location: Room 205/207)

http://sc13.supercomputing.org/schedule/event_detail.php?evid=pap708

About the Authors

LingLing Liu is a Professor in the School of Computer Science at Georgia Institute of Technology. She directs the research programs in Distributed Data Intensive Systems Lab (DiSL), examining various aspects of large scale data intensive systems. Prof. Ling Liu is an internationally recognized expert in the areas of Cloud computing, Distributed Computing, Big Data technologies, Database systems and Service oriented computing. Prof. Liu is a recipient of IEEE Computer Society Technical Achievement Award in 2012. Currently Prof. Liu is the editor in chief of IEEE Transactions on Service Computing, and serves on the editorial board of half dozen international journals, including Journal of Parallel and Distributed Computing (JPDC), ACM Transactions on Internet Technology (TOIT), ACM Transactions on Web (TWEB). Dr. Liu’s current research is primarily sponsored by NSF, IBM, and Intel.

 

luiKisung Lee is a Ph.D student in the School of Computer Science at Georgia Tech. He received his BS and MS degree in computer science from KAIST in 2005 and 2007 respectively. He had worked for ETRI as a researcher from 2007 to 2010. He is conducting research in distributed and parallel processing of big data in the Cloud, mobile computing and social network analysis.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

ASC18: Final Results Revealed & Wrapped Up

May 17, 2018

It was an exciting week at ASC18 in Nanyang, China. The student teams braved extreme heat, extremely difficult applications, and extreme competition in order to cross the cluster competition finish line. The gala awards ceremony took place on Wednesday. The auditorium was packed with student teams, various dignitaries, the media, and other interested parties. So what happened? Read more…

By Dan Olds

ASC18: Tough Applications & Tough Luck

May 17, 2018

The applications at the ASC18 Student Cluster Competition were tough. Tougher than the $3.99 steak special at your local greasy spoon restaurant. The apps are so tough that even Chuck Norris backs away from them slowly. Read more…

By Dan Olds

Spring Meetings Underscore Quantum Computing’s Rise

May 17, 2018

The month of April 2018 saw four very important and interesting meetings to discuss the state of quantum computing technologies, their potential impacts, and the technology challenges ahead. These discussions happened in Read more…

By Alex R. Larzelere

HPE Extreme Performance Solutions

HPC and AI Convergence is Accelerating New Levels of Intelligence

Data analytics is the most valuable tool in the digital marketplace – so much so that organizations are employing high performance computing (HPC) capabilities to rapidly collect, share, and analyze endless streams of data. Read more…

IBM Accelerated Insights

Mastering the Big Data Challenge in Cognitive Healthcare

Patrick Chain, genomics researcher at Los Alamos National Laboratory, posed a question in a recent blog: What if a nurse could swipe a patient’s saliva and run a quick genetic test to determine if the patient’s sore throat was caused by a cold virus or a bacterial infection? Read more…

Quantum Network Hub Opens in Japan

May 17, 2018

Following on the launch of its Q Commercial quantum network last December with 12 industrial and academic partners, the official Japanese hub at Keio University is now open to facilitate the exploration of quantum applications important to science and business. The news comes a week after IBM announced that North Carolina State University was the first U.S. university to join its Q Network. Read more…

By Tiffany Trader

ASC18: Final Results Revealed & Wrapped Up

May 17, 2018

It was an exciting week at ASC18 in Nanyang, China. The student teams braved extreme heat, extremely difficult applications, and extreme competition in order to cross the cluster competition finish line. The gala awards ceremony took place on Wednesday. The auditorium was packed with student teams, various dignitaries, the media, and other interested parties. So what happened? Read more…

By Dan Olds

Spring Meetings Underscore Quantum Computing’s Rise

May 17, 2018

The month of April 2018 saw four very important and interesting meetings to discuss the state of quantum computing technologies, their potential impacts, and th Read more…

By Alex R. Larzelere

Quantum Network Hub Opens in Japan

May 17, 2018

Following on the launch of its Q Commercial quantum network last December with 12 industrial and academic partners, the official Japanese hub at Keio University is now open to facilitate the exploration of quantum applications important to science and business. The news comes a week after IBM announced that North Carolina State University was the first U.S. university to join its Q Network. Read more…

By Tiffany Trader

Democratizing HPC: OSC Releases Version 1.3 of OnDemand

May 16, 2018

Making HPC resources readily available and easier to use for scientists who may have less HPC expertise is an ongoing challenge. Open OnDemand is a project by t Read more…

By John Russell

PRACE 2017 Annual Report: Exascale Aspirations; Industry Collaboration; HPC Training

May 15, 2018

The Partnership for Advanced Computing in Europe (PRACE) today released its annual report showcasing 2017 activities and providing a glimpse into thinking about Read more…

By John Russell

US Forms AI Brain Trust

May 11, 2018

Amid calls for a U.S. strategy for promoting AI development, the Trump administration is forming a senior-level panel to help coordinate government and industry research efforts. The Select Committee on Artificial Intelligence was announced Thursday (May 10) during a White House summit organized by the Office of Science and Technology Policy (OSTP). Read more…

By George Leopold

Emerging Advanced Scale Tech Trends Focus of Annual Tabor Conference

May 9, 2018

At Tabor Communications' annual Advanced Scale Forum (ASF) held this week in Austin, the focus was on enterprise adoption of HPC-class technologies and high performance data analytics (HPDA). It’s a confab that brings together end users (CIOs, IT planners, department heads) and vendors and encourages... Read more…

By the Editorial Team

Google I/O 2018: AI Everywhere; TPU 3.0 Delivers 100+ Petaflops but Requires Liquid Cooling

May 9, 2018

All things AI dominated discussion at yesterday’s opening of Google’s I/O 2018 developers meeting covering much of Google's near-term product roadmap. The e Read more…

By John Russell

MLPerf – Will New Machine Learning Benchmark Help Propel AI Forward?

May 2, 2018

Let the AI benchmarking wars begin. Today, a diverse group from academia and industry – Google, Baidu, Intel, AMD, Harvard, and Stanford among them – releas Read more…

By John Russell

How the Cloud Is Falling Short for HPC

March 15, 2018

The last couple of years have seen cloud computing gradually build some legitimacy within the HPC world, but still the HPC industry lies far behind enterprise I Read more…

By Chris Downing

Russian Nuclear Engineers Caught Cryptomining on Lab Supercomputer

February 12, 2018

Nuclear scientists working at the All-Russian Research Institute of Experimental Physics (RFNC-VNIIEF) have been arrested for using lab supercomputing resources to mine crypto-currency, according to a report in Russia’s Interfax News Agency. Read more…

By Tiffany Trader

Inventor Claims to Have Solved Floating Point Error Problem

January 17, 2018

"The decades-old floating point error problem has been solved," proclaims a press release from inventor Alan Jorgensen. The computer scientist has filed for and Read more…

By Tiffany Trader

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown Read more…

By Tiffany Trader

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Deep Learning at 15 PFlops Enables Training for Extreme Weather Identification at Scale

March 19, 2018

Petaflop per second deep learning training performance on the NERSC (National Energy Research Scientific Computing Center) Cori supercomputer has given climate Read more…

By Rob Farber

AI Cloud Competition Heats Up: Google’s TPUs, Amazon Building AI Chip

February 12, 2018

Competition in the white hot AI (and public cloud) market pits Google against Amazon this week, with Google offering AI hardware on its cloud platform intended Read more…

By Doug Black

Leading Solution Providers

US Plans $1.8 Billion Spend on DOE Exascale Supercomputing

April 11, 2018

On Monday, the United States Department of Energy announced its intention to procure up to three exascale supercomputers at a cost of up to $1.8 billion with th Read more…

By Tiffany Trader

Lenovo Unveils Warm Water Cooled ThinkSystem SD650 in Rampup to LRZ Install

February 22, 2018

This week Lenovo took the wraps off the ThinkSystem SD650 high-density server with third-generation direct water cooling technology developed in tandem with par Read more…

By Tiffany Trader

HPC and AI – Two Communities Same Future

January 25, 2018

According to Al Gara (Intel Fellow, Data Center Group), high performance computing and artificial intelligence will increasingly intertwine as we transition to Read more…

By Rob Farber

Google Chases Quantum Supremacy with 72-Qubit Processor

March 7, 2018

Google pulled ahead of the pack this week in the race toward "quantum supremacy," with the introduction of a new 72-qubit quantum processor called Bristlecone. Read more…

By Tiffany Trader

HPE Wins $57 Million DoD Supercomputing Contract

February 20, 2018

Hewlett Packard Enterprise (HPE) today revealed details of its massive $57 million HPC contract with the U.S. Department of Defense (DoD). The deal calls for HP Read more…

By Tiffany Trader

CFO Steps down in Executive Shuffle at Supermicro

January 31, 2018

Supermicro yesterday announced senior management shuffling including prominent departures, the completion of an audit linked to its delayed Nasdaq filings, and Read more…

By John Russell

Deep Learning Portends ‘Sea Change’ for Oil and Gas Sector

February 1, 2018

The billowing compute and data demands that spurred the oil and gas industry to be the largest commercial users of high-performance computing are now propelling Read more…

By Tiffany Trader

Nvidia Ups Hardware Game with 16-GPU DGX-2 Server and 18-Port NVSwitch

March 27, 2018

Nvidia unveiled a raft of new products from its annual technology conference in San Jose today, and despite not offering up a new chip architecture, there were still a few surprises in store for HPC hardware aficionados. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This