Machine learning has made enormous strides in the few years, owing in large part to powerful and efficient parallel processing provided by general-purpose GPUs. The latest example of this trend is exemplified by a partnership between New York University’s Center for Data Science and NVIDIA. The mission, says the pair, is to develop next-gen deep learning applications and algorithms for large-scale GPU-accelerated systems.
NVIDIA’s Kimberly Powell shares the news in a recent blog post, noting that until recently, many deep learning researchers worked on systems with just one GPU. The field underwent a burst of activity, facilitated by the rise of algorithms, the availability of large datasets (thanks to sensor data and the Internet) and of course parallel processing via GPUs. Trailblazers in the space set new records in applications like feature detection and language processing with fairly humble, from an HPC perspective, GPU-accelerated systems, but it wasn’t long before the models hit the compute ceiling.
For NYU, the logical next step was moving to a multi-GPU cluster to support bigger models with more training parameters. Their new deep learning computing system, called “ScaLeNet,” is an eight-node Cirrascale cluster with 32 NVIDIA Tesla K80 dual-GPU accelerators.
“The new high-performance system will let NYU researchers take on bigger challenges, and create deep learning models that let computers do human-like perceptual tasks,” writes Powell.
“Multi-GPU machines are a necessary tool for future progress in AI and deep learning,” says deep learning pioneer Yann LeCun, who founded the NYU Center for Data Science. “Potential applications include self-driving cars, medical image analysis systems, real-time speech-to-speech translation, and systems that can truly understand natural language and hold dialogs with people.”
LeCun, who is also the director of AI Research at Facebook, goes on to cite potential applications that extend far beyond these initial use cases.
“CDS has research projects that apply machine and deep learning to the physical, life and social sciences,” he says. “This includes Bayesian models of cosmology and high-energy physics, computational models of the visual and motor cortex, deep learning systems for medical and biological image analysis, as well as machine-learning models of social behavior and economics.”
LeCun is one of the general chairs for the upcoming International Conference on Learning Representations, taking place May 7-9 in San Diego. He will be presenting a paper describing a fast, multi-GPU implementation of a convolutional neural network, used for image and video understanding.