Visit additional Tabor Communication Publications
February 28, 2011
NVIDIA is set to release a new CUDA toolkit to developers this Friday with the 4th generation of its popular GPU software suite. The company says CUDA 4.0 is designed to make parallel programming simpler, thus bringing more application developers into the GPGPU fold. Some of the new capabilities also foreshadow Project Denver, the codename for the company's future CPU-GPU architecture for workstations, servers, and supercomputers.
The 4.0 toolkit brings with it three new technologies: GPUDirect 2.0 for peer-to-peer communication among graphics chips on the same compute node; Unified Virtual Addressing (UVA) to provide a single address space that melds CPU and GPU memories; and Thrust, a set of C++ library of data structures and algorithms aimed at parallel programming.
According to Sumit Gupta, senior product manager for NVIDIA Tesla GPU Computing Unit, the enhancements are geared toward raising the abstraction level for the GPGPU programmer, making it easier for less GPU-savvy developers to take advantage of the architecture's high-performance data parallelism. CUDA already has a good deal of traction in the HPC community, and is even becoming well-supported by CAE software vendors like ANSYS, SIMULIA, and ACUSIM. With 4.0, NVIDIA is looking to cast a wider net and bring more mainstream developers into the GPU computing camp.
"The focus for us now is to get more people to use GPUs," says Gupta. "Clearly, we have a pretty good mindshare in the HPC community at this point. Now it's just going broader and deeper."
For example, GPUDirect 2.0, is the second generation of the technology whose original version allowed third-party PCI devices like network adapters and SSDs to bypass the CPU and tap directly into GPU memory. The 1.0 technology is supported by Mellanox and QLogic, who have incorporated the GPUDirect smarts into their respective InfiniBand adapters.
Likewise, the second version of GPUDirect enables multiple GPUs in the same node or workstation to access each other's memory over the PCI bus, without having to copy data to the system memory. For multi-GPU applications, the new GPUDirect promises both better performance (less copying of data and less CPU overhead) as well as less onerous programming (no routine calls to copy data to back and forth to system memory). According to Gupta, at some future date they'd like to apply this peer-to-peer GPU communication capability out to the cluster interconnect.
Extending the memory abstraction even further is the new Unified Virtual Addressing feature. In this case, CUDA provides a single address space that virtually merges system memory on the CPU side with the memory of one or more connected GPUs.This gives the application a view of all the heterogeneous memories as a single memory space.
Today a developer has to keep track of where data is and explicitly copy data to GPU memory if it’s not there before calling a function that uses the data. With UVA, the function itself can disambiguate where the data is and copy it appropriately. This, in particular, helps libraries become more self-contained and reduces the burden on the application developer.
Similarly for multiple GPUs, functions that use the GPU can now figure out which device the data is on and copy the data between them, instead of the developer “pre-conditioning” or copying the data before calling the function. The general idea here, Gupta says, is speed up the original CPU-to-GPU application port, and let the developers use the tools to work out any needed optimizations.
The UVA feature also has an eye toward the Project Denver architecture, which will integrate ARM CPUs and NVIDIA GPUs on the same chip, presumably making separate physical memory spaces a thing of the past. Thus, all CUDA applications developed with UVA should slide easily onto these heterogeneous platforms. "We always knew the Denver architecture was coming, so we've always planned for it in the CUDA programming models," says Gupta. "Features like Unified Virtual Addressing are meant to extend out to future architectures."
The other big enhancement in CUDA 4.0 is Thrust, a C++ template library that contains algorithms and data structures aimed at parallel programming. Once again, the idea is raise the abstraction level, in this case, for C++ programmers, so that developing parallelized applications is much more straightforward. Although C++ is not widely employed in traditional HPC (outside its use by quants for financial applications), the language is widely adopted across IT, from the consumer side to the enterprise.
To help boost performance, at compile time Thrust will attempt to choose the fastest code path for the hardware it's to be run on. In this case, that means it will divide the work between the GPUs and CPUs to maximize throughput -- yet another nod to the upcoming Denver architecture. NVIDIA claims Thrust routines such as parallel sorting are 5 to 100 times faster than the corresponding implementations of the Standard Template Library (STL) and Threading Building Blocks (TBB).
Besides the big three enhancements, CUDA 4.0 will also sport some new capabilities like being able to share a GPU with multiple CPU threads. So for example, if you have four threads running on a quad-core CPU, all those threads can drive computations on a single GPU. Previously, you would have needed four GPUs to do this or would have been restricted to executing each thread sequentially on the GPU side. Conversely, CUDA 4.0 will also let you drive multiple GPUs using a single CPU thread.
Also included in the new toolkit is an MPI implementation that automatically moves data to and from the GPU memory over InfiniBand when an application does an MPI send or receive, a new NPP image and computer vision library, an auto performance analysis capability for the Visual Profiler tool, a GPU binary disassembler, and some new features in cuda-gdb debugger.
The CUDA 4.0 toolkit will be released free of charge to registered developers on March 4th. By announcing it a few days in advance, NVIDIA is hoping to attract some new blood into the CUDA Registered Developer Program. No date is set for general release of the toolkit, but usually there's just a couple of months lag time between the release candidate and the final version.
Jun 19, 2013 |
Supercomputer architectures have evolved considerably over the last 20 years, particularly in the number of processors that are linked together. One aspect of HPC architecture that hasn't changed is the MPI programming model.
Jun 18, 2013 |
The world's largest supercomputers, like Tianhe-2, are great at traditional, compute-intensive HPC workloads, such as simulating atomic decay or modeling tornados. But data-intensive applications--such as mining big data sets for connections--is a different sort of workload, and runs best on a different sort of computer.
Jun 18, 2013 |
Researchers are finding innovative uses for Gordon, the 285 teraflop supercomputer housed at the San Diego Supercomputer Center (SDSC) that has a unique Flash-based storage system. Since going online, researchers have put the incredibly fast I/O to use on a wide variety of workloads, ranging from chemistry to political science.
Jun 17, 2013 |
The advent of low-power mobile processors and cloud delivery models is changing the economics of computing. But just as an economy car is good at different things than a full size truck, an HPC workload still has certain computing demands that neither the fastest smartphone nor the most elastic cloud cluster can fulfill.
Jun 14, 2013 |
For all the progress we've made in IT over the last 50 years, there's one area of life that has steadfastly eluded the grasp of computers: understanding human language. Now, researchers at the Texas Advanced Computing Center (TACC) are utilizing a Hadoop cluster on its Longhorn supercomputer to move the state of the art of language processing a little bit further.
05/10/2013 | Cleversafe, Cray, DDN, NetApp, & Panasas | From Wall Street to Hollywood, drug discovery to homeland security, companies and organizations of all sizes and stripes are coming face to face with the challenges – and opportunities – afforded by Big Data. Before anyone can utilize these extraordinary data repositories, however, they must first harness and manage their data stores, and do so utilizing technologies that underscore affordability, security, and scalability.
04/15/2013 | Bull | “50% of HPC users say their largest jobs scale to 120 cores or less.” How about yours? Are your codes ready to take advantage of today’s and tomorrow’s ultra-parallel HPC systems? Download this White Paper by Analysts Intersect360 Research to see what Bull and Intel’s Center for Excellence in Parallel Programming can do for your codes.
Join HPCwire Editor Nicole Hemsoth and Dr. David Bader from Georgia Tech as they take center stage on opening night at Atlanta's first Big Data Kick Off Week, filmed in front of a live audience. Nicole and David look at the evolution of HPC, today's big data challenges, discuss real world solutions, and reveal their predictions. Exactly what does the future holds for HPC?
Join our webinar to learn how IT managers can migrate to a more resilient, flexible and scalable solution that grows with the data center. Mellanox VMS is future-proof, efficient and brings significant CAPEX and OPEX savings. The VMS is available today.