The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing
October 30, 2008
Compilers and More: Optimizing GPU Kernels
1 | 2 | 3 | 4 | 5 All »
My last column discussed some of the complexities of programming GPUs today, focusing on how to interface the host program with the GPU. Here we focus on programming the GPU itself. As with last time, we'll look at a simple single-precision matrix multiplication, equivalent to the BLAS SGEMM routine.
Matmul is a highly parallel algorithm, but let me emphasize that parallelism does not equate to performance. We need to carefully sculpt our algorithm to match the parallelism available in the architecture in order to reap the benefits. This is true whether we are targeting a GPU, a multicore x64, or even a single core with packed SSE operations. As an example, I took the simple matmul loop (in C, but with the matrices stored column-major):
for( int j = 0; j < m; ++j )
for( int k = 0; k < p; ++k )
for( int i = 0; i < n; ++i )
a[i+pitch_a*j] += b[i+pitch_b*k] * c[k+pitch_c*j];
modified it several ways and ran it on an Intel Xeon (3GHz, 6MB cache, 16GB memory, Penryn) using 4096x4096 matrices (to compare with results we'll see below). With the loop in the order shown (stride-1 inner loop), the program ran at 1.7 GFLOPs; this is compiled C performance (using pgcc -fast). We can improve that by tiling or blocking the loops, organizing the matmul as a a bunch of submatrix multiplications, sized so each submatrix matmul fits in the processor cache. This improves performance to 5.7 GFLOPs, and it jumps to over 22 GFLOPs when we use OpenMP directives and run on all four cores. Advanced compilers help by automatically managing the vectorization, unrolling, memory alignments, adding prefetch instructions, and so forth.
We're going to see several matmul GPU kernels, with performance on our GPU development system, with an NVIDIA GeForce GTX 280 (1GB memory, 30 multiprocessors), using NVIDIA's CUDA language. The host is a Linux (OpenSUSE 11.0) triple-core AMD Phenom (2.1 GHz, 500KB cache, 4GB memory), though the host hardly matters; the performance for these experiments is entirely dominated by the GPU code.
As on the CPU, performance on a GPU can be fragile; small changes to the program can make large differences in performance. It's easy to write a slow program. This was a characteristic of High Performance Fortran, one that (my opinion) was a major cause of its downfall; while HPF made it easier to write parallel programs, it didn't make parallel programs fast. That is the job of the HPC programmer; the same will be true for accelerators, GPUs, and even multicore CPUs.
GPUs deliver their dramatic high performance through a well-balanced, carefully managed, highly parallel architecture. Algorithms running on the GPU must be parallelized and balanced as well; this does not come for free. Program development may cost extra time and effort to understand and use the appropriate programming model, a model that may not match the simple scalar processor with cache model we are comfortable with on x64 hosts. However, the analysis and programming techniques used to develop GPU algorithms will probably help you develop multicore programs as well. A good programming model with good compilers and tools can relieve you of much busywork, but you still have to think, and you still have to understand algorithms and architecture, and you should expect no less.
From here on below, I show many versions of matmul; if you're not a programmer or want to skip over the details, look for the performance tags below, until the Summary; you don't want to miss the conclusions. If you are a programmer and want to see all the code, you'll find all the sources in a kernels tarfile at the PGI Web site.
In my last column, I proposed a simple matmul kernel for the GPU and focused on the host code to drive the kernel. We'll use that simple kernel to start the discussion. What I had done is taken the matmul loop (as shown above), strip-mined the stride-1 i loop to the CUDA SIMD width of 32:
for( int is = 0; is < n; is += 32 )run the i element loop as a thread block, and run the is strip loop and j loop in parallel: Page: 1 of 5
for( int i = is; i < is+32; ++i )
for( int j = 0; j < m; ++j )
for( int k = 0; k < p; ++k )
a[i+pitch_a*j] += b[i+pitch_b*k] * c[k+pitch_c*j];
1 | 2 | 3 | 4 | 5 All »
Article Tools
Share Options
(Digg, Technorati, more)
Subscribe
Sponsored Links
RSS Feeds
Feeds by Topic
- Applications
- Developer Tools
- Interconnects
- Middleware
- Networks
- Processors
- Storage
- Systems
- Visualization
Feeds by Industry
Feeds by Content Type
Top Headlines
3D Seismic Data: Taking a Smarter Approach to Interpretation
Jul 09 | Engineer Live | The demand for computational tools to underpin the 3D seismic interpretation process has never been more apparent. Read more...
Engineering Unemployment Soared in 2Q to 8.6%
Jul 08 | EE Times | Unemployment for U.S. engineers has reached record levels, according to government figures. Read more...
Gartner Adjusts 2009 IT Spend Downward Again
Jul 08 | Network World | Global spending for 2009 projected to drop 6 percent, for a total of $3.2 trillion. Read more...
Concurrent and Parallel Are Not The Same
Jul 08 | Linux Magazine | Portability or efficiency? Neither is guaranteed when writing explicit parallel code. Read more...
800 TFLOP Real-Time Ray Tracing GPU Unveiled, Not for Gamers
Jul 07 | Ars Technica | Japanese company builds custom ASIC to accelerate real-time ray traced rendering for the auto industry. Read more...
Featured Whitepapers
Building High Performance Computing in a Green and Modular Solution Building Block
Apr 14 | | Many HPC IT departments are feeling the rising pressure to deliver more capacity computing and performance while trying to reduce the total cost of ownership. This white paper discusses how an environmentally-friendly and open-standards HPC building block based computing system using flexible interconnect options helps address capacity computing needs.
Multimedia
Webcast: Dell Expands HPC Access and Adoption with Intel Cluster Ready Program
Source: Addison Snell, GM/VP, Tabor Research; sponsored by Dell
Many organizations that could benefit from the use of HPC clusters find that it is complicated to get the systems up and running because of limited IT resources or the complexities of the clusters themselves. Learn how the Intel Cluster Ready program, for which Dell was an original partner, seeks to address this challenge for entry level and mid-range HPC users.
Video White Paper: Architecting a Better Network Storage Solution
BlueArc's Titan architecture represents an evolutionary step in file servers by creating a hardware-based file system that can scale bandwidth, IOPS, and overall data capacity well beyond conventional software-based devices. With its ability to virtualize a massive storage pool of up to four usable petabytes of tiered storage, Titan can scale with growing data requirements, offering a competitive advantage for businesses, researchers, or other enterprises seeking to better manage data growth while still ensuring optimal performance.
Webcast: HPC Development Solutions: Sun Studio & Sun HPC ClusterTools
Sun Studio Compilers and Tools and Sun HPC ClusterTools allow you to create high performance parallel applications for OpenSolaris, Solaris and Linux. Sun Studio Express 11/08 includes MPI performance analysis capabilities and full OpenMP 3.0 compiler support. Learn about all this and the latest in Sun HPC ClusterTools 8.1.
Newsletters
Stay informed! Subscribe to HPCwire email Newsletters.
HPC Job Bank
-
HPC Systems Administrator
McGill University -
High Performance Computing Data Center Manager
National Renewable Energy Laboratory -
High Performance Computing Systems Administrator
Idaho National Laboratory -
Solution Engineer- HPC
Dell Inc -
Parallel Programmer
Cardiff University -
Principal Software Research Engineer
Indiana University -
Post Doctoral Researcher - Exascale Stream Computing
IBM - Visit the HPCwire Job Bank
Featured Events
-
July 13-16, 2009
WORLDCOMP'09
Las Vegas , NV
USA
-
August 24-28, 2009
Data Mining: Levels I, II, & III
Denver , CO
USA