Compilers and More: Gloptimizations

By Michael Wolfe

November 9, 2007

In my last column, I discussed the performance improvements on one of the SPEC CPU2000 floating point benchmarks, 172.mgrid, between 1999 and 2006, and introduced an optimization (which we call Loop-Carried Redundancy Elimination) that improves mgrid significantly. The SPEC CPU committee has been quite careful with benchmarks that are sensitive to compiler tricks. The original SPEC ’89 benchmark suite included 030.matrix300 (dense matrix multiplication). Since matrix multiplication is one of a set of core operations performed in many numerical applications, it was considered a good candidate for a benchmark. But, to quote Reinhold Weicker from the SPEC Newsletter (December 1995): “However, if a program becomes a benchmark, compiler authors can become very creative.”

In this case, compilers were modified to aggressively tile the loops; in each tile, the program performs a sub-matrix multiplication where the submatrices fit into the cache memory, improving the total performance by up to a factor of eight. (The term “loop tiling” or “iteration space tiling” was actually suggested by my wife in 1986; I was looking for a better term to use in a paper, she was looking at the kitchen floor.) Since this one optimization had such a big impact, and didn’t apply to the other benchmarks, the SPEC committee decided that it was not a fair contributor to an overall performance metric, and matrix300 was retired in 1992.

I noted last time that Dell had published a SPEC CPU2000 run in November 1999 on a 733 MHz Pentium III Precision Workstation 420, and another in October 2006 on a 2.66 GHz Core 2 Extreme Precision Workstation 390. I chose to compare these two runs simply because they were from the same vendor seven years apart. The base CFP2000 proved from 242 to 2679, a factor of just over 11. The big winner was 179.art, which improved from a ratio of 313 to 9306, almost a factor of 30 speedup. What accounts for this factor of 30? Can we expect similar improvements for any other program?

One way to achieve great speedups is to make the base case really slow; then, even simple optimizations can deliver a nice improvement. Sometimes, the program is written in a way that seems intentionally aimed at making it run slowly. Such a case appears in ART.

The ART benchmark (Adaptive Resonance Theory neural network for image ecognition) is written in C, and the critical loop is:

    struct{
double *I; double W, X, V, U, P, Q, R;
} * f1_layer;
double **bus;
....
for( tj = 0; tj < numf2s; tj++ ){
Y[tj].y = 0;
if( !Y[tj].reset )
for( ti = 0; ti < numf1s; ti++ )
Y[tj].y += f1_layer[ti].P * bus[ti][tj];
}

This loop appears in several places in the program, and much of the time of the program is spent in it; speed up this loop and the program runs fast.

However, there are several problems here. The struct f1_layer has 8 members, each 8 bytes long (assuming 64-bit pointers). That means each element of f1_layer takes up a whole 64-byte cache line. So while we’re traversing the array f1_layer in the inner loop, we only use one value out of each cache line. Even worse, the two-dimensional array bus is dynamically allocated, and we are traversing down the columns; that means two memory accesses (to get the row pointer bus[ti], then the value bus[ti][tj]). In both cases, the program abuses memory bandwidth in the inner loop, unless the whole dataset fits in the cache.

A few more comments: it turns out the inner loop has a trip count of about 10,000, while the outer loop has only 6 iterations, and the conditional around the inner loop never trips (reset is always 0), at least not in the benchmark datasets, so the inner loop is always executed. These facts are known only at runtime, however.

I’ve seen three ways to make this loop run much faster.  Some compilers implement what has been called the ART hack, reorganizing the data for the struct f1_layer. The array-of-struct is reorganized to become a struct of arrays. It takes the same amount of space, but now each element f1_layer[ti].P become f1_layer.P[ti], so consecutive elements are adjacent in the cache, and the performance of the cache and the whole program improves greatly.

    struct{
double **I; double *W, *X, *V, *U, *P, *Q, *R;
} f1_layer;
double **bus;
....
for( tj = 0; tj < numf2s; tj++ ){
Y[tj].y = 0;
if( !Y[tj].reset )
for( ti = 0; ti < numf1s; ti++ )
Y[tj].y += f1_layer.P[ti] * bus[ti][tj];
}

I did this by hand on an older Intel 3.4GHz EM64T Xeon, using the PGI compilers, and measured a 49% performance improvement. You can imagine what kind of whole program analysis would allow or disallow this reorganization, and it is very rarely feasible. However, this benchmark is so important that some compilers take this step. Here’s an optimization (to abuse the term) that would never have been invented if not for this particular benchmark.

A different trick is to interchange the two loops. The problem being solved is the accesses to the array bus, which are non-adjacent, have no locality, and require two loads per iteration, problems which go away if the two loops are interchanged. However, after interchanging, the new inner loop has a very short trip count (though the compiler probably doesn’t know that). The other more serious problem is that interchanging the loops brings the conditional inside both loops.

    struct{
double *I; double W, X, V, U, P, Q, R;
} * f1_layer;
double **bus;
....
for( tj = 0; tj < numf2s; tj++ )
Y[tj].y = 0;
for( ti = 0; ti < numf1s; ti++ ){
for( tj = 0; tj < numf2s; tj++ )
if( !Y[tj].reset )
Y[tj].y += f1_layer[ti].P * bus[ti][tj];
}

On the same Xeon, I measured a speedup of 2.33. However, this is quite dangerous, from a performance standpoint; instead of ‘numf2s’ conditional tests, there are now ‘numf1s*numf2s’ tests. If the test tripped at all, the modified version could execute quite a bit slower than the original. Yet, at least one compiler implements this and gets more than a factor of two improvement in the whole program, mostly due to this loop transformation.

The third trick, not implemented by any compiler (yet, that I know of), is to leave the loops the way they are and to transpose the array bus (and a similarly accessed array, tds). It turns out these arrays are dynamically allocated, but accessed in the wrong dimension. If we transpose the dimensions, the performance of this loop and the whole program really screams. On that same Xeon, I measured a speedup of 4.3.

This last is somewhat harder to implement automatically. The most severe problem is that C doesn’t really have two-dimensional arrays; to get a two-dimensional dynamic array, the program has to first allocate a one-dimensional vector of pointers. To transpose an array, the program has to allocate a vector of a different size. Purists argue that the modified program might allocate more space, and hence might run out of memory for some large datasets.

Moreover, the performance improvements are not very stable. The speedups I measured on the EM64T were 1.49 for array-of-struct reorganization, 2.3 for loop interchange, and 4.3 for for array transposition. Last month, I compared performance using an older 733MHz Intel Pentium III and a 2.66GHz Intel Core 2 Extreme. The speedups for these three optimizations on these two machines (and the EM64T) are summarized here:

       EM64T   PIII      Core2
        1.49       1.64      1.08    array-of-struct reorganization
        2.33       1.00      1.03    loop interchange
        4.34       1.19      1.48    array transpose

I am guessing that these optimizations have much less effect on the latest Core 2 because the data all fits in the much larger cache.

So, last month I discussed compiler tuning for 172.mgrid, where a new optimization improved its performance about 20%; happily, we’ve found this new feature applies in many other user applications as well. This month, I presented compiler tuning for 179.art, where aggressive optimization can improve the performance by factors (on some machines). I won’t go into the hacks, tricks, and other tuning for the other SPEC CPU2000 benchmarks; the number of compiler tricks is linearly related to the number of benchmarks.

With the increased importance of benchmark performance, compiler development teams are encouraged to aggressive (or even heroic) efforts to optimize performance on these programs. The only reason they do so is that vendors and users (and compiler product managers) place artificial importance on certain benchmark results. Yet, these optimizations, while impressive, may be of little value, or no value, or even negative value among the more general user domain, or even on the same program with different inputs. As shown here for ART, the value of the optimization may decrease to essentially zero as the architecture progresses.

Therefore, in the spirit of the Golden Raspberry award (http://www.razzies.com/) and the Ig Nobel Prizes (http://www.improbable.com/ig), I propose that we honor the most outrageous, benchmark-specific optimization with an award of its own, the Gloptimization award, or Gloppy for short. The award itself would be a pile of glop (or representation thereof), defined in my American Heritage Dictionary as “n. 1. A messy mixture, as of food. 2. Something, as writing, that is worthless. v. 1. To cover with glop. 2. To put glop on.” The award, at least, is appropriate. More credit should be given to an optimization that applies to a single benchmark; extra credit if it measureably slows down other benchmarks or applications. Posthumous awards can be proposed for optimizations that are actually removed from compilers as the targeted benchmarks are themselves retired. The optimizations mentioned above would certainly be candidates for a Gloppy, though perhaps not winners.

The lesson for customers is that standard benchmarks are valuable tools, but should not be the only or primary means of comparing system performance. The best measure is always the performance of your own applications, to which the systems and accompanying software are not specifically tuned.

SPEC (R) is a registered trademark of the Standard Performance Evaluation Corporation (http://www.spec.org/).

—–

Michael Wolfe has developed compilers for over 30 years in both academia and industry, and is now a senior compiler engineer at The Portland Group, Inc. (www.pgroup.com), a wholly-owned subsidiary of STMicroelectronics, Inc. The opinions stated here are those of the author, and do not represent opinions of The Portland Group, Inc. or STMicroelectronics, Inc.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

RSC Reports 500Tflops, Hot Water Cooled System Deployed at JINR

April 18, 2018

RSC, developer of supercomputers and advanced HPC systems based in Russia, today reported deployment of “the world's first 100% ‘hot water’ liquid cooled supercomputer” at Joint Institute for Nuclear Research (JI Read more…

By Staff

New Device Spots Quantum Particle ‘Fingerprint’

April 18, 2018

Majorana particles have been observed by university researchers employing a device consisting of layers of magnetic insulators on a superconducting material. The advance opens the door to controlling the elusive particle Read more…

By George Leopold

Cray Rolls Out AMD-Based CS500; More to Follow?

April 18, 2018

Cray was the latest OEM to bring AMD back into the fold with introduction today of a CS500 option based on AMD’s Epyc processor line. The move follows Cray’s introduction of an ARM-based system (XC-50) last November. Read more…

By John Russell

HPE Extreme Performance Solutions

Hybrid HPC is Speeding Time to Insight and Revolutionizing Medicine

High performance computing (HPC) is a key driver of success in many verticals today, and health and life science industries are extensively leveraging these capabilities. Read more…

Hennessy & Patterson: A New Golden Age for Computer Architecture

April 17, 2018

On Monday June 4, 2018, 2017 A.M. Turing Award Winners John L. Hennessy and David A. Patterson will deliver the Turing Lecture at the 45th International Symposium on Computer Architecture (ISCA) in Los Angeles. The Read more…

By Staff

Cray Rolls Out AMD-Based CS500; More to Follow?

April 18, 2018

Cray was the latest OEM to bring AMD back into the fold with introduction today of a CS500 option based on AMD’s Epyc processor line. The move follows Cray’ Read more…

By John Russell

IBM: Software Ecosystem for OpenPOWER is Ready for Prime Time

April 16, 2018

With key pieces of the IBM/OpenPOWER versus Intel/x86 gambit settling into place – e.g., the arrival of Power9 chips and Power9-based systems, hyperscaler sup Read more…

By John Russell

US Plans $1.8 Billion Spend on DOE Exascale Supercomputing

April 11, 2018

On Monday, the United States Department of Energy announced its intention to procure up to three exascale supercomputers at a cost of up to $1.8 billion with th Read more…

By Tiffany Trader

Cloud-Readiness and Looking Beyond Application Scaling

April 11, 2018

There are two aspects to consider when determining if an application is suitable for running in the cloud. The first, which we will discuss here under the title Read more…

By Chris Downing

Transitioning from Big Data to Discovery: Data Management as a Keystone Analytics Strategy

April 9, 2018

The past 10-15 years has seen a stark rise in the density, size, and diversity of scientific data being generated in every scientific discipline in the world. Key among the sciences has been the explosion of laboratory technologies that generate large amounts of data in life-sciences and healthcare research. Large amounts of data are now being stored in very large storage name spaces, with little to no organization and a general unease about how to approach analyzing it. Read more…

By Ari Berman, BioTeam, Inc.

IBM Expands Quantum Computing Network

April 5, 2018

IBM is positioning itself as a first mover in establishing the era of commercial quantum computing. The company believes in order for quantum to work, taming qu Read more…

By Tiffany Trader

FY18 Budget & CORAL-2 – Exascale USA Continues to Move Ahead

April 2, 2018

It was not pretty. However, despite some twists and turns, the federal government’s Fiscal Year 2018 (FY18) budget is complete and ended with some very positi Read more…

By Alex R. Larzelere

Nvidia Ups Hardware Game with 16-GPU DGX-2 Server and 18-Port NVSwitch

March 27, 2018

Nvidia unveiled a raft of new products from its annual technology conference in San Jose today, and despite not offering up a new chip architecture, there were still a few surprises in store for HPC hardware aficionados. Read more…

By Tiffany Trader

Inventor Claims to Have Solved Floating Point Error Problem

January 17, 2018

"The decades-old floating point error problem has been solved," proclaims a press release from inventor Alan Jorgensen. The computer scientist has filed for and Read more…

By Tiffany Trader

Researchers Measure Impact of ‘Meltdown’ and ‘Spectre’ Patches on HPC Workloads

January 17, 2018

Computer scientists from the Center for Computational Research, State University of New York (SUNY), University at Buffalo have examined the effect of Meltdown Read more…

By Tiffany Trader

Russian Nuclear Engineers Caught Cryptomining on Lab Supercomputer

February 12, 2018

Nuclear scientists working at the All-Russian Research Institute of Experimental Physics (RFNC-VNIIEF) have been arrested for using lab supercomputing resources to mine crypto-currency, according to a report in Russia’s Interfax News Agency. Read more…

By Tiffany Trader

How the Cloud Is Falling Short for HPC

March 15, 2018

The last couple of years have seen cloud computing gradually build some legitimacy within the HPC world, but still the HPC industry lies far behind enterprise I Read more…

By Chris Downing

Chip Flaws ‘Meltdown’ and ‘Spectre’ Loom Large

January 4, 2018

The HPC and wider tech community have been abuzz this week over the discovery of critical design flaws that impact virtually all contemporary microprocessors. T Read more…

By Tiffany Trader

How Meltdown and Spectre Patches Will Affect HPC Workloads

January 10, 2018

There have been claims that the fixes for the Meltdown and Spectre security vulnerabilities, named the KPTI (aka KAISER) patches, are going to affect applicatio Read more…

By Rosemary Francis

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Fast Forward: Five HPC Predictions for 2018

December 21, 2017

What’s on your list of high (and low) lights for 2017? Volta 100’s arrival on the heels of the P100? Appearance, albeit late in the year, of IBM’s Power9? Read more…

By John Russell

Leading Solution Providers

Deep Learning at 15 PFlops Enables Training for Extreme Weather Identification at Scale

March 19, 2018

Petaflop per second deep learning training performance on the NERSC (National Energy Research Scientific Computing Center) Cori supercomputer has given climate Read more…

By Rob Farber

Lenovo Unveils Warm Water Cooled ThinkSystem SD650 in Rampup to LRZ Install

February 22, 2018

This week Lenovo took the wraps off the ThinkSystem SD650 high-density server with third-generation direct water cooling technology developed in tandem with par Read more…

By Tiffany Trader

AI Cloud Competition Heats Up: Google’s TPUs, Amazon Building AI Chip

February 12, 2018

Competition in the white hot AI (and public cloud) market pits Google against Amazon this week, with Google offering AI hardware on its cloud platform intended Read more…

By Doug Black

HPC and AI – Two Communities Same Future

January 25, 2018

According to Al Gara (Intel Fellow, Data Center Group), high performance computing and artificial intelligence will increasingly intertwine as we transition to Read more…

By Rob Farber

New Blueprint for Converging HPC, Big Data

January 18, 2018

After five annual workshops on Big Data and Extreme-Scale Computing (BDEC), a group of international HPC heavyweights including Jack Dongarra (University of Te Read more…

By John Russell

US Plans $1.8 Billion Spend on DOE Exascale Supercomputing

April 11, 2018

On Monday, the United States Department of Energy announced its intention to procure up to three exascale supercomputers at a cost of up to $1.8 billion with th Read more…

By Tiffany Trader

Momentum Builds for US Exascale

January 9, 2018

2018 looks to be a great year for the U.S. exascale program. The last several months of 2017 revealed a number of important developments that help put the U.S. Read more…

By Alex R. Larzelere

Google Chases Quantum Supremacy with 72-Qubit Processor

March 7, 2018

Google pulled ahead of the pack this week in the race toward "quantum supremacy," with the introduction of a new 72-qubit quantum processor called Bristlecone. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This