Compilers and More: A Computing Larrabee
The buzz and excitement around Intel’s Larrabee processor continues to build. Intel has been careful to present Larrabee as a graphics processor that can also be used for highly parallel tasks such as game physics, avoiding the claim that it will be appropriate for HPC. The Intel marketing department isn’t stupid, of course; they don’t want to erode the market for their own very high-powered and successful (and highly profitable) Core-2 (and beyond) server processors. Nonetheless, it will be hard to prevent experimentation and even productization of HPC systems with Larrabee processors, either as the main CPU or as an attached accelerator, unless Intel chooses to control the supply.
I don’t claim to be competent to evaluate the Larrabee (or any other processor) for its graphics features or performance, but we can discuss Larrabee as a compute engine. Larrabee looks an awful lot like an x86 cluster node; anyone who has experience building HPC clusters has pretty good intuition about the design tradeoffs that make for a balanced and effective system. So it can be interesting to explore the Larrabee architecture, to look at the design choices Intel made, and what alternatives they might have considered or might consider in the future.
Let’s look at an abstraction of the Larrabee architecture. Larrabee supports several kinds of parallelism. It was recently stated that Larrabee will have 32 cores in its first implementation. That gives a MIMD parallelism factor of 32, 32 threads running in parallel on separate cores. Each core has a vector processing unit that augments the SSE instruction set; the VPU can support 16 single precision operations in vector mode. That gives a SIMD or vector parallelism factor of 16, assuming the vector instructions are implemented fully in SIMD mode. Most vector processors use pipelining to reduce the transistor count, but I doubt that Intel is limited by silicon real estate. For double precision, the vector width is cut in half to eight. Larrabee also supports four thread contexts per core; when one thread stalls on a level-1 cache miss, the core switches to one of the other thread contexts. This keeps the core busy while the cache is handling the miss. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, and a 256KB L2 unified cache. Intel describes the latter as a 256KB subset of a global 8MB L2 cache. Since each core accesses and stores data in its own 256KB L2 subset, perhaps someone will explain to me why this is different than 32 separate coherent 256KB L2 caches. The Larrabee control unit implements dual instruction issue in simple cases, depending on the compiler to schedule compatible adjacent instructions.
Compare this to a quad-core Intel Nehalem processor. Nehalem has a MIMD parallelism factor of 4, the SSE width is also 4 (single precision), and it supports two simultaneous threads with Intel hyperthreading. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, 256KB L2 unified cache, and there is a large (8MB or more) shared L3 cache. What Nehalem has that Larrabee doesn’t, besides the huge L3 cache, is a very powerful control unit, allowing multiscalar execution (multiple instructions issued per clock), out-of-order execution (instructions can be reordered at execution time), and register renaming (many more hardware registers than the 16 available to the programmer, necessary to effectively support out-of-order execution). With hyperthreading enabled, a Nehalem can not only keep two thread contexts active, it can issue instructions from both threads simultaneously, in the same clock cycle. This helps to keep the functional units busy, using the instruction-level parallelism of both threads at the same time.
However, Nehalem’s power comes at a cost, namely that very large, complex, expensive control unit, and that complexity permeates the core design throughout the microarchitecture. The Larrabee goes back to a simpler time, apparently back to the original Pentium dual-issue, in-order control unit. This simplifies the microarchitecture, freeing up chip real estate for other purposes, in this case for more cores.
We could have a discussion here comparing the goals and solutions addressed by Larrabee (relative to Nehalem), to the goals and solutions addressed by RISC processors in the 1980s (relative to CISC, specifically Digital VAX and IBM mainframes). In both cases, designers simplified the control unit in order to free chip resources and expand other capabilities. For RISC, this meant a larger register file and cache, while for Larrabee, it means more cores. Both solutions depend on software to deliver performance; both RISC and Larrabee expect compilers to do a better job of instruction selection and scheduling, for instance. But let’s leave that discussion for another day.
Comparison: Larrabee vs. Nehalem
The feature scorecard between Larrabee and Nehalem looks like this:
|16||4||vector width (single-precision)|
|2||4||instruction issue width|
With all those cores, how can Larrabee not outperform a Nehalem on highly parallel code? Just to get the same instruction issue rate as Larrabee, a Nehalem would need a clock rate about four times faster; to get the same vector operation bandwidth, Nehalem would need another factor of four. However, that depends on a fully-parallel application at full rate in both cases. Amdahl’s Law will apply, and Nehalem certainly screams on the sequential parts of the program. We’ll come back to this, but consider also the cache and memory interfaces.
Designing a parallel system like Larrabee means making many choices and tradeoffs. What if we change some of the Larrabee parameters, for instance, reduce the vector width, and use those transistors for more cores? The tradeoff is that a vector unit is quite regular, relatively simple compared to a whole core. If we cut the vector instruction set back to SSE widths (four wide, single precision), and use those transistors instead for more cores, we might only get 1.5X or 2X more cores. Let’s be optimistic and assume 2X, and leave the rest of the design the same, as much as we can; in particular, let’s assume the total L2 cache size stays at 8MB total, but that we can add L1 cache for each new core. We’ll call this modified design Kurlabee, with the following feature scorecard:
|16||4||vector width (single-precision)|
|2||2||instruction issue width|
For irregular parallel programs, Kurlabee has the advantage, with 64 cores, of keeping twice as many threads progressing. However, for highly parallel, regular (vector) code, Larrabee can generate 512 results per cycle (32X16), whereas Kurlabee can only generate 256. For the graphics market, where the applications are highly regular, Larrabee seems to a better design. Also keep in mind that the multithreading width is only four, which is designed to keep the core busy while tolerating an L1 cache miss; if there are many L2 cache misses, the core will likely stall. The Kurlabee design has a smaller L2 cache per core, so those extra cores will likely spend more time waiting on memory.
So, if longer vectors are good, why not make them even longer? What if we halve the number of cores, and use those resources for longer vector operations? As before, we’ll use a 2:1 ratio of vector length to core count, and keep the rest of the design relatively constant. We’ll call this design Moabee:
|16||64||vector width (single-precision)|
|2||2||instruction issue width|
For those highly parallel, regular programs, Moabee can generate a stunning 1024 results per cycle, so it might have a strong advantage. However, there is the tradeoff for irregular codes, and while the vector length increased by a factor of four, the L2 cache size (per core) only doubled, and the L1 cache size stayed the same. In fact, the 32KB L1 data cache is only twice the size of the Moabee vector register file (16 vector registers X 64 elements X 4 bytes X 4 threads), so there could be serious cache pressure problems. Other issues with longer vectors are that the Larrabee doesn’t have a true vector instruction set; in particular, it doesn’t have a vector length register, allowing arbitrary-length short vectors. This means that unless the program or compiler knows that the application operations are an exact multiple of the vector register width, an extra copy of the loop must be generated to handle the remainder operations. Longer vector widths, as with Moabee, only exacerbate this problem.
Making design tradeoffs like this takes much more effort, thought, and application modeling than I’ve put into this discussion. I haven’t even looked at other tradeoffs, such as cores for cache, or supporting more simultaneous threads, and made gross assumptions about the technology. We can assume that Intel has done a more thorough analysis, using the appropriate tools and expertise, and that Larrabee was carefully designed for its market.
While Larrabee is not primitive, we can expect Intel to evolve the design over time. It’s important that any effort in porting and tuning codes for Larrabee not be wasted on future processor generations. That means future Larrabee-family processors should be software-compatible with the current processor; Intel knows this, having made a living from compatibility over the past 30 years. So can we guess how Larrabee will evolve over the years? Will anything change beyond core count?
The table below lists my guess at seven performance parameters for Larrabee that might change in future versions:
|instruction issue width||2||4?|
|cache size per core||320KB||640KB?|
core count: Increasing the core count would seem to be the easiest change, and we can assume that as technology improves, there will be room for more cores. Of course, those extra cores will be hungry for more memory bandwidth.
multithread count: The multithread count is currently set at four, enough to tolerate L1 cache misses. This is exposed in the programming model, as we discuss in the next section, and highly-tuned applications take advantage of it. That makes it harder to design applications optimized for Larrabee that will also perform well on a successor with a different multithread count. I predict the multithread count could increase, but probably won’t, or only by a factor of two.
vector width: Might there be a successor that supports the Larrabee vector width as well as longer vectors? A better solution would have been to add a true vector instruction set, letting the software detect the maximum hardware vector length at runtime, so an application can run on processors with longer or shorter maximum vector lengths. Instead, Intel stuck with the packed operand paradigm, although allowing for indexed fetch and store. I think Intel missed an important opportunity here. Nevertheless, longer vectors are an easy way to add to peak performance.
issue width: One of the design decisions for Larrabee was to simplify the control unit, reducing the instruction issue width. Why would anyone go back to more complex control units? Recall the original RISC argument: simplify the control unit and use those resources for other purposes. The entire RISC argument was technology based: given a limited transistor budget, how best to use it; the original Berkeley RISC processor was about 45K transistors. Now look at current RISC successors (PowerPC, Sparc): complex control units, superscalar instruction issue, out-of-order and speculative execution. Given a much larger transistor budget, the control unit decision can be reconsidered. Larrabee successors may similarly be pushed towards more complex control, allowing more instruction-level parallelism within the core.
cache size: In the CPU world, the cache memory is used to reduce accesses to main memory as much as possible; if the program and data fit mostly in the cache, the program will run much, much faster. Most high-end GPUs have limited size caches, instead focusing on high bandwidth memory designs to support very large data sets, for which caches are of less use. I predict that for Larrabee and successors, cache per core will be less important than the bandwidth to main memory.
clock rate: At the high end of the CPU world, clock rates seem to have topped out at about 5GHz; we’ve not seen any discussion of Larrabee clock rates, but early Larrabee processors will likely not be near that peak, so there is certainly some headroom there as well. If the memory bandwidth scales with the clock rate, this will give direct performance benefits.
memory model: At 32 cores, Larrabee can still support coherent shared memory. If we look at the history of shared memory multiprocessors, very few used shared memory beyond 32 processors. Large SGI systems have a shared address space up to thousands of processors, but it’s explicitly NUMA, and memory placement in the network with respect to the processors does affect performance. Can a single chip manycore system support uniform coherent shared memory for large core counts? Perhaps higher multithreading depths (see below) will make it feasible.
All of these will put pressure on the memory interface. More cores, faster cores or wider vectors require more data.
Memory and Multithreading
The original multithreading design, the Denelcor Heterogeneous Element Processor (HEP), dates back to 1978. Burton Smith, now at Microsoft, was the architect; his belief was that reducing memory latency was expensive and not general enough. Any solution is optimized for certain applications and will fail for others. In particular, for large datasets, the real problem is true memory bandwidth. However, if you have enough parallelism in your program, you can buy memory bandwidth relatively cheaply, and tolerate the latency by exploiting slack parallelism through multithreading.
While the HEP was not particularly commercially successful, multithreading lives on in current NVIDIA and ATI graphics processors. Both use high degrees of multithreading to tolerate long latencies to the graphics device memory, avoiding the need for data caches and locality in order to keep the processors busy and productive.
Larrabee uses limited multithreading (4 threads) to tolerate the latency of an L1 cache miss; it doesn’t have enough threads to tolerate a miss that has to go to memory. Because the four threads on a core share cache, it’s important that those threads work on shared data, as much as possible, to avoid cache thrashing. One way to optimize for this is to use a scout thread on each core to prefetch data to the cache, where it is processed by the other three threads.
This leaves open the critical question of the actual memory bandwidth to the Larrabee processor. High-end processors like Nehalem have very large cache memories; programs with data sets that fit in the cache can run quite fast. However, programs with very large data sets that have to be accessed from memory will see significant performance degredation, not only because of the relatively long latency to main memory, but because the basic memory bandwidth is less than that supported by the cache. If Larrabee is designed with a high bandwidth memory interface, it will be able to keep the cores busy, even if it has to expend some fraction of its threads preloading the cache for the other threads. However, if the Larrabee memory bandwidth can’t keep up with the demand from all the cores, then many HPC programs will likely work just as fast or faster on a high-end Nehalem, which has the huge L3 cache and faster single thread performance.
The Larrabee native programming model, as described by Intel, uses an extended POSIX Threads API with a lightweight task stealing scheduler. There is no hardware thread scheduling, no hardware synchronization registers, no significant hardware to support parallel applications beyond the coherent cache mechanism. This increases generality and flexibility greatly, but also does not address the cost of task creation and management. Larrabee applications will tend towards larger tasks, to amortize the task creation and scheduling costs. This is somewhat at odds with the need to fill 128 threads (32 cores x 4 threads/core) to get full performance. The dependence on cache locality makes thread affinity scheduling critical. We’ll have to wait and see how well the operating system works in this model.
We’re all eagerly awaiting the first real Larrabee deliveries and performance results; I wouldn’t be surprised to see something at SC’09. I may have missed important architectural details, so perhaps we’ll have to have a followup column when we have real hardware available. I’m sure we’ll see several early projects to move HPC applications onto Larrabee, comparing performance in particular to an NVIDIA GPU. NVIDIA just announced the architecture of its next generation Fermi GPU, with many features aimed specifically at the compute market. We can’t compare performance without actual hardware, but in my next column I’ll compare the architectural features of the Larrabee to those of the NVIDIA Tesla and Fermi, in the HPC space.
About the Author
Michael Wolfe has developed compilers for over 30 years in both academia and industry, and is now a senior compiler engineer at The Portland Group, Inc. (www.pgroup.com), a wholly-owned subsidiary of STMicroelectronics, Inc. The opinions stated here are those of the author, and do not represent opinions of The Portland Group, Inc. or STMicroelectronics, Inc.