Compilers and More: A Computing Larrabee

By Michael Wolfe

October 29, 2009

The buzz and excitement around Intel’s Larrabee processor continues to build. Intel has been careful to present Larrabee as a graphics processor that can also be used for highly parallel tasks such as game physics, avoiding the claim that it will be appropriate for HPC. The Intel marketing department isn’t stupid, of course; they don’t want to erode the market for their own very high-powered and successful (and highly profitable) Core-2 (and beyond) server processors. Nonetheless, it will be hard to prevent experimentation and even productization of HPC systems with Larrabee processors, either as the main CPU or as an attached accelerator, unless Intel chooses to control the supply.

I don’t claim to be competent to evaluate the Larrabee (or any other processor) for its graphics features or performance, but we can discuss Larrabee as a compute engine. Larrabee looks an awful lot like an x86 cluster node; anyone who has experience building HPC clusters has pretty good intuition about the design tradeoffs that make for a balanced and effective system. So it can be interesting to explore the Larrabee architecture, to look at the design choices Intel made, and what alternatives they might have considered or might consider in the future.

Larrabee Architecture

Let’s look at an abstraction of the Larrabee architecture. Larrabee supports several kinds of parallelism. It was recently stated that Larrabee will have 32 cores in its first implementation. That gives a MIMD parallelism factor of 32, 32 threads running in parallel on separate cores. Each core has a vector processing unit that augments the SSE instruction set; the VPU can support 16 single precision operations in vector mode. That gives a SIMD or vector parallelism factor of 16, assuming the vector instructions are implemented fully in SIMD mode. Most vector processors use pipelining to reduce the transistor count, but I doubt that Intel is limited by silicon real estate. For double precision, the vector width is cut in half to eight. Larrabee also supports four thread contexts per core; when one thread stalls on a level-1 cache miss, the core switches to one of the other thread contexts. This keeps the core busy while the cache is handling the miss. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, and a 256KB L2 unified cache. Intel describes the latter as a 256KB subset of a global 8MB L2 cache. Since each core accesses and stores data in its own 256KB L2 subset, perhaps someone will explain to me why this is different than 32 separate coherent 256KB L2 caches. The Larrabee control unit implements dual instruction issue in simple cases, depending on the compiler to schedule compatible adjacent instructions.
PGI Larrabee Block Diagram
Compare this to a quad-core Intel Nehalem processor. Nehalem has a MIMD parallelism factor of 4, the SSE width is also 4 (single precision), and it supports two simultaneous threads with Intel hyperthreading. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, 256KB L2 unified cache, and there is a large (8MB or more) shared L3 cache. What Nehalem has that Larrabee doesn’t, besides the huge L3 cache, is a very powerful control unit, allowing multiscalar execution (multiple instructions issued per clock), out-of-order execution (instructions can be reordered at execution time), and register renaming (many more hardware registers than the 16 available to the programmer, necessary to effectively support out-of-order execution). With hyperthreading enabled, a Nehalem can not only keep two thread contexts active, it can issue instructions from both threads simultaneously, in the same clock cycle. This helps to keep the functional units busy, using the instruction-level parallelism of both threads at the same time.

However, Nehalem’s power comes at a cost, namely that very large, complex, expensive control unit, and that complexity permeates the core design throughout the microarchitecture. The Larrabee goes back to a simpler time, apparently back to the original Pentium dual-issue, in-order control unit. This simplifies the microarchitecture, freeing up chip real estate for other purposes, in this case for more cores.

We could have a discussion here comparing the goals and solutions addressed by Larrabee (relative to Nehalem), to the goals and solutions addressed by RISC processors in the 1980s (relative to CISC, specifically Digital VAX and IBM mainframes). In both cases, designers simplified the control unit in order to free chip resources and expand other capabilities. For RISC, this meant a larger register file and cache, while for Larrabee, it means more cores. Both solutions depend on software to deliver performance; both RISC and Larrabee expect compilers to do a better job of instruction selection and scheduling, for instance. But let’s leave that discussion for another day.

Comparison: Larrabee vs. Nehalem

The feature scorecard between Larrabee and Nehalem looks like this:

Larrabee Nehalem
32 4 cores
64KB 64KB L1 cache/core
256KB 256KB L2 cache/core
  2048KB L3 cache/core
16 4 vector width (single-precision)
4 2 multithreading width
2 4 instruction issue width

With all those cores, how can Larrabee not outperform a Nehalem on highly parallel code? Just to get the same instruction issue rate as Larrabee, a Nehalem would need a clock rate about four times faster; to get the same vector operation bandwidth, Nehalem would need another factor of four. However, that depends on a fully-parallel application at full rate in both cases. Amdahl’s Law will apply, and Nehalem certainly screams on the sequential parts of the program. We’ll come back to this, but consider also the cache and memory interfaces.

Design Options

Designing a parallel system like Larrabee means making many choices and tradeoffs. What if we change some of the Larrabee parameters, for instance, reduce the vector width, and use those transistors for more cores? The tradeoff is that a vector unit is quite regular, relatively simple compared to a whole core. If we cut the vector instruction set back to SSE widths (four wide, single precision), and use those transistors instead for more cores, we might only get 1.5X or 2X more cores. Let’s be optimistic and assume 2X, and leave the rest of the design the same, as much as we can; in particular, let’s assume the total L2 cache size stays at 8MB total, but that we can add L1 cache for each new core. We’ll call this modified design Kurlabee, with the following feature scorecard:

Larrabee Kurlabee
32 64 cores
64KB 64KB L1 cache/core
256KB 128KB L2 cache/core
16 4 vector width (single-precision)
4 4 multithreading width
2 2 instruction issue width

For irregular parallel programs, Kurlabee has the advantage, with 64 cores, of keeping twice as many threads progressing. However, for highly parallel, regular (vector) code, Larrabee can generate 512 results per cycle (32X16), whereas Kurlabee can only generate 256. For the graphics market, where the applications are highly regular, Larrabee seems to a better design. Also keep in mind that the multithreading width is only four, which is designed to keep the core busy while tolerating an L1 cache miss; if there are many L2 cache misses, the core will likely stall. The Kurlabee design has a smaller L2 cache per core, so those extra cores will likely spend more time waiting on memory.

So, if longer vectors are good, why not make them even longer? What if we halve the number of cores, and use those resources for longer vector operations? As before, we’ll use a 2:1 ratio of vector length to core count, and keep the rest of the design relatively constant. We’ll call this design Moabee:

Larrabee Moabee
32 16 cores
64KB 64KB L1 cache/core
256KB 512KB L2 cache/core
16 64 vector width (single-precision)
4 4 multithreading width
2 2 instruction issue width

For those highly parallel, regular programs, Moabee can generate a stunning 1024 results per cycle, so it might have a strong advantage. However, there is the tradeoff for irregular codes, and while the vector length increased by a factor of four, the L2 cache size (per core) only doubled, and the L1 cache size stayed the same. In fact, the 32KB L1 data cache is only twice the size of the Moabee vector register file (16 vector registers X 64 elements X 4 bytes X 4 threads), so there could be serious cache pressure problems. Other issues with longer vectors are that the Larrabee doesn’t have a true vector instruction set; in particular, it doesn’t have a vector length register, allowing arbitrary-length short vectors. This means that unless the program or compiler knows that the application operations are an exact multiple of the vector register width, an extra copy of the loop must be generated to handle the remainder operations. Longer vector widths, as with Moabee, only exacerbate this problem.

Making design tradeoffs like this takes much more effort, thought, and application modeling than I’ve put into this discussion. I haven’t even looked at other tradeoffs, such as cores for cache, or supporting more simultaneous threads, and made gross assumptions about the technology. We can assume that Intel has done a more thorough analysis, using the appropriate tools and expertise, and that Larrabee was carefully designed for its market.

Evolution

While Larrabee is not primitive, we can expect Intel to evolve the design over time. It’s important that any effort in porting and tuning codes for Larrabee not be wasted on future processor generations. That means future Larrabee-family processors should be software-compatible with the current processor; Intel knows this, having made a living from compatibility over the past 30 years. So can we guess how Larrabee will evolve over the years? Will anything change beyond core count?

The table below lists my guess at seven performance parameters for Larrabee that might change in future versions:

  today future
core count 32 128?
multithread count 4 8?
vector width 16 64?
instruction issue width 2 4?
cache size per core 320KB 640KB?
clock rate xxGHz 2xxGHz?
memory model shared messages?

core count: Increasing the core count would seem to be the easiest change, and we can assume that as technology improves, there will be room for more cores. Of course, those extra cores will be hungry for more memory bandwidth.

multithread count: The multithread count is currently set at four, enough to tolerate L1 cache misses. This is exposed in the programming model, as we discuss in the next section, and highly-tuned applications take advantage of it. That makes it harder to design applications optimized for Larrabee that will also perform well on a successor with a different multithread count. I predict the multithread count could increase, but probably won’t, or only by a factor of two.

vector width: Might there be a successor that supports the Larrabee vector width as well as longer vectors? A better solution would have been to add a true vector instruction set, letting the software detect the maximum hardware vector length at runtime, so an application can run on processors with longer or shorter maximum vector lengths. Instead, Intel stuck with the packed operand paradigm, although allowing for indexed fetch and store. I think Intel missed an important opportunity here. Nevertheless, longer vectors are an easy way to add to peak performance.

issue width: One of the design decisions for Larrabee was to simplify the control unit, reducing the instruction issue width. Why would anyone go back to more complex control units? Recall the original RISC argument: simplify the control unit and use those resources for other purposes. The entire RISC argument was technology based: given a limited transistor budget, how best to use it; the original Berkeley RISC processor was about 45K transistors. Now look at current RISC successors (PowerPC, Sparc): complex control units, superscalar instruction issue, out-of-order and speculative execution. Given a much larger transistor budget, the control unit decision can be reconsidered. Larrabee successors may similarly be pushed towards more complex control, allowing more instruction-level parallelism within the core.

cache size: In the CPU world, the cache memory is used to reduce accesses to main memory as much as possible; if the program and data fit mostly in the cache, the program will run much, much faster. Most high-end GPUs have limited size caches, instead focusing on high bandwidth memory designs to support very large data sets, for which caches are of less use. I predict that for Larrabee and successors, cache per core will be less important than the bandwidth to main memory.

clock rate: At the high end of the CPU world, clock rates seem to have topped out at about 5GHz; we’ve not seen any discussion of Larrabee clock rates, but early Larrabee processors will likely not be near that peak, so there is certainly some headroom there as well. If the memory bandwidth scales with the clock rate, this will give direct performance benefits.

memory model: At 32 cores, Larrabee can still support coherent shared memory. If we look at the history of shared memory multiprocessors, very few used shared memory beyond 32 processors. Large SGI systems have a shared address space up to thousands of processors, but it’s explicitly NUMA, and memory placement in the network with respect to the processors does affect performance. Can a single chip manycore system support uniform coherent shared memory for large core counts? Perhaps higher multithreading depths (see below) will make it feasible.

All of these will put pressure on the memory interface. More cores, faster cores or wider vectors require more data.

Memory and Multithreading

The original multithreading design, the Denelcor Heterogeneous Element Processor (HEP), dates back to 1978. Burton Smith, now at Microsoft, was the architect; his belief was that reducing memory latency was expensive and not general enough. Any solution is optimized for certain applications and will fail for others. In particular, for large datasets, the real problem is true memory bandwidth. However, if you have enough parallelism in your program, you can buy memory bandwidth relatively cheaply, and tolerate the latency by exploiting slack parallelism through multithreading.

While the HEP was not particularly commercially successful, multithreading lives on in current NVIDIA and ATI graphics processors. Both use high degrees of multithreading to tolerate long latencies to the graphics device memory, avoiding the need for data caches and locality in order to keep the processors busy and productive.

Larrabee uses limited multithreading (4 threads) to tolerate the latency of an L1 cache miss; it doesn’t have enough threads to tolerate a miss that has to go to memory. Because the four threads on a core share cache, it’s important that those threads work on shared data, as much as possible, to avoid cache thrashing. One way to optimize for this is to use a scout thread on each core to prefetch data to the cache, where it is processed by the other three threads.

This leaves open the critical question of the actual memory bandwidth to the Larrabee processor. High-end processors like Nehalem have very large cache memories; programs with data sets that fit in the cache can run quite fast. However, programs with very large data sets that have to be accessed from memory will see significant performance degredation, not only because of the relatively long latency to main memory, but because the basic memory bandwidth is less than that supported by the cache. If Larrabee is designed with a high bandwidth memory interface, it will be able to keep the cores busy, even if it has to expend some fraction of its threads preloading the cache for the other threads. However, if the Larrabee memory bandwidth can’t keep up with the demand from all the cores, then many HPC programs will likely work just as fast or faster on a high-end Nehalem, which has the huge L3 cache and faster single thread performance.

Programming

The Larrabee native programming model, as described by Intel, uses an extended POSIX Threads API with a lightweight task stealing scheduler. There is no hardware thread scheduling, no hardware synchronization registers, no significant hardware to support parallel applications beyond the coherent cache mechanism. This increases generality and flexibility greatly, but also does not address the cost of task creation and management. Larrabee applications will tend towards larger tasks, to amortize the task creation and scheduling costs. This is somewhat at odds with the need to fill 128 threads (32 cores x 4 threads/core) to get full performance. The dependence on cache locality makes thread affinity scheduling critical. We’ll have to wait and see how well the operating system works in this model.

Summary

We’re all eagerly awaiting the first real Larrabee deliveries and performance results; I wouldn’t be surprised to see something at SC’09. I may have missed important architectural details, so perhaps we’ll have to have a followup column when we have real hardware available. I’m sure we’ll see several early projects to move HPC applications onto Larrabee, comparing performance in particular to an NVIDIA GPU. NVIDIA just announced the architecture of its next generation Fermi GPU, with many features aimed specifically at the compute market. We can’t compare performance without actual hardware, but in my next column I’ll compare the architectural features of the Larrabee to those of the NVIDIA Tesla and Fermi, in the HPC space.

About the Author

Michael Wolfe has developed compilers for over 30 years in both academia and industry, and is now a senior compiler engineer at The Portland Group, Inc. (www.pgroup.com), a wholly-owned subsidiary of STMicroelectronics, Inc. The opinions stated here are those of the author, and do not represent opinions of The Portland Group, Inc. or STMicroelectronics, Inc.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

NSF Project Sets Up First Machine Learning Cyberinfrastructure – CHASE-CI

July 25, 2017

Earlier this month, the National Science Foundation issued a $1 million grant to Larry Smarr, director of Calit2, and a group of his colleagues to create a community infrastructure in support of machine learning research Read more…

By John Russell

DARPA Continues Investment in Post-Moore’s Technologies

July 24, 2017

The U.S. military long ago ceded dominance in electronics innovation to Silicon Valley, the DoD-backed powerhouse that has driven microelectronic generation for decades. With Moore's Law clearly running out of steam, the Read more…

By George Leopold

Graphcore Readies Launch of 16nm Colossus-IPU Chip

July 20, 2017

A second $30 million funding round for U.K. AI chip developer Graphcore sets up the company to go to market with its “intelligent processing unit” (IPU) in 2017 with scale-up production for enterprise datacenters and Read more…

By Tiffany Trader

HPE Extreme Performance Solutions

HPE Servers Deliver High Performance Remote Visualization

Whether generating seismic simulations, locating new productive oil reservoirs, or constructing complex models of the earth’s subsurface, energy, oil, and gas (EO&G) is a highly data-driven industry. Read more…

Trinity Supercomputer’s Haswell and KNL Partitions Are Merged

July 19, 2017

Trinity supercomputer’s two partitions – one based on Intel Xeon Haswell processors and the other on Xeon Phi Knights Landing – have been fully integrated are now available for use on classified work in the Nationa Read more…

By HPCwire Staff

NSF Project Sets Up First Machine Learning Cyberinfrastructure – CHASE-CI

July 25, 2017

Earlier this month, the National Science Foundation issued a $1 million grant to Larry Smarr, director of Calit2, and a group of his colleagues to create a comm Read more…

By John Russell

Graphcore Readies Launch of 16nm Colossus-IPU Chip

July 20, 2017

A second $30 million funding round for U.K. AI chip developer Graphcore sets up the company to go to market with its “intelligent processing unit” (IPU) in Read more…

By Tiffany Trader

Fujitsu Continues HPC, AI Push

July 19, 2017

Summer is well under way, but the so-called summertime slowdown, linked with hot temperatures and longer vacations, does not seem to have impacted Fujitsu's out Read more…

By Tiffany Trader

Researchers Use DNA to Store and Retrieve Digital Movie

July 18, 2017

From abacus to pencil and paper to semiconductor chips, the technology of computing has always been an ever-changing target. The human brain is probably the com Read more…

By John Russell

The Exascale FY18 Budget – The Next Step

July 17, 2017

On July 12, 2017, the U.S. federal budget for its Exascale Computing Initiative (ECI) took its next step forward. On that day, the full Appropriations Committee Read more…

By Alex R. Larzelere

Women in HPC Luncheon Shines Light on Female-Friendly Hiring Practices

July 13, 2017

The second annual Women in HPC luncheon was held on June 20, 2017, during the International Supercomputing Conference in Frankfurt, Germany. The luncheon provid Read more…

By Tiffany Trader

Satellite Advances, NSF Computation Power Rapid Mapping of Earth’s Surface

July 13, 2017

New satellite technologies have completely changed the game in mapping and geographical data gathering, reducing costs and placing a new emphasis on time series Read more…

By Ken Chiacchia and Tiffany Jolley

Intel Skylake: Xeon Goes from Chip to Platform

July 13, 2017

With yesterday’s New York unveiling of the new “Skylake” Xeon Scalable processors, Intel made multiple runs at multiple competitive threats and strategic Read more…

By Doug Black

Google Pulls Back the Covers on Its First Machine Learning Chip

April 6, 2017

This week Google released a report detailing the design and performance characteristics of the Tensor Processing Unit (TPU), its custom ASIC for the inference Read more…

By Tiffany Trader

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Just how close real-wo Read more…

By John Russell

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its a Read more…

By Tiffany Trader

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the cam Read more…

By John Russell

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

In this contributed perspective piece, Intel’s Jim Jeffers makes the case that CPU-based visualization is now widely adopted and as such is no longer a contrarian view, but is rather an exascale requirement. Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

Facebook Open Sources Caffe2; Nvidia, Intel Rush to Optimize

April 18, 2017

From its F8 developer conference in San Jose, Calif., today, Facebook announced Caffe2, a new open-source, cross-platform framework for deep learning. Caffe2 is the successor to Caffe, the deep learning framework developed by Berkeley AI Research and community contributors. Read more…

By Tiffany Trader

Leading Solution Providers

How ‘Knights Mill’ Gets Its Deep Learning Flops

June 22, 2017

Intel, the subject of much speculation regarding the delayed, rewritten or potentially canceled “Aurora” contract (the Argonne Lab part of the CORAL “ Read more…

By Tiffany Trader

Reinders: “AVX-512 May Be a Hidden Gem” in Intel Xeon Scalable Processors

June 29, 2017

Imagine if we could use vector processing on something other than just floating point problems.  Today, GPUs and CPUs work tirelessly to accelerate algorithms Read more…

By James Reinders

Russian Researchers Claim First Quantum-Safe Blockchain

May 25, 2017

The Russian Quantum Center today announced it has overcome the threat of quantum cryptography by creating the first quantum-safe blockchain, securing cryptocurrencies like Bitcoin, along with classified government communications and other sensitive digital transfers. Read more…

By Doug Black

MIT Mathematician Spins Up 220,000-Core Google Compute Cluster

April 21, 2017

On Thursday, Google announced that MIT math professor and computational number theorist Andrew V. Sutherland had set a record for the largest Google Compute Engine (GCE) job. Sutherland ran the massive mathematics workload on 220,000 GCE cores using preemptible virtual machine instances. Read more…

By Tiffany Trader

Google Debuts TPU v2 and will Add to Google Cloud

May 25, 2017

Not long after stirring attention in the deep learning/AI community by revealing the details of its Tensor Processing Unit (TPU), Google last week announced the Read more…

By John Russell

Groq This: New AI Chips to Give GPUs a Run for Deep Learning Money

April 24, 2017

CPUs and GPUs, move over. Thanks to recent revelations surrounding Google’s new Tensor Processing Unit (TPU), the computing world appears to be on the cusp of Read more…

By Alex Woodie

Six Exascale PathForward Vendors Selected; DoE Providing $258M

June 15, 2017

The much-anticipated PathForward awards for hardware R&D in support of the Exascale Computing Project were announced today with six vendors selected – AMD Read more…

By John Russell

Top500 Results: Latest List Trends and What’s in Store

June 19, 2017

Greetings from Frankfurt and the 2017 International Supercomputing Conference where the latest Top500 list has just been revealed. Although there were no major Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This