Compilers and More: A Computing Larrabee

By Michael Wolfe

October 29, 2009

The buzz and excitement around Intel’s Larrabee processor continues to build. Intel has been careful to present Larrabee as a graphics processor that can also be used for highly parallel tasks such as game physics, avoiding the claim that it will be appropriate for HPC. The Intel marketing department isn’t stupid, of course; they don’t want to erode the market for their own very high-powered and successful (and highly profitable) Core-2 (and beyond) server processors. Nonetheless, it will be hard to prevent experimentation and even productization of HPC systems with Larrabee processors, either as the main CPU or as an attached accelerator, unless Intel chooses to control the supply.

I don’t claim to be competent to evaluate the Larrabee (or any other processor) for its graphics features or performance, but we can discuss Larrabee as a compute engine. Larrabee looks an awful lot like an x86 cluster node; anyone who has experience building HPC clusters has pretty good intuition about the design tradeoffs that make for a balanced and effective system. So it can be interesting to explore the Larrabee architecture, to look at the design choices Intel made, and what alternatives they might have considered or might consider in the future.

Larrabee Architecture

Let’s look at an abstraction of the Larrabee architecture. Larrabee supports several kinds of parallelism. It was recently stated that Larrabee will have 32 cores in its first implementation. That gives a MIMD parallelism factor of 32, 32 threads running in parallel on separate cores. Each core has a vector processing unit that augments the SSE instruction set; the VPU can support 16 single precision operations in vector mode. That gives a SIMD or vector parallelism factor of 16, assuming the vector instructions are implemented fully in SIMD mode. Most vector processors use pipelining to reduce the transistor count, but I doubt that Intel is limited by silicon real estate. For double precision, the vector width is cut in half to eight. Larrabee also supports four thread contexts per core; when one thread stalls on a level-1 cache miss, the core switches to one of the other thread contexts. This keeps the core busy while the cache is handling the miss. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, and a 256KB L2 unified cache. Intel describes the latter as a 256KB subset of a global 8MB L2 cache. Since each core accesses and stores data in its own 256KB L2 subset, perhaps someone will explain to me why this is different than 32 separate coherent 256KB L2 caches. The Larrabee control unit implements dual instruction issue in simple cases, depending on the compiler to schedule compatible adjacent instructions.
PGI Larrabee Block Diagram
Compare this to a quad-core Intel Nehalem processor. Nehalem has a MIMD parallelism factor of 4, the SSE width is also 4 (single precision), and it supports two simultaneous threads with Intel hyperthreading. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, 256KB L2 unified cache, and there is a large (8MB or more) shared L3 cache. What Nehalem has that Larrabee doesn’t, besides the huge L3 cache, is a very powerful control unit, allowing multiscalar execution (multiple instructions issued per clock), out-of-order execution (instructions can be reordered at execution time), and register renaming (many more hardware registers than the 16 available to the programmer, necessary to effectively support out-of-order execution). With hyperthreading enabled, a Nehalem can not only keep two thread contexts active, it can issue instructions from both threads simultaneously, in the same clock cycle. This helps to keep the functional units busy, using the instruction-level parallelism of both threads at the same time.

However, Nehalem’s power comes at a cost, namely that very large, complex, expensive control unit, and that complexity permeates the core design throughout the microarchitecture. The Larrabee goes back to a simpler time, apparently back to the original Pentium dual-issue, in-order control unit. This simplifies the microarchitecture, freeing up chip real estate for other purposes, in this case for more cores.

We could have a discussion here comparing the goals and solutions addressed by Larrabee (relative to Nehalem), to the goals and solutions addressed by RISC processors in the 1980s (relative to CISC, specifically Digital VAX and IBM mainframes). In both cases, designers simplified the control unit in order to free chip resources and expand other capabilities. For RISC, this meant a larger register file and cache, while for Larrabee, it means more cores. Both solutions depend on software to deliver performance; both RISC and Larrabee expect compilers to do a better job of instruction selection and scheduling, for instance. But let’s leave that discussion for another day.

Comparison: Larrabee vs. Nehalem

The feature scorecard between Larrabee and Nehalem looks like this:

Larrabee Nehalem
32 4 cores
64KB 64KB L1 cache/core
256KB 256KB L2 cache/core
  2048KB L3 cache/core
16 4 vector width (single-precision)
4 2 multithreading width
2 4 instruction issue width

With all those cores, how can Larrabee not outperform a Nehalem on highly parallel code? Just to get the same instruction issue rate as Larrabee, a Nehalem would need a clock rate about four times faster; to get the same vector operation bandwidth, Nehalem would need another factor of four. However, that depends on a fully-parallel application at full rate in both cases. Amdahl’s Law will apply, and Nehalem certainly screams on the sequential parts of the program. We’ll come back to this, but consider also the cache and memory interfaces.

Design Options

Designing a parallel system like Larrabee means making many choices and tradeoffs. What if we change some of the Larrabee parameters, for instance, reduce the vector width, and use those transistors for more cores? The tradeoff is that a vector unit is quite regular, relatively simple compared to a whole core. If we cut the vector instruction set back to SSE widths (four wide, single precision), and use those transistors instead for more cores, we might only get 1.5X or 2X more cores. Let’s be optimistic and assume 2X, and leave the rest of the design the same, as much as we can; in particular, let’s assume the total L2 cache size stays at 8MB total, but that we can add L1 cache for each new core. We’ll call this modified design Kurlabee, with the following feature scorecard:

Larrabee Kurlabee
32 64 cores
64KB 64KB L1 cache/core
256KB 128KB L2 cache/core
16 4 vector width (single-precision)
4 4 multithreading width
2 2 instruction issue width

For irregular parallel programs, Kurlabee has the advantage, with 64 cores, of keeping twice as many threads progressing. However, for highly parallel, regular (vector) code, Larrabee can generate 512 results per cycle (32X16), whereas Kurlabee can only generate 256. For the graphics market, where the applications are highly regular, Larrabee seems to a better design. Also keep in mind that the multithreading width is only four, which is designed to keep the core busy while tolerating an L1 cache miss; if there are many L2 cache misses, the core will likely stall. The Kurlabee design has a smaller L2 cache per core, so those extra cores will likely spend more time waiting on memory.

So, if longer vectors are good, why not make them even longer? What if we halve the number of cores, and use those resources for longer vector operations? As before, we’ll use a 2:1 ratio of vector length to core count, and keep the rest of the design relatively constant. We’ll call this design Moabee:

Larrabee Moabee
32 16 cores
64KB 64KB L1 cache/core
256KB 512KB L2 cache/core
16 64 vector width (single-precision)
4 4 multithreading width
2 2 instruction issue width

For those highly parallel, regular programs, Moabee can generate a stunning 1024 results per cycle, so it might have a strong advantage. However, there is the tradeoff for irregular codes, and while the vector length increased by a factor of four, the L2 cache size (per core) only doubled, and the L1 cache size stayed the same. In fact, the 32KB L1 data cache is only twice the size of the Moabee vector register file (16 vector registers X 64 elements X 4 bytes X 4 threads), so there could be serious cache pressure problems. Other issues with longer vectors are that the Larrabee doesn’t have a true vector instruction set; in particular, it doesn’t have a vector length register, allowing arbitrary-length short vectors. This means that unless the program or compiler knows that the application operations are an exact multiple of the vector register width, an extra copy of the loop must be generated to handle the remainder operations. Longer vector widths, as with Moabee, only exacerbate this problem.

Making design tradeoffs like this takes much more effort, thought, and application modeling than I’ve put into this discussion. I haven’t even looked at other tradeoffs, such as cores for cache, or supporting more simultaneous threads, and made gross assumptions about the technology. We can assume that Intel has done a more thorough analysis, using the appropriate tools and expertise, and that Larrabee was carefully designed for its market.

Evolution

While Larrabee is not primitive, we can expect Intel to evolve the design over time. It’s important that any effort in porting and tuning codes for Larrabee not be wasted on future processor generations. That means future Larrabee-family processors should be software-compatible with the current processor; Intel knows this, having made a living from compatibility over the past 30 years. So can we guess how Larrabee will evolve over the years? Will anything change beyond core count?

The table below lists my guess at seven performance parameters for Larrabee that might change in future versions:

  today future
core count 32 128?
multithread count 4 8?
vector width 16 64?
instruction issue width 2 4?
cache size per core 320KB 640KB?
clock rate xxGHz 2xxGHz?
memory model shared messages?

core count: Increasing the core count would seem to be the easiest change, and we can assume that as technology improves, there will be room for more cores. Of course, those extra cores will be hungry for more memory bandwidth.

multithread count: The multithread count is currently set at four, enough to tolerate L1 cache misses. This is exposed in the programming model, as we discuss in the next section, and highly-tuned applications take advantage of it. That makes it harder to design applications optimized for Larrabee that will also perform well on a successor with a different multithread count. I predict the multithread count could increase, but probably won’t, or only by a factor of two.

vector width: Might there be a successor that supports the Larrabee vector width as well as longer vectors? A better solution would have been to add a true vector instruction set, letting the software detect the maximum hardware vector length at runtime, so an application can run on processors with longer or shorter maximum vector lengths. Instead, Intel stuck with the packed operand paradigm, although allowing for indexed fetch and store. I think Intel missed an important opportunity here. Nevertheless, longer vectors are an easy way to add to peak performance.

issue width: One of the design decisions for Larrabee was to simplify the control unit, reducing the instruction issue width. Why would anyone go back to more complex control units? Recall the original RISC argument: simplify the control unit and use those resources for other purposes. The entire RISC argument was technology based: given a limited transistor budget, how best to use it; the original Berkeley RISC processor was about 45K transistors. Now look at current RISC successors (PowerPC, Sparc): complex control units, superscalar instruction issue, out-of-order and speculative execution. Given a much larger transistor budget, the control unit decision can be reconsidered. Larrabee successors may similarly be pushed towards more complex control, allowing more instruction-level parallelism within the core.

cache size: In the CPU world, the cache memory is used to reduce accesses to main memory as much as possible; if the program and data fit mostly in the cache, the program will run much, much faster. Most high-end GPUs have limited size caches, instead focusing on high bandwidth memory designs to support very large data sets, for which caches are of less use. I predict that for Larrabee and successors, cache per core will be less important than the bandwidth to main memory.

clock rate: At the high end of the CPU world, clock rates seem to have topped out at about 5GHz; we’ve not seen any discussion of Larrabee clock rates, but early Larrabee processors will likely not be near that peak, so there is certainly some headroom there as well. If the memory bandwidth scales with the clock rate, this will give direct performance benefits.

memory model: At 32 cores, Larrabee can still support coherent shared memory. If we look at the history of shared memory multiprocessors, very few used shared memory beyond 32 processors. Large SGI systems have a shared address space up to thousands of processors, but it’s explicitly NUMA, and memory placement in the network with respect to the processors does affect performance. Can a single chip manycore system support uniform coherent shared memory for large core counts? Perhaps higher multithreading depths (see below) will make it feasible.

All of these will put pressure on the memory interface. More cores, faster cores or wider vectors require more data.

Memory and Multithreading

The original multithreading design, the Denelcor Heterogeneous Element Processor (HEP), dates back to 1978. Burton Smith, now at Microsoft, was the architect; his belief was that reducing memory latency was expensive and not general enough. Any solution is optimized for certain applications and will fail for others. In particular, for large datasets, the real problem is true memory bandwidth. However, if you have enough parallelism in your program, you can buy memory bandwidth relatively cheaply, and tolerate the latency by exploiting slack parallelism through multithreading.

While the HEP was not particularly commercially successful, multithreading lives on in current NVIDIA and ATI graphics processors. Both use high degrees of multithreading to tolerate long latencies to the graphics device memory, avoiding the need for data caches and locality in order to keep the processors busy and productive.

Larrabee uses limited multithreading (4 threads) to tolerate the latency of an L1 cache miss; it doesn’t have enough threads to tolerate a miss that has to go to memory. Because the four threads on a core share cache, it’s important that those threads work on shared data, as much as possible, to avoid cache thrashing. One way to optimize for this is to use a scout thread on each core to prefetch data to the cache, where it is processed by the other three threads.

This leaves open the critical question of the actual memory bandwidth to the Larrabee processor. High-end processors like Nehalem have very large cache memories; programs with data sets that fit in the cache can run quite fast. However, programs with very large data sets that have to be accessed from memory will see significant performance degredation, not only because of the relatively long latency to main memory, but because the basic memory bandwidth is less than that supported by the cache. If Larrabee is designed with a high bandwidth memory interface, it will be able to keep the cores busy, even if it has to expend some fraction of its threads preloading the cache for the other threads. However, if the Larrabee memory bandwidth can’t keep up with the demand from all the cores, then many HPC programs will likely work just as fast or faster on a high-end Nehalem, which has the huge L3 cache and faster single thread performance.

Programming

The Larrabee native programming model, as described by Intel, uses an extended POSIX Threads API with a lightweight task stealing scheduler. There is no hardware thread scheduling, no hardware synchronization registers, no significant hardware to support parallel applications beyond the coherent cache mechanism. This increases generality and flexibility greatly, but also does not address the cost of task creation and management. Larrabee applications will tend towards larger tasks, to amortize the task creation and scheduling costs. This is somewhat at odds with the need to fill 128 threads (32 cores x 4 threads/core) to get full performance. The dependence on cache locality makes thread affinity scheduling critical. We’ll have to wait and see how well the operating system works in this model.

Summary

We’re all eagerly awaiting the first real Larrabee deliveries and performance results; I wouldn’t be surprised to see something at SC’09. I may have missed important architectural details, so perhaps we’ll have to have a followup column when we have real hardware available. I’m sure we’ll see several early projects to move HPC applications onto Larrabee, comparing performance in particular to an NVIDIA GPU. NVIDIA just announced the architecture of its next generation Fermi GPU, with many features aimed specifically at the compute market. We can’t compare performance without actual hardware, but in my next column I’ll compare the architectural features of the Larrabee to those of the NVIDIA Tesla and Fermi, in the HPC space.

About the Author

Michael Wolfe has developed compilers for over 30 years in both academia and industry, and is now a senior compiler engineer at The Portland Group, Inc. (www.pgroup.com), a wholly-owned subsidiary of STMicroelectronics, Inc. The opinions stated here are those of the author, and do not represent opinions of The Portland Group, Inc. or STMicroelectronics, Inc.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Russian Researchers Claim First Quantum-Safe Blockchain

May 25, 2017

The Russian Quantum Center today announced it has overcome the threat of quantum cryptography by creating the first quantum-safe blockchain, securing cryptocurr Read more…

By Doug Black

Google Debuts TPU v2 and will Add to Google Cloud

May 25, 2017

Not long after stirring attention in the deep learning/AI community by revealing the details of its Tensor Processing Unit (TPU), Google last week announced the Read more…

By John Russell

Nvidia CEO Predicts AI ‘Cambrian Explosion’

May 25, 2017

The processing power and cloud access to developer tools used to train machine-learning models are making artificial intelligence ubiquitous across computing pl Read more…

By George Leopold

PGAS Use will Rise on New H/W Trends, Says Reinders

May 25, 2017

If you have not already tried using PGAS, it is time to consider adding PGAS to the programming techniques you know. Partitioned Global Array Space, commonly kn Read more…

By James Reinders

HPE Extreme Performance Solutions

Exploring the Three Models of Remote Visualization

The explosion of data and advancement of digital technologies are dramatically changing the way many companies do business. With the help of high performance computing (HPC) solutions and data analytics platforms, manufacturers are developing products faster, healthcare providers are improving patient care, and energy companies are improving planning, exploration, and production. Read more…

Exascale Escapes 2018 Budget Axe; Rest of Science Suffers

May 23, 2017

President Trump's proposed $4.1 trillion FY 2018 budget is good for U.S. exascale computing development, but grim for the rest of science and technology spend Read more…

By Tiffany Trader

Hedge Funds (with Supercomputing help) Rank First Among Investors

May 22, 2017

In case you didn’t know, The Quants Run Wall Street Now, or so says a headline in today’s Wall Street Journal. Quant-run hedge funds now control the largest Read more…

By John Russell

IBM, D-Wave Report Quantum Computing Advances

May 18, 2017

IBM said this week it has built and tested a pair of quantum computing processors, including a prototype of a commercial version. That progress follows an an Read more…

By George Leopold

PRACEdays 2017 Wraps Up in Barcelona

May 18, 2017

Barcelona has been absolutely lovely; the weather, the food, the people. I am, sadly, finishing my last day at PRACEdays 2017 with two sessions: an in-depth loo Read more…

By Kim McMahon

PGAS Use will Rise on New H/W Trends, Says Reinders

May 25, 2017

If you have not already tried using PGAS, it is time to consider adding PGAS to the programming techniques you know. Partitioned Global Array Space, commonly kn Read more…

By James Reinders

Exascale Escapes 2018 Budget Axe; Rest of Science Suffers

May 23, 2017

President Trump's proposed $4.1 trillion FY 2018 budget is good for U.S. exascale computing development, but grim for the rest of science and technology spend Read more…

By Tiffany Trader

Cray Offers Supercomputing as a Service, Targets Biotechs First

May 16, 2017

Leading supercomputer vendor Cray and datacenter/cloud provider the Markley Group today announced plans to jointly deliver supercomputing as a service. The init Read more…

By John Russell

HPE’s Memory-centric The Machine Coming into View, Opens ARMs to 3rd-party Developers

May 16, 2017

Announced three years ago, HPE’s The Machine is said to be the largest R&D program in the venerable company’s history, one that could be progressing tow Read more…

By Doug Black

What’s Up with Hyperion as It Transitions From IDC?

May 15, 2017

If you’re wondering what’s happening with Hyperion Research – formerly the IDC HPC group – apparently you are not alone, says Steve Conway, now senior V Read more…

By John Russell

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

HPE Launches Servers, Services, and Collaboration at GTC

May 10, 2017

Hewlett Packard Enterprise (HPE) today launched a new liquid cooled GPU-driven Apollo platform based on SGI ICE architecture, a new collaboration with NVIDIA, a Read more…

By John Russell

IBM PowerAI Tools Aim to Ease Deep Learning Data Prep, Shorten Training 

May 10, 2017

A new set of GPU-powered AI software announced by IBM today brings automation to many of the tedious, time consuming and complex aspects of AI project on-rampin Read more…

By Doug Black

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Just how close real-wo Read more…

By John Russell

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the cam Read more…

By John Russell

Google Pulls Back the Covers on Its First Machine Learning Chip

April 6, 2017

This week Google released a report detailing the design and performance characteristics of the Tensor Processing Unit (TPU), its custom ASIC for the inference Read more…

By Tiffany Trader

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its a Read more…

By Tiffany Trader

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Last week, Google reported that its custom ASIC Tensor Processing Unit (TPU) was 15-30x faster for inferencing workloads than Nvidia's K80 GPU (see our coverage Read more…

By Tiffany Trader

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

Since our first formal product releases of OSPRay and OpenSWR libraries in 2016, CPU-based Software Defined Visualization (SDVis) has achieved wide-spread adopt Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a ne Read more…

By Tiffany Trader

Nvidia’s Mammoth Volta GPU Aims High for AI, HPC

May 10, 2017

At Nvidia's GPU Technology Conference (GTC17) in San Jose, Calif., this morning, CEO Jensen Huang announced the company's much-anticipated Volta architecture a Read more…

By Tiffany Trader

Leading Solution Providers

Facebook Open Sources Caffe2; Nvidia, Intel Rush to Optimize

April 18, 2017

From its F8 developer conference in San Jose, Calif., today, Facebook announced Caffe2, a new open-source, cross-platform framework for deep learning. Caffe2 is Read more…

By Tiffany Trader

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which w Read more…

By Tiffany Trader

Is Liquid Cooling Ready to Go Mainstream?

February 13, 2017

Lost in the frenzy of SC16 was a substantial rise in the number of vendors showing server oriented liquid cooling technologies. Three decades ago liquid cooling Read more…

By Steve Campbell

MIT Mathematician Spins Up 220,000-Core Google Compute Cluster

April 21, 2017

On Thursday, Google announced that MIT math professor and computational number theorist Andrew V. Sutherland had set a record for the largest Google Compute Eng Read more…

By Tiffany Trader

IBM Wants to be “Red Hat” of Deep Learning

January 26, 2017

IBM today announced the addition of TensorFlow and Chainer deep learning frameworks to its PowerAI suite of deep learning tools, which already includes popular Read more…

By John Russell

US Supercomputing Leaders Tackle the China Question

March 15, 2017

As China continues to prove its supercomputing mettle via the Top500 list and the forward march of its ambitious plans to stand up an exascale machine by 2020, Read more…

By Tiffany Trader

HPC Technique Propels Deep Learning at Scale

February 21, 2017

Researchers from Baidu's Silicon Valley AI Lab (SVAIL) have adapted a well-known HPC communication technique to boost the speed and scale of their neural networ Read more…

By Tiffany Trader

DOE Supercomputer Achieves Record 45-Qubit Quantum Simulation

April 13, 2017

In order to simulate larger and larger quantum systems and usher in an age of "quantum supremacy," researchers are stretching the limits of today's most advance Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This