Compilers and More: A Computing Larrabee

By Michael Wolfe

October 29, 2009

The buzz and excitement around Intel’s Larrabee processor continues to build. Intel has been careful to present Larrabee as a graphics processor that can also be used for highly parallel tasks such as game physics, avoiding the claim that it will be appropriate for HPC. The Intel marketing department isn’t stupid, of course; they don’t want to erode the market for their own very high-powered and successful (and highly profitable) Core-2 (and beyond) server processors. Nonetheless, it will be hard to prevent experimentation and even productization of HPC systems with Larrabee processors, either as the main CPU or as an attached accelerator, unless Intel chooses to control the supply.

I don’t claim to be competent to evaluate the Larrabee (or any other processor) for its graphics features or performance, but we can discuss Larrabee as a compute engine. Larrabee looks an awful lot like an x86 cluster node; anyone who has experience building HPC clusters has pretty good intuition about the design tradeoffs that make for a balanced and effective system. So it can be interesting to explore the Larrabee architecture, to look at the design choices Intel made, and what alternatives they might have considered or might consider in the future.

Larrabee Architecture

Let’s look at an abstraction of the Larrabee architecture. Larrabee supports several kinds of parallelism. It was recently stated that Larrabee will have 32 cores in its first implementation. That gives a MIMD parallelism factor of 32, 32 threads running in parallel on separate cores. Each core has a vector processing unit that augments the SSE instruction set; the VPU can support 16 single precision operations in vector mode. That gives a SIMD or vector parallelism factor of 16, assuming the vector instructions are implemented fully in SIMD mode. Most vector processors use pipelining to reduce the transistor count, but I doubt that Intel is limited by silicon real estate. For double precision, the vector width is cut in half to eight. Larrabee also supports four thread contexts per core; when one thread stalls on a level-1 cache miss, the core switches to one of the other thread contexts. This keeps the core busy while the cache is handling the miss. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, and a 256KB L2 unified cache. Intel describes the latter as a 256KB subset of a global 8MB L2 cache. Since each core accesses and stores data in its own 256KB L2 subset, perhaps someone will explain to me why this is different than 32 separate coherent 256KB L2 caches. The Larrabee control unit implements dual instruction issue in simple cases, depending on the compiler to schedule compatible adjacent instructions.
PGI Larrabee Block Diagram
Compare this to a quad-core Intel Nehalem processor. Nehalem has a MIMD parallelism factor of 4, the SSE width is also 4 (single precision), and it supports two simultaneous threads with Intel hyperthreading. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, 256KB L2 unified cache, and there is a large (8MB or more) shared L3 cache. What Nehalem has that Larrabee doesn’t, besides the huge L3 cache, is a very powerful control unit, allowing multiscalar execution (multiple instructions issued per clock), out-of-order execution (instructions can be reordered at execution time), and register renaming (many more hardware registers than the 16 available to the programmer, necessary to effectively support out-of-order execution). With hyperthreading enabled, a Nehalem can not only keep two thread contexts active, it can issue instructions from both threads simultaneously, in the same clock cycle. This helps to keep the functional units busy, using the instruction-level parallelism of both threads at the same time.

However, Nehalem’s power comes at a cost, namely that very large, complex, expensive control unit, and that complexity permeates the core design throughout the microarchitecture. The Larrabee goes back to a simpler time, apparently back to the original Pentium dual-issue, in-order control unit. This simplifies the microarchitecture, freeing up chip real estate for other purposes, in this case for more cores.

We could have a discussion here comparing the goals and solutions addressed by Larrabee (relative to Nehalem), to the goals and solutions addressed by RISC processors in the 1980s (relative to CISC, specifically Digital VAX and IBM mainframes). In both cases, designers simplified the control unit in order to free chip resources and expand other capabilities. For RISC, this meant a larger register file and cache, while for Larrabee, it means more cores. Both solutions depend on software to deliver performance; both RISC and Larrabee expect compilers to do a better job of instruction selection and scheduling, for instance. But let’s leave that discussion for another day.

Comparison: Larrabee vs. Nehalem

The feature scorecard between Larrabee and Nehalem looks like this:

Larrabee Nehalem
32 4 cores
64KB 64KB L1 cache/core
256KB 256KB L2 cache/core
  2048KB L3 cache/core
16 4 vector width (single-precision)
4 2 multithreading width
2 4 instruction issue width

With all those cores, how can Larrabee not outperform a Nehalem on highly parallel code? Just to get the same instruction issue rate as Larrabee, a Nehalem would need a clock rate about four times faster; to get the same vector operation bandwidth, Nehalem would need another factor of four. However, that depends on a fully-parallel application at full rate in both cases. Amdahl’s Law will apply, and Nehalem certainly screams on the sequential parts of the program. We’ll come back to this, but consider also the cache and memory interfaces.

Design Options

Designing a parallel system like Larrabee means making many choices and tradeoffs. What if we change some of the Larrabee parameters, for instance, reduce the vector width, and use those transistors for more cores? The tradeoff is that a vector unit is quite regular, relatively simple compared to a whole core. If we cut the vector instruction set back to SSE widths (four wide, single precision), and use those transistors instead for more cores, we might only get 1.5X or 2X more cores. Let’s be optimistic and assume 2X, and leave the rest of the design the same, as much as we can; in particular, let’s assume the total L2 cache size stays at 8MB total, but that we can add L1 cache for each new core. We’ll call this modified design Kurlabee, with the following feature scorecard:

Larrabee Kurlabee
32 64 cores
64KB 64KB L1 cache/core
256KB 128KB L2 cache/core
16 4 vector width (single-precision)
4 4 multithreading width
2 2 instruction issue width

For irregular parallel programs, Kurlabee has the advantage, with 64 cores, of keeping twice as many threads progressing. However, for highly parallel, regular (vector) code, Larrabee can generate 512 results per cycle (32X16), whereas Kurlabee can only generate 256. For the graphics market, where the applications are highly regular, Larrabee seems to a better design. Also keep in mind that the multithreading width is only four, which is designed to keep the core busy while tolerating an L1 cache miss; if there are many L2 cache misses, the core will likely stall. The Kurlabee design has a smaller L2 cache per core, so those extra cores will likely spend more time waiting on memory.

So, if longer vectors are good, why not make them even longer? What if we halve the number of cores, and use those resources for longer vector operations? As before, we’ll use a 2:1 ratio of vector length to core count, and keep the rest of the design relatively constant. We’ll call this design Moabee:

Larrabee Moabee
32 16 cores
64KB 64KB L1 cache/core
256KB 512KB L2 cache/core
16 64 vector width (single-precision)
4 4 multithreading width
2 2 instruction issue width

For those highly parallel, regular programs, Moabee can generate a stunning 1024 results per cycle, so it might have a strong advantage. However, there is the tradeoff for irregular codes, and while the vector length increased by a factor of four, the L2 cache size (per core) only doubled, and the L1 cache size stayed the same. In fact, the 32KB L1 data cache is only twice the size of the Moabee vector register file (16 vector registers X 64 elements X 4 bytes X 4 threads), so there could be serious cache pressure problems. Other issues with longer vectors are that the Larrabee doesn’t have a true vector instruction set; in particular, it doesn’t have a vector length register, allowing arbitrary-length short vectors. This means that unless the program or compiler knows that the application operations are an exact multiple of the vector register width, an extra copy of the loop must be generated to handle the remainder operations. Longer vector widths, as with Moabee, only exacerbate this problem.

Making design tradeoffs like this takes much more effort, thought, and application modeling than I’ve put into this discussion. I haven’t even looked at other tradeoffs, such as cores for cache, or supporting more simultaneous threads, and made gross assumptions about the technology. We can assume that Intel has done a more thorough analysis, using the appropriate tools and expertise, and that Larrabee was carefully designed for its market.

Evolution

While Larrabee is not primitive, we can expect Intel to evolve the design over time. It’s important that any effort in porting and tuning codes for Larrabee not be wasted on future processor generations. That means future Larrabee-family processors should be software-compatible with the current processor; Intel knows this, having made a living from compatibility over the past 30 years. So can we guess how Larrabee will evolve over the years? Will anything change beyond core count?

The table below lists my guess at seven performance parameters for Larrabee that might change in future versions:

  today future
core count 32 128?
multithread count 4 8?
vector width 16 64?
instruction issue width 2 4?
cache size per core 320KB 640KB?
clock rate xxGHz 2xxGHz?
memory model shared messages?

core count: Increasing the core count would seem to be the easiest change, and we can assume that as technology improves, there will be room for more cores. Of course, those extra cores will be hungry for more memory bandwidth.

multithread count: The multithread count is currently set at four, enough to tolerate L1 cache misses. This is exposed in the programming model, as we discuss in the next section, and highly-tuned applications take advantage of it. That makes it harder to design applications optimized for Larrabee that will also perform well on a successor with a different multithread count. I predict the multithread count could increase, but probably won’t, or only by a factor of two.

vector width: Might there be a successor that supports the Larrabee vector width as well as longer vectors? A better solution would have been to add a true vector instruction set, letting the software detect the maximum hardware vector length at runtime, so an application can run on processors with longer or shorter maximum vector lengths. Instead, Intel stuck with the packed operand paradigm, although allowing for indexed fetch and store. I think Intel missed an important opportunity here. Nevertheless, longer vectors are an easy way to add to peak performance.

issue width: One of the design decisions for Larrabee was to simplify the control unit, reducing the instruction issue width. Why would anyone go back to more complex control units? Recall the original RISC argument: simplify the control unit and use those resources for other purposes. The entire RISC argument was technology based: given a limited transistor budget, how best to use it; the original Berkeley RISC processor was about 45K transistors. Now look at current RISC successors (PowerPC, Sparc): complex control units, superscalar instruction issue, out-of-order and speculative execution. Given a much larger transistor budget, the control unit decision can be reconsidered. Larrabee successors may similarly be pushed towards more complex control, allowing more instruction-level parallelism within the core.

cache size: In the CPU world, the cache memory is used to reduce accesses to main memory as much as possible; if the program and data fit mostly in the cache, the program will run much, much faster. Most high-end GPUs have limited size caches, instead focusing on high bandwidth memory designs to support very large data sets, for which caches are of less use. I predict that for Larrabee and successors, cache per core will be less important than the bandwidth to main memory.

clock rate: At the high end of the CPU world, clock rates seem to have topped out at about 5GHz; we’ve not seen any discussion of Larrabee clock rates, but early Larrabee processors will likely not be near that peak, so there is certainly some headroom there as well. If the memory bandwidth scales with the clock rate, this will give direct performance benefits.

memory model: At 32 cores, Larrabee can still support coherent shared memory. If we look at the history of shared memory multiprocessors, very few used shared memory beyond 32 processors. Large SGI systems have a shared address space up to thousands of processors, but it’s explicitly NUMA, and memory placement in the network with respect to the processors does affect performance. Can a single chip manycore system support uniform coherent shared memory for large core counts? Perhaps higher multithreading depths (see below) will make it feasible.

All of these will put pressure on the memory interface. More cores, faster cores or wider vectors require more data.

Memory and Multithreading

The original multithreading design, the Denelcor Heterogeneous Element Processor (HEP), dates back to 1978. Burton Smith, now at Microsoft, was the architect; his belief was that reducing memory latency was expensive and not general enough. Any solution is optimized for certain applications and will fail for others. In particular, for large datasets, the real problem is true memory bandwidth. However, if you have enough parallelism in your program, you can buy memory bandwidth relatively cheaply, and tolerate the latency by exploiting slack parallelism through multithreading.

While the HEP was not particularly commercially successful, multithreading lives on in current NVIDIA and ATI graphics processors. Both use high degrees of multithreading to tolerate long latencies to the graphics device memory, avoiding the need for data caches and locality in order to keep the processors busy and productive.

Larrabee uses limited multithreading (4 threads) to tolerate the latency of an L1 cache miss; it doesn’t have enough threads to tolerate a miss that has to go to memory. Because the four threads on a core share cache, it’s important that those threads work on shared data, as much as possible, to avoid cache thrashing. One way to optimize for this is to use a scout thread on each core to prefetch data to the cache, where it is processed by the other three threads.

This leaves open the critical question of the actual memory bandwidth to the Larrabee processor. High-end processors like Nehalem have very large cache memories; programs with data sets that fit in the cache can run quite fast. However, programs with very large data sets that have to be accessed from memory will see significant performance degredation, not only because of the relatively long latency to main memory, but because the basic memory bandwidth is less than that supported by the cache. If Larrabee is designed with a high bandwidth memory interface, it will be able to keep the cores busy, even if it has to expend some fraction of its threads preloading the cache for the other threads. However, if the Larrabee memory bandwidth can’t keep up with the demand from all the cores, then many HPC programs will likely work just as fast or faster on a high-end Nehalem, which has the huge L3 cache and faster single thread performance.

Programming

The Larrabee native programming model, as described by Intel, uses an extended POSIX Threads API with a lightweight task stealing scheduler. There is no hardware thread scheduling, no hardware synchronization registers, no significant hardware to support parallel applications beyond the coherent cache mechanism. This increases generality and flexibility greatly, but also does not address the cost of task creation and management. Larrabee applications will tend towards larger tasks, to amortize the task creation and scheduling costs. This is somewhat at odds with the need to fill 128 threads (32 cores x 4 threads/core) to get full performance. The dependence on cache locality makes thread affinity scheduling critical. We’ll have to wait and see how well the operating system works in this model.

Summary

We’re all eagerly awaiting the first real Larrabee deliveries and performance results; I wouldn’t be surprised to see something at SC’09. I may have missed important architectural details, so perhaps we’ll have to have a followup column when we have real hardware available. I’m sure we’ll see several early projects to move HPC applications onto Larrabee, comparing performance in particular to an NVIDIA GPU. NVIDIA just announced the architecture of its next generation Fermi GPU, with many features aimed specifically at the compute market. We can’t compare performance without actual hardware, but in my next column I’ll compare the architectural features of the Larrabee to those of the NVIDIA Tesla and Fermi, in the HPC space.

About the Author

Michael Wolfe has developed compilers for over 30 years in both academia and industry, and is now a senior compiler engineer at The Portland Group, Inc. (www.pgroup.com), a wholly-owned subsidiary of STMicroelectronics, Inc. The opinions stated here are those of the author, and do not represent opinions of The Portland Group, Inc. or STMicroelectronics, Inc.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Machine Learning at HPC User Forum: Drilling into Specific Use Cases

September 22, 2017

The 66th HPC User Forum held September 5-7, in Milwaukee, Wisconsin, at the elegant and historic Pfister Hotel, highlighting the 1893 Victorian décor and art of “The Grand Hotel Of The West,” contrasted nicely with Read more…

By Arno Kolster

Google Cloud Makes Good on Promise to Add Nvidia P100 GPUs

September 21, 2017

Google has taken down the notice on its cloud platform website that says Nvidia Tesla P100s are “coming soon.” That's because the search giant has announced the beta launch of the high-end P100 Nvidia Tesla GPUs on t Read more…

By George Leopold

Cray Wins $48M Supercomputer Contract from KISTI

September 21, 2017

It was a good day for Cray which won a $48 million contract from the Korea Institute of Science and Technology Information (KISTI) for a 128-rack CS500 cluster supercomputer. The new system, equipped with Intel Xeon Scal Read more…

By John Russell

HPE Extreme Performance Solutions

HPE Prepares Customers for Success with the HPC Software Portfolio

High performance computing (HPC) software is key to harnessing the full power of HPC environments. Development and management tools enable IT departments to streamline installation and maintenance of their systems as well as create, optimize, and run their HPC applications. Read more…

Adolfy Hoisie to Lead Brookhaven’s Computing for National Security Effort

September 21, 2017

Brookhaven National Laboratory announced today that Adolfy Hoisie will chair its newly formed Computing for National Security department, which is part of Brookhaven’s new Computational Science Initiative (CSI). Read more…

By John Russell

Machine Learning at HPC User Forum: Drilling into Specific Use Cases

September 22, 2017

The 66th HPC User Forum held September 5-7, in Milwaukee, Wisconsin, at the elegant and historic Pfister Hotel, highlighting the 1893 Victorian décor and art o Read more…

By Arno Kolster

Stanford University and UberCloud Achieve Breakthrough in Living Heart Simulations

September 21, 2017

Cardiac arrhythmia can be an undesirable and potentially lethal side effect of drugs. During this condition, the electrical activity of the heart turns chaotic, Read more…

By Wolfgang Gentzsch, UberCloud, and Francisco Sahli, Stanford University

PNNL’s Center for Advanced Tech Evaluation Seeks Wider HPC Community Ties

September 21, 2017

Two years ago the Department of Energy established the Center for Advanced Technology Evaluation (CENATE) at Pacific Northwest National Laboratory (PNNL). CENAT Read more…

By John Russell

Exascale Computing Project Names Doug Kothe as Director

September 20, 2017

The Department of Energy’s Exascale Computing Project (ECP) has named Doug Kothe as its new director effective October 1. He replaces Paul Messina, who is stepping down after two years to return to Argonne National Laboratory. Kothe is a 32-year veteran of DOE’s National Laboratory System. Read more…

Takeaways from the Milwaukee HPC User Forum

September 19, 2017

Milwaukee’s elegant Pfister Hotel hosted approximately 100 attendees for the 66th HPC User Forum (September 5-7, 2017). In the original home city of Pabst Blu Read more…

By Merle Giles

Kathy Yelick Charts the Promise and Progress of Exascale Science

September 15, 2017

On Friday, Sept. 8, Kathy Yelick of Lawrence Berkeley National Laboratory and the University of California, Berkeley, delivered the keynote address on “Breakthrough Science at the Exascale” at the ACM Europe Conference in Barcelona. In conjunction with her presentation, Yelick agreed to a short Q&A discussion with HPCwire. Read more…

By Tiffany Trader

DARPA Pledges Another $300 Million for Post-Moore’s Readiness

September 14, 2017

The Defense Advanced Research Projects Agency (DARPA) launched a giant funding effort to ensure the United States can sustain the pace of electronic innovation vital to both a flourishing economy and a secure military. Under the banner of the Electronics Resurgence Initiative (ERI), some $500-$800 million will be invested in post-Moore’s Law technologies. Read more…

By Tiffany Trader

IBM Breaks Ground for Complex Quantum Chemistry

September 14, 2017

IBM has reported the use of a novel algorithm to simulate BeH2 (beryllium-hydride) on a quantum computer. This is the largest molecule so far simulated on a quantum computer. The technique, which used six qubits of a seven-qubit system, is an important step forward and may suggest an approach to simulating ever larger molecules. Read more…

By John Russell

How ‘Knights Mill’ Gets Its Deep Learning Flops

June 22, 2017

Intel, the subject of much speculation regarding the delayed, rewritten or potentially canceled “Aurora” contract (the Argonne Lab part of the CORAL “ Read more…

By Tiffany Trader

Reinders: “AVX-512 May Be a Hidden Gem” in Intel Xeon Scalable Processors

June 29, 2017

Imagine if we could use vector processing on something other than just floating point problems.  Today, GPUs and CPUs work tirelessly to accelerate algorithms Read more…

By James Reinders

NERSC Scales Scientific Deep Learning to 15 Petaflops

August 28, 2017

A collaborative effort between Intel, NERSC and Stanford has delivered the first 15-petaflops deep learning software running on HPC platforms and is, according Read more…

By Rob Farber

Oracle Layoffs Reportedly Hit SPARC and Solaris Hard

September 7, 2017

Oracle’s latest layoffs have many wondering if this is the end of the line for the SPARC processor and Solaris OS development. As reported by multiple sources Read more…

By John Russell

Six Exascale PathForward Vendors Selected; DoE Providing $258M

June 15, 2017

The much-anticipated PathForward awards for hardware R&D in support of the Exascale Computing Project were announced today with six vendors selected – AMD Read more…

By John Russell

Russian Researchers Claim First Quantum-Safe Blockchain

May 25, 2017

The Russian Quantum Center today announced it has overcome the threat of quantum cryptography by creating the first quantum-safe blockchain, securing cryptocurrencies like Bitcoin, along with classified government communications and other sensitive digital transfers. Read more…

By Doug Black

Top500 Results: Latest List Trends and What’s in Store

June 19, 2017

Greetings from Frankfurt and the 2017 International Supercomputing Conference where the latest Top500 list has just been revealed. Although there were no major Read more…

By Tiffany Trader

IBM Clears Path to 5nm with Silicon Nanosheets

June 5, 2017

Two years since announcing the industry’s first 7nm node test chip, IBM and its research alliance partners GlobalFoundries and Samsung have developed a proces Read more…

By Tiffany Trader

Leading Solution Providers

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Graphcore Readies Launch of 16nm Colossus-IPU Chip

July 20, 2017

A second $30 million funding round for U.K. AI chip developer Graphcore sets up the company to go to market with its “intelligent processing unit” (IPU) in Read more…

By Tiffany Trader

Google Debuts TPU v2 and will Add to Google Cloud

May 25, 2017

Not long after stirring attention in the deep learning/AI community by revealing the details of its Tensor Processing Unit (TPU), Google last week announced the Read more…

By John Russell

Google Releases Deeplearn.js to Further Democratize Machine Learning

August 17, 2017

Spreading the use of machine learning tools is one of the goals of Google’s PAIR (People + AI Research) initiative, which was introduced in early July. Last w Read more…

By John Russell

EU Funds 20 Million Euro ARM+FPGA Exascale Project

September 7, 2017

At the Barcelona Supercomputer Centre on Wednesday (Sept. 6), 16 partners gathered to launch the EuroEXA project, which invests €20 million over three-and-a-half years into exascale-focused research and development. Led by the Horizon 2020 program, EuroEXA picks up the banner of a triad of partner projects — ExaNeSt, EcoScale and ExaNoDe — building on their work... Read more…

By Tiffany Trader

Amazon Debuts New AMD-based GPU Instances for Graphics Acceleration

September 12, 2017

Last week Amazon Web Services (AWS) streaming service, AppStream 2.0, introduced a new GPU instance called Graphics Design intended to accelerate graphics. The Read more…

By John Russell

Cray Moves to Acquire the Seagate ClusterStor Line

July 28, 2017

This week Cray announced that it is picking up Seagate's ClusterStor HPC storage array business for an undisclosed sum. "In short we're effectively transitioning the bulk of the ClusterStor product line to Cray," said CEO Peter Ungaro. Read more…

By Tiffany Trader

GlobalFoundries: 7nm Chips Coming in 2018, EUV in 2019

June 13, 2017

GlobalFoundries has formally announced that its 7nm technology is ready for customer engagement with product tape outs expected for the first half of 2018. The Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This