Compilers and More: A Computing Larrabee

By Michael Wolfe

October 29, 2009

The buzz and excitement around Intel’s Larrabee processor continues to build. Intel has been careful to present Larrabee as a graphics processor that can also be used for highly parallel tasks such as game physics, avoiding the claim that it will be appropriate for HPC. The Intel marketing department isn’t stupid, of course; they don’t want to erode the market for their own very high-powered and successful (and highly profitable) Core-2 (and beyond) server processors. Nonetheless, it will be hard to prevent experimentation and even productization of HPC systems with Larrabee processors, either as the main CPU or as an attached accelerator, unless Intel chooses to control the supply.

I don’t claim to be competent to evaluate the Larrabee (or any other processor) for its graphics features or performance, but we can discuss Larrabee as a compute engine. Larrabee looks an awful lot like an x86 cluster node; anyone who has experience building HPC clusters has pretty good intuition about the design tradeoffs that make for a balanced and effective system. So it can be interesting to explore the Larrabee architecture, to look at the design choices Intel made, and what alternatives they might have considered or might consider in the future.

Larrabee Architecture

Let’s look at an abstraction of the Larrabee architecture. Larrabee supports several kinds of parallelism. It was recently stated that Larrabee will have 32 cores in its first implementation. That gives a MIMD parallelism factor of 32, 32 threads running in parallel on separate cores. Each core has a vector processing unit that augments the SSE instruction set; the VPU can support 16 single precision operations in vector mode. That gives a SIMD or vector parallelism factor of 16, assuming the vector instructions are implemented fully in SIMD mode. Most vector processors use pipelining to reduce the transistor count, but I doubt that Intel is limited by silicon real estate. For double precision, the vector width is cut in half to eight. Larrabee also supports four thread contexts per core; when one thread stalls on a level-1 cache miss, the core switches to one of the other thread contexts. This keeps the core busy while the cache is handling the miss. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, and a 256KB L2 unified cache. Intel describes the latter as a 256KB subset of a global 8MB L2 cache. Since each core accesses and stores data in its own 256KB L2 subset, perhaps someone will explain to me why this is different than 32 separate coherent 256KB L2 caches. The Larrabee control unit implements dual instruction issue in simple cases, depending on the compiler to schedule compatible adjacent instructions.
PGI Larrabee Block Diagram
Compare this to a quad-core Intel Nehalem processor. Nehalem has a MIMD parallelism factor of 4, the SSE width is also 4 (single precision), and it supports two simultaneous threads with Intel hyperthreading. Each core has a 32KB L1 instruction cache, 32KB L1 data cache, 256KB L2 unified cache, and there is a large (8MB or more) shared L3 cache. What Nehalem has that Larrabee doesn’t, besides the huge L3 cache, is a very powerful control unit, allowing multiscalar execution (multiple instructions issued per clock), out-of-order execution (instructions can be reordered at execution time), and register renaming (many more hardware registers than the 16 available to the programmer, necessary to effectively support out-of-order execution). With hyperthreading enabled, a Nehalem can not only keep two thread contexts active, it can issue instructions from both threads simultaneously, in the same clock cycle. This helps to keep the functional units busy, using the instruction-level parallelism of both threads at the same time.

However, Nehalem’s power comes at a cost, namely that very large, complex, expensive control unit, and that complexity permeates the core design throughout the microarchitecture. The Larrabee goes back to a simpler time, apparently back to the original Pentium dual-issue, in-order control unit. This simplifies the microarchitecture, freeing up chip real estate for other purposes, in this case for more cores.

We could have a discussion here comparing the goals and solutions addressed by Larrabee (relative to Nehalem), to the goals and solutions addressed by RISC processors in the 1980s (relative to CISC, specifically Digital VAX and IBM mainframes). In both cases, designers simplified the control unit in order to free chip resources and expand other capabilities. For RISC, this meant a larger register file and cache, while for Larrabee, it means more cores. Both solutions depend on software to deliver performance; both RISC and Larrabee expect compilers to do a better job of instruction selection and scheduling, for instance. But let’s leave that discussion for another day.

Comparison: Larrabee vs. Nehalem

The feature scorecard between Larrabee and Nehalem looks like this:

Larrabee Nehalem
32 4 cores
64KB 64KB L1 cache/core
256KB 256KB L2 cache/core
  2048KB L3 cache/core
16 4 vector width (single-precision)
4 2 multithreading width
2 4 instruction issue width

With all those cores, how can Larrabee not outperform a Nehalem on highly parallel code? Just to get the same instruction issue rate as Larrabee, a Nehalem would need a clock rate about four times faster; to get the same vector operation bandwidth, Nehalem would need another factor of four. However, that depends on a fully-parallel application at full rate in both cases. Amdahl’s Law will apply, and Nehalem certainly screams on the sequential parts of the program. We’ll come back to this, but consider also the cache and memory interfaces.

Design Options

Designing a parallel system like Larrabee means making many choices and tradeoffs. What if we change some of the Larrabee parameters, for instance, reduce the vector width, and use those transistors for more cores? The tradeoff is that a vector unit is quite regular, relatively simple compared to a whole core. If we cut the vector instruction set back to SSE widths (four wide, single precision), and use those transistors instead for more cores, we might only get 1.5X or 2X more cores. Let’s be optimistic and assume 2X, and leave the rest of the design the same, as much as we can; in particular, let’s assume the total L2 cache size stays at 8MB total, but that we can add L1 cache for each new core. We’ll call this modified design Kurlabee, with the following feature scorecard:

Larrabee Kurlabee
32 64 cores
64KB 64KB L1 cache/core
256KB 128KB L2 cache/core
16 4 vector width (single-precision)
4 4 multithreading width
2 2 instruction issue width

For irregular parallel programs, Kurlabee has the advantage, with 64 cores, of keeping twice as many threads progressing. However, for highly parallel, regular (vector) code, Larrabee can generate 512 results per cycle (32X16), whereas Kurlabee can only generate 256. For the graphics market, where the applications are highly regular, Larrabee seems to a better design. Also keep in mind that the multithreading width is only four, which is designed to keep the core busy while tolerating an L1 cache miss; if there are many L2 cache misses, the core will likely stall. The Kurlabee design has a smaller L2 cache per core, so those extra cores will likely spend more time waiting on memory.

So, if longer vectors are good, why not make them even longer? What if we halve the number of cores, and use those resources for longer vector operations? As before, we’ll use a 2:1 ratio of vector length to core count, and keep the rest of the design relatively constant. We’ll call this design Moabee:

Larrabee Moabee
32 16 cores
64KB 64KB L1 cache/core
256KB 512KB L2 cache/core
16 64 vector width (single-precision)
4 4 multithreading width
2 2 instruction issue width

For those highly parallel, regular programs, Moabee can generate a stunning 1024 results per cycle, so it might have a strong advantage. However, there is the tradeoff for irregular codes, and while the vector length increased by a factor of four, the L2 cache size (per core) only doubled, and the L1 cache size stayed the same. In fact, the 32KB L1 data cache is only twice the size of the Moabee vector register file (16 vector registers X 64 elements X 4 bytes X 4 threads), so there could be serious cache pressure problems. Other issues with longer vectors are that the Larrabee doesn’t have a true vector instruction set; in particular, it doesn’t have a vector length register, allowing arbitrary-length short vectors. This means that unless the program or compiler knows that the application operations are an exact multiple of the vector register width, an extra copy of the loop must be generated to handle the remainder operations. Longer vector widths, as with Moabee, only exacerbate this problem.

Making design tradeoffs like this takes much more effort, thought, and application modeling than I’ve put into this discussion. I haven’t even looked at other tradeoffs, such as cores for cache, or supporting more simultaneous threads, and made gross assumptions about the technology. We can assume that Intel has done a more thorough analysis, using the appropriate tools and expertise, and that Larrabee was carefully designed for its market.

Evolution

While Larrabee is not primitive, we can expect Intel to evolve the design over time. It’s important that any effort in porting and tuning codes for Larrabee not be wasted on future processor generations. That means future Larrabee-family processors should be software-compatible with the current processor; Intel knows this, having made a living from compatibility over the past 30 years. So can we guess how Larrabee will evolve over the years? Will anything change beyond core count?

The table below lists my guess at seven performance parameters for Larrabee that might change in future versions:

  today future
core count 32 128?
multithread count 4 8?
vector width 16 64?
instruction issue width 2 4?
cache size per core 320KB 640KB?
clock rate xxGHz 2xxGHz?
memory model shared messages?

core count: Increasing the core count would seem to be the easiest change, and we can assume that as technology improves, there will be room for more cores. Of course, those extra cores will be hungry for more memory bandwidth.

multithread count: The multithread count is currently set at four, enough to tolerate L1 cache misses. This is exposed in the programming model, as we discuss in the next section, and highly-tuned applications take advantage of it. That makes it harder to design applications optimized for Larrabee that will also perform well on a successor with a different multithread count. I predict the multithread count could increase, but probably won’t, or only by a factor of two.

vector width: Might there be a successor that supports the Larrabee vector width as well as longer vectors? A better solution would have been to add a true vector instruction set, letting the software detect the maximum hardware vector length at runtime, so an application can run on processors with longer or shorter maximum vector lengths. Instead, Intel stuck with the packed operand paradigm, although allowing for indexed fetch and store. I think Intel missed an important opportunity here. Nevertheless, longer vectors are an easy way to add to peak performance.

issue width: One of the design decisions for Larrabee was to simplify the control unit, reducing the instruction issue width. Why would anyone go back to more complex control units? Recall the original RISC argument: simplify the control unit and use those resources for other purposes. The entire RISC argument was technology based: given a limited transistor budget, how best to use it; the original Berkeley RISC processor was about 45K transistors. Now look at current RISC successors (PowerPC, Sparc): complex control units, superscalar instruction issue, out-of-order and speculative execution. Given a much larger transistor budget, the control unit decision can be reconsidered. Larrabee successors may similarly be pushed towards more complex control, allowing more instruction-level parallelism within the core.

cache size: In the CPU world, the cache memory is used to reduce accesses to main memory as much as possible; if the program and data fit mostly in the cache, the program will run much, much faster. Most high-end GPUs have limited size caches, instead focusing on high bandwidth memory designs to support very large data sets, for which caches are of less use. I predict that for Larrabee and successors, cache per core will be less important than the bandwidth to main memory.

clock rate: At the high end of the CPU world, clock rates seem to have topped out at about 5GHz; we’ve not seen any discussion of Larrabee clock rates, but early Larrabee processors will likely not be near that peak, so there is certainly some headroom there as well. If the memory bandwidth scales with the clock rate, this will give direct performance benefits.

memory model: At 32 cores, Larrabee can still support coherent shared memory. If we look at the history of shared memory multiprocessors, very few used shared memory beyond 32 processors. Large SGI systems have a shared address space up to thousands of processors, but it’s explicitly NUMA, and memory placement in the network with respect to the processors does affect performance. Can a single chip manycore system support uniform coherent shared memory for large core counts? Perhaps higher multithreading depths (see below) will make it feasible.

All of these will put pressure on the memory interface. More cores, faster cores or wider vectors require more data.

Memory and Multithreading

The original multithreading design, the Denelcor Heterogeneous Element Processor (HEP), dates back to 1978. Burton Smith, now at Microsoft, was the architect; his belief was that reducing memory latency was expensive and not general enough. Any solution is optimized for certain applications and will fail for others. In particular, for large datasets, the real problem is true memory bandwidth. However, if you have enough parallelism in your program, you can buy memory bandwidth relatively cheaply, and tolerate the latency by exploiting slack parallelism through multithreading.

While the HEP was not particularly commercially successful, multithreading lives on in current NVIDIA and ATI graphics processors. Both use high degrees of multithreading to tolerate long latencies to the graphics device memory, avoiding the need for data caches and locality in order to keep the processors busy and productive.

Larrabee uses limited multithreading (4 threads) to tolerate the latency of an L1 cache miss; it doesn’t have enough threads to tolerate a miss that has to go to memory. Because the four threads on a core share cache, it’s important that those threads work on shared data, as much as possible, to avoid cache thrashing. One way to optimize for this is to use a scout thread on each core to prefetch data to the cache, where it is processed by the other three threads.

This leaves open the critical question of the actual memory bandwidth to the Larrabee processor. High-end processors like Nehalem have very large cache memories; programs with data sets that fit in the cache can run quite fast. However, programs with very large data sets that have to be accessed from memory will see significant performance degredation, not only because of the relatively long latency to main memory, but because the basic memory bandwidth is less than that supported by the cache. If Larrabee is designed with a high bandwidth memory interface, it will be able to keep the cores busy, even if it has to expend some fraction of its threads preloading the cache for the other threads. However, if the Larrabee memory bandwidth can’t keep up with the demand from all the cores, then many HPC programs will likely work just as fast or faster on a high-end Nehalem, which has the huge L3 cache and faster single thread performance.

Programming

The Larrabee native programming model, as described by Intel, uses an extended POSIX Threads API with a lightweight task stealing scheduler. There is no hardware thread scheduling, no hardware synchronization registers, no significant hardware to support parallel applications beyond the coherent cache mechanism. This increases generality and flexibility greatly, but also does not address the cost of task creation and management. Larrabee applications will tend towards larger tasks, to amortize the task creation and scheduling costs. This is somewhat at odds with the need to fill 128 threads (32 cores x 4 threads/core) to get full performance. The dependence on cache locality makes thread affinity scheduling critical. We’ll have to wait and see how well the operating system works in this model.

Summary

We’re all eagerly awaiting the first real Larrabee deliveries and performance results; I wouldn’t be surprised to see something at SC’09. I may have missed important architectural details, so perhaps we’ll have to have a followup column when we have real hardware available. I’m sure we’ll see several early projects to move HPC applications onto Larrabee, comparing performance in particular to an NVIDIA GPU. NVIDIA just announced the architecture of its next generation Fermi GPU, with many features aimed specifically at the compute market. We can’t compare performance without actual hardware, but in my next column I’ll compare the architectural features of the Larrabee to those of the NVIDIA Tesla and Fermi, in the HPC space.

About the Author

Michael Wolfe has developed compilers for over 30 years in both academia and industry, and is now a senior compiler engineer at The Portland Group, Inc. (www.pgroup.com), a wholly-owned subsidiary of STMicroelectronics, Inc. The opinions stated here are those of the author, and do not represent opinions of The Portland Group, Inc. or STMicroelectronics, Inc.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Ohio Supercomputer Center Dedicates ‘Owens’ Cluster

March 29, 2017

In a dedication ceremony held earlier today (March 29), officials from Ohio Supercomputer Center (OSC) along with state representatives gathered to celebrate the launch of OSC’s newest cluster: Read more…

By Tiffany Trader

EU Ratchets up the Race to Exascale Computing

March 29, 2017

The race to expand HPC infrastructure, including exascale machines, to advance national and regional interests ratcheted up a notch yesterday with announcement that seven European countries – Read more…

By John Russell

Data-Hungry Algorithms and the Thirst for AI

March 29, 2017

At Tabor Communications’ Leverage Big Data + EnterpriseHPC Summit in Florida last week, esteemed HPC professional Jay Boisseau, chief HPC technology strategist at Dell EMC, engaged the audience with his presentation, “Big Computing, Big Data, Big Trends, Big Results.” Read more…

By Tiffany Trader

Bill Gropp – Pursuing the Next Big Thing at NCSA

March 28, 2017

About eight months ago Bill Gropp was elevated to acting director of the National Center for Supercomputing Applications (NCSA). Read more…

By John Russell

HPE Extreme Performance Solutions

Leveraging the Power of Big Data to Improve Customer Satisfaction & Brand Loyalty

In the dynamic world of retail, retailers must find ways to recognize and effectively respond to shopping behaviors, patterns, and trends in order to succeed. Read more…

UK to Launch Six Major HPC Centers

March 27, 2017

Six high performance computing centers will be formally launched in the U.K. later this week intended to provide wider access to HPC resources to U.K. Read more…

By John Russell

AI in the News: Rao in at Intel, Ng out at Baidu, Nvidia on at Tencent Cloud

March 26, 2017

Just as AI has become the leitmotif of the advanced scale computing market, infusing much of the conversation about HPC in commercial and industrial spheres, it also is impacting high-level management changes in the industry. Read more…

By Doug Black

Scalable Informatics Ceases Operations

March 23, 2017

On the same day we reported on the uncertain future for HPC compiler company PathScale, we are sad to learn that another HPC vendor, Scalable Informatics, is closing its doors. Read more…

By Tiffany Trader

‘Strategies in Biomedical Data Science’ Advances IT-Research Synergies

March 23, 2017

“Strategies in Biomedical Data Science: Driving Force for Innovation” by Jay A. Etchings is both an introductory text and a field guide for anyone working with biomedical data. Read more…

By Tiffany Trader

Data-Hungry Algorithms and the Thirst for AI

March 29, 2017

At Tabor Communications’ Leverage Big Data + EnterpriseHPC Summit in Florida last week, esteemed HPC professional Jay Boisseau, chief HPC technology strategist at Dell EMC, engaged the audience with his presentation, “Big Computing, Big Data, Big Trends, Big Results.” Read more…

By Tiffany Trader

Bill Gropp – Pursuing the Next Big Thing at NCSA

March 28, 2017

About eight months ago Bill Gropp was elevated to acting director of the National Center for Supercomputing Applications (NCSA). Read more…

By John Russell

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its assets. Read more…

By Tiffany Trader

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Read more…

By John Russell

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the campaign. Read more…

By John Russell

CPU-based Visualization Positions for Exascale Supercomputing

March 16, 2017

In this contributed perspective piece, Intel’s Jim Jeffers makes the case that CPU-based visualization is now widely adopted and as such is no longer a contrarian view, but is rather an exascale requirement. Read more…

By Jim Jeffers, Principal Engineer and Engineering Leader, Intel

US Supercomputing Leaders Tackle the China Question

March 15, 2017

Joint DOE-NSA report responds to the increased global pressures impacting the competitiveness of U.S. supercomputing. Read more…

By Tiffany Trader

New Japanese Supercomputing Project Targets Exascale

March 14, 2017

Another Japanese supercomputing project was revealed this week, this one from emerging supercomputer maker, ExaScaler Inc., and Keio University. The partners are working on an original supercomputer design with exascale aspirations. Read more…

By Tiffany Trader

For IBM/OpenPOWER: Success in 2017 = (Volume) Sales

January 11, 2017

To a large degree IBM and the OpenPOWER Foundation have done what they said they would – assembling a substantial and growing ecosystem and bringing Power-based products to market, all in about three years. Read more…

By John Russell

Quantum Bits: D-Wave and VW; Google Quantum Lab; IBM Expands Access

March 21, 2017

For a technology that’s usually characterized as far off and in a distant galaxy, quantum computing has been steadily picking up steam. Read more…

By John Russell

Trump Budget Targets NIH, DOE, and EPA; No Mention of NSF

March 16, 2017

President Trump’s proposed U.S. fiscal 2018 budget issued today sharply cuts science spending while bolstering military spending as he promised during the campaign. Read more…

By John Russell

HPC Compiler Company PathScale Seeks Life Raft

March 23, 2017

HPCwire has learned that HPC compiler company PathScale has fallen on difficult times and is asking the community for help or actively seeking a buyer for its assets. Read more…

By Tiffany Trader

TSUBAME3.0 Points to Future HPE Pascal-NVLink-OPA Server

February 17, 2017

Since our initial coverage of the TSUBAME3.0 supercomputer yesterday, more details have come to light on this innovative project. Of particular interest is a new board design for NVLink-equipped Pascal P100 GPUs that will create another entrant to the space currently occupied by Nvidia's DGX-1 system, IBM's "Minsky" platform and the Supermicro SuperServer (1028GQ-TXR). Read more…

By Tiffany Trader

Tokyo Tech’s TSUBAME3.0 Will Be First HPE-SGI Super

February 16, 2017

In a press event Friday afternoon local time in Japan, Tokyo Institute of Technology (Tokyo Tech) announced its plans for the TSUBAME3.0 supercomputer, which will be Japan’s “fastest AI supercomputer,” Read more…

By Tiffany Trader

IBM Wants to be “Red Hat” of Deep Learning

January 26, 2017

IBM today announced the addition of TensorFlow and Chainer deep learning frameworks to its PowerAI suite of deep learning tools, which already includes popular offerings such as Caffe, Theano, and Torch. Read more…

By John Russell

Lighting up Aurora: Behind the Scenes at the Creation of the DOE’s Upcoming 200 Petaflops Supercomputer

December 1, 2016

In April 2015, U.S. Department of Energy Undersecretary Franklin Orr announced that Intel would be the prime contractor for Aurora: Read more…

By Jan Rowell

Leading Solution Providers

Is Liquid Cooling Ready to Go Mainstream?

February 13, 2017

Lost in the frenzy of SC16 was a substantial rise in the number of vendors showing server oriented liquid cooling technologies. Three decades ago liquid cooling was pretty much the exclusive realm of the Cray-2 and IBM mainframe class products. That’s changing. We are now seeing an emergence of x86 class server products with exotic plumbing technology ranging from Direct-to-Chip to servers and storage completely immersed in a dielectric fluid. Read more…

By Steve Campbell

Enlisting Deep Learning in the War on Cancer

December 7, 2016

Sometime in Q2 2017 the first ‘results’ of the Joint Design of Advanced Computing Solutions for Cancer (JDACS4C) will become publicly available according to Rick Stevens. He leads one of three JDACS4C pilot projects pressing deep learning (DL) into service in the War on Cancer. Read more…

By John Russell

BioTeam’s Berman Charts 2017 HPC Trends in Life Sciences

January 4, 2017

Twenty years ago high performance computing was nearly absent from life sciences. Today it’s used throughout life sciences and biomedical research. Genomics and the data deluge from modern lab instruments are the main drivers, but so is the longer-term desire to perform predictive simulation in support of Precision Medicine (PM). There’s even a specialized life sciences supercomputer, ‘Anton’ from D.E. Shaw Research, and the Pittsburgh Supercomputing Center is standing up its second Anton 2 and actively soliciting project proposals. There’s a lot going on. Read more…

By John Russell

HPC Startup Advances Auto-Parallelization’s Promise

January 23, 2017

The shift from single core to multicore hardware has made finding parallelism in codes more important than ever, but that hasn’t made the task of parallel programming any easier. Read more…

By Tiffany Trader

HPC Technique Propels Deep Learning at Scale

February 21, 2017

Researchers from Baidu’s Silicon Valley AI Lab (SVAIL) have adapted a well-known HPC communication technique to boost the speed and scale of their neural network training and now they are sharing their implementation with the larger deep learning community. Read more…

By Tiffany Trader

US Supercomputing Leaders Tackle the China Question

March 15, 2017

Joint DOE-NSA report responds to the increased global pressures impacting the competitiveness of U.S. supercomputing. Read more…

By Tiffany Trader

CPU Benchmarking: Haswell Versus POWER8

June 2, 2015

With OpenPOWER activity ramping up and IBM’s prominent role in the upcoming DOE machines Summit and Sierra, it’s a good time to look at how the IBM POWER CPU stacks up against the x86 Xeon Haswell CPU from Intel. Read more…

By Tiffany Trader

IDG to Be Bought by Chinese Investors; IDC to Spin Out HPC Group

January 19, 2017

US-based publishing and investment firm International Data Group, Inc. (IDG) will be acquired by a pair of Chinese investors, China Oceanwide Holdings Group Co., Ltd. Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This