No Go for GloFo at 7nm; and the Fujitsu A64FX post-K CPU

By Dairsie Latimer

September 5, 2018

It’s been a news worthy couple of weeks in the semiconductor and HPC industry. There were several HPC relevant disclosures at Hot Chips 2018 to whet appetites. This included details of the newest entrant to the HPC processor space, Fujitsu’s A64FX. However, more far reaching news came last week as Global Foundries announced it has decided to put its 7 nm node on hold, and entirely stopped development of nodes beyond 7 nm. This has some clear ramifications for the HPC industry.

Three green bottles

With Global Foundries shuttering development there are now only three companies left in the game at 10/7 nm; TSMC, Samsung and Intel. Of these, two are already significantly delayed reaching volume manufacturing with their current next generation process. TSMC were already in pole position at 7 nm, with numerous designs already taped out and sampling, and now it would seem that they are effectively the only game in town for now. As a result, in a year or two we may even notice some signs of supply constraints if TSMC’s 7 nm customers saturate their production capacity.

What’s behind the Global Foundries’ decision?

Put simply it’s one of simple demand economics. Non-recurring engineering costs at lower process nodes (including 10 and 7 nm) are spiralling, with some industry analysts suggesting that they are now approaching $200 million if one factors in all of the costs to reach volume manufacturing.

This will limit the number of customers actually wanting to or indeed able to afford to use the lower geometry nodes for mass market manufacturing and that previous nodes will be longer lived. As Global Foundries really only had AMD and IBM as potential customers at 7 nm it seems that lack of demand at 7 nm for their process (compared to TSMC’s) is the real driver behind this decision.

Despite the initial DNA of Global Foundries, the relationship with AMD has blown hot and cold for years. Once AMD clearly signalled their manufacturing intentions for next generation 7 nm silicon (Rome) was at TSCM, this meant that Global Foundries’ lead customer at 7 nm had effectively disappeared. No sense in making a further $2-4 billion investment in a fab that would never reach anything approaching the production levels necessary to pay for it.

It also leaves IBM in a potentially difficult position as Global Foundries are one of their process technology partners (having sold their foundry business to them some while back). It’s too soon for any public disclosure how this will impact IBM’s Power roadmap (which was linked with 10 nm for Power10 but which Global Foundries skipped to concentrate on 7 nm.) All this could mean a disruptive realignment for their ASIC teams if they are to move to Samsung (who are part of a research alliance comprising IBM, Global Foundries and Samsung) or TSMC for manufacturing 10/7 nm.

The Global Foundries CEO noted that projections for 2022 are almost two thirds of volume manufacturing will still be on 12 nm and above and so older nodes will be profitable for a good while yet. Global Foundries also has some interesting opportunities with their FD-SOI processes in the growing extreme mobile and RF markets, where absolute power efficiency is more important than transistor density driving Moore’s law.

Is this really the end of the road for silicon scaling?

We’ve seen for the last four or five years bulk CMOS silicon scaling (and Moore’s law) is stalling (arguably from the 22/14 nm transition). We’ve seen another major inflection point at the 10/7 nm transition and while there is a tentative roadmap to sub 5 nm (or even 2 nm) it’s based on optimistic projections for EUV availability and innovations in transistor design that have proven elusive thus far.

More worryingly the traditional separation of concerns between architecture design and backend (place and route of synthesised designs) is becoming blurred as even the increasingly restrictive design rules struggle to ensure viable designs for volume manufacture.

There are an increasing number of effects that process engineers, as well as standard cell designers, need to track and solve for, many of which can translate to significant yield variations even from wafer to wafer, let alone process generation to process generation and foundry to foundry.

Process variations of up to 10 percent may now actually take away much of the full advantage of using a new process node, unless the standard cell providers and EDA vendors can provide easily utilised enhancements to their logic libraries and place and route techniques. Now more than ever there needs to be a tight coupling between the customer, the foundry and EDA tool vendor to ensure that performance and yields are kept at economically viable levels.

Developments in transistor design, especially around the use of variable pitch nanosheets (which has very some attractive properties) rather than the increasingly difficult to manufacture finFETs, may well mean that there is still some limited density scaling benefit to moving to lower geometry nodes, but at a greatly reduced rate compared to the heyday of Moore’s law.

It’s more than likely that the curve that describes the actual cost per transistor, which has declined over time, has done more than flatten out recently and we may well see a further reverse in real terms. Couple this with the spiralling investments required to continue lithography scaling (for both foundry and customer) and the rapidly dwindling number of companies able to do so it’s clear that we are actually witnessing the last major inflection point for Moore’s law as it is currently constituted.

If costs per transistor no longer decrease as a result of minimal area scaling on lower geometry nodes then other means of transistor density scaling at a package level will need to be adopted. This will lead to an increasing focus on die stacking, monolithic 3D fabrication and multi-chip-modules, as well as potentially increased integration of on package communications links. Expect to see solutions utilising these techniques proliferate in HPC in the next few years.

Fujitsu’s A64FX

All of which neatly brings us onto one of the most interesting announcements at Hot Chips 30 and probably by far the most anticipated, which was the public unveiling of key details for the Fujitsu A64FX post-K CPU architecture.

The A64FX is made up of a relatively modest 8.7 billion transistors and is baked on a 7 nm process node. No official details of who’s but it don’t take much of a genius to work it out. It’s the first CPU to implement Arm’s Scalable Vector Extensions (SVE), specifically intended for high performance computing (and AI workloads) as well as a host of other interesting system level features.

In many respects it represents an ARM flavoured version of the tried and tested recipe followed by Fujitsu for the K Computer (and the later Sparc64-XIfx), but with a few notable enhancements. Prof. Satoshi Matsuoka also noted (on Twitter) that the TDP for the A64FX was likely to be eye catching (in a good way). Given the high level of integration it will be interesting to see how they do here but I expect they’re hoping to hit around the 160-180W mark. Working back from the quoted peak DP TFlops of 2.7, this implies a relatively modest 1.75GHz clock speed which will help. Then again if you are hoping to be able to scale to 200-300k nodes (and 384 nodes in a 48U rack) a power sipping TDP is pretty much a prerequisite.

Other notable metrics include 1TB/s memory bandwidth delivered by four 8GB stacks of HBM2 memory (rather than the HMC on the prior Sparc64-XIfx), each one associated with a Core Memory Group of cores (12 compute + 1 helper on a crossbar) all connected via coherent caches to the system ring bus and the Tofu3 port. The Tofu3 fabric supplies the main off chip connectivity (the PCIe3 is there for peripherals) and is currently a modest upgrade from the Tofu2 interconnect on the Sparc64-XIfx.

One interesting aspect to consider is the amount of memory per core (assuming 1GB pinned per helper core) which is around 0.6GB. For many simulation environments (where bandwidth rather than working set size is king) this may not prove a huge handicap but there are workloads which will definitely fall out of node on the ‘current’ A64FX memory configs.

The upshot is that Fujitsu clearly believe that this is well balanced system, compared to some other precursor HPC CPUs that bare some striking similarities. Expectation is that the A64FX may make it to market in 2019, prior to deployment in the post-K machine in 2021. It also perhaps leaves a tantalising glimpse of further possible refinements over the next couple of years before the post-K system is due to be delivered. I wouldn’t be surprised to see some more cores enabled per CMG, along with less probable enhancements to the HMB memory interfaces (faster as well as larger) as well as to the Tofu3 PHYs (more bandwidth if not necessarily lower latency).

Roll on 2019 and beyond.

About the Author

Dairsie Latimer, Technical Advisor at Red Oak Consulting, has a somewhat eclectic background, having worked in a variety of roles on supplier side and client side across the commercial and public sectors as an consultant and software engineer. Following an early career in computer graphics, micro-architecture design and full stack software development, he has over twelve years’ specialist experience in the HPC sector, ranging from developing low-level libraries and software for novel computing architectures to porting complex HPC applications to a range of accelerators. Dairise joined Red Oak Consulting (@redoakHPC) in 2010 bringing his wealth of experience to both the business and customers.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Top Ten Ways AI Affects HPC in 2019

March 26, 2019

AI workloads are becoming ubiquitous, including running on the world’s fastest computers — thereby changing what we call HPC forever. As every organization plans for the future, AI workloads are on our minds — how Read more…

By James Reinders

GTC 2019: Chief Scientist Bill Dally Provides Glimpse into Nvidia Research Engine

March 22, 2019

Amid the frenzy of GTC this week – Nvidia’s annual conference showcasing all things GPU (and now AI) – William Dally, chief scientist and SVP of research, provided a brief but insightful portrait of Nvidia’s rese Read more…

By John Russell

ORNL Helps Identify Challenges of Extremely Heterogeneous Architectures

March 21, 2019

Exponential growth in classical computing over the last two decades has produced hardware and software that support lightning-fast processing speeds, but advancements are topping out as computing architectures reach thei Read more…

By Laurie Varma

HPE Extreme Performance Solutions

HPE and Intel® Omni-Path Architecture: How to Power a Cloud

Learn how HPE and Intel® Omni-Path Architecture provide critical infrastructure for leading Nordic HPC provider’s HPCFLOW cloud service.

powercloud_blog.jpgFor decades, HPE has been at the forefront of high-performance computing, and we’ve powered some of the fastest and most robust supercomputers in the world. Read more…

IBM Accelerated Insights

Insurance: Where’s the Risk?

Insurers are facing extreme competitive challenges in their core businesses. Property and Casualty (P&C) and Life and Health (L&H) firms alike are highly impacted by the ongoing globalization, increasing regulation, and digital transformation of their client bases. Read more…

Interview with 2019 Person to Watch Jim Keller

March 21, 2019

On the heels of Intel's reaffirmation that it will deliver the first U.S. exascale computer in 2021, which will feature the company's new Intel Xe architecture, we bring you our interview with our 2019 Person to Watch Jim Keller, head of the Silicon Engineering Group at Intel. Read more…

By HPCwire Editorial Team

Top Ten Ways AI Affects HPC in 2019

March 26, 2019

AI workloads are becoming ubiquitous, including running on the world’s fastest computers — thereby changing what we call HPC forever. As every organization Read more…

By James Reinders

GTC 2019: Chief Scientist Bill Dally Provides Glimpse into Nvidia Research Engine

March 22, 2019

Amid the frenzy of GTC this week – Nvidia’s annual conference showcasing all things GPU (and now AI) – William Dally, chief scientist and SVP of research, Read more…

By John Russell

At GTC: Nvidia Expands Scope of Its AI and Datacenter Ecosystem

March 19, 2019

In the high-stakes race to provide the AI life-cycle solution of choice, three of the biggest horses in the field are IBM, Intel and Nvidia. While the latter is only a fraction of the size of its two bigger rivals, and has been in business for only a fraction of the time, Nvidia continues to impress with an expanding array of new GPU-based hardware, software, robotics, partnerships and... Read more…

By Doug Black

Nvidia Debuts Clara AI Toolkit with Pre-Trained Models for Radiology Use

March 19, 2019

AI’s push into healthcare got a boost yesterday with Nvidia’s release of the Clara Deploy AI toolkit which includes 13 pre-trained models for use in radiolo Read more…

By John Russell

It’s Official: Aurora on Track to Be First US Exascale Computer in 2021

March 18, 2019

The U.S. Department of Energy along with Intel and Cray confirmed today that an Intel/Cray supercomputer, "Aurora," capable of sustained performance of one exaf Read more…

By Tiffany Trader

Why Nvidia Bought Mellanox: ‘Future Datacenters Will Be…Like High Performance Computers’

March 14, 2019

“Future datacenters of all kinds will be built like high performance computers,” said Nvidia CEO Jensen Huang during a phone briefing on Monday after Nvidia revealed scooping up the high performance networking company Mellanox for $6.9 billion. Read more…

By Tiffany Trader

Oil and Gas Supercloud Clears Out Remaining Knights Landing Inventory: All 38,000 Wafers

March 13, 2019

The McCloud HPC service being built by Australia’s DownUnder GeoSolutions (DUG) outside Houston is set to become the largest oil and gas cloud in the world th Read more…

By Tiffany Trader

Quick Take: Trump’s 2020 Budget Spares DoE-funded HPC but Slams NSF and NIH

March 12, 2019

U.S. President Donald Trump’s 2020 budget request, released yesterday, proposes deep cuts in many science programs but seems to spare HPC funding by the Depar Read more…

By John Russell

Quantum Computing Will Never Work

November 27, 2018

Amid the gush of money and enthusiastic predictions being thrown at quantum computing comes a proposed cold shower in the form of an essay by physicist Mikhail Read more…

By John Russell

The Case Against ‘The Case Against Quantum Computing’

January 9, 2019

It’s not easy to be a physicist. Richard Feynman (basically the Jimi Hendrix of physicists) once said: “The first principle is that you must not fool yourse Read more…

By Ben Criger

Why Nvidia Bought Mellanox: ‘Future Datacenters Will Be…Like High Performance Computers’

March 14, 2019

“Future datacenters of all kinds will be built like high performance computers,” said Nvidia CEO Jensen Huang during a phone briefing on Monday after Nvidia revealed scooping up the high performance networking company Mellanox for $6.9 billion. Read more…

By Tiffany Trader

ClusterVision in Bankruptcy, Fate Uncertain

February 13, 2019

ClusterVision, European HPC specialists that have built and installed over 20 Top500-ranked systems in their nearly 17-year history, appear to be in the midst o Read more…

By Tiffany Trader

Intel Reportedly in $6B Bid for Mellanox

January 30, 2019

The latest rumors and reports around an acquisition of Mellanox focus on Intel, which has reportedly offered a $6 billion bid for the high performance interconn Read more…

By Doug Black

Looking for Light Reading? NSF-backed ‘Comic Books’ Tackle Quantum Computing

January 28, 2019

Still baffled by quantum computing? How about turning to comic books (graphic novels for the well-read among you) for some clarity and a little humor on QC. The Read more…

By John Russell

It’s Official: Aurora on Track to Be First US Exascale Computer in 2021

March 18, 2019

The U.S. Department of Energy along with Intel and Cray confirmed today that an Intel/Cray supercomputer, "Aurora," capable of sustained performance of one exaf Read more…

By Tiffany Trader

Contract Signed for New Finnish Supercomputer

December 13, 2018

After the official contract signing yesterday, configuration details were made public for the new BullSequana system that the Finnish IT Center for Science (CSC Read more…

By Tiffany Trader

Leading Solution Providers

SC 18 Virtual Booth Video Tour

Advania @ SC18 AMD @ SC18
ASRock Rack @ SC18
DDN Storage @ SC18
HPE @ SC18
IBM @ SC18
Lenovo @ SC18 Mellanox Technologies @ SC18
NVIDIA @ SC18
One Stop Systems @ SC18
Oracle @ SC18 Panasas @ SC18
Supermicro @ SC18 SUSE @ SC18 TYAN @ SC18
Verne Global @ SC18

Deep500: ETH Researchers Introduce New Deep Learning Benchmark for HPC

February 5, 2019

ETH researchers have developed a new deep learning benchmarking environment – Deep500 – they say is “the first distributed and reproducible benchmarking s Read more…

By John Russell

IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans

January 10, 2019

IBM made three significant quantum computing announcements at CES this week. One was introduction of IBM Q System One; it’s really the integration of IBM’s Read more…

By John Russell

IBM Bets $2B Seeking 1000X AI Hardware Performance Boost

February 7, 2019

For now, AI systems are mostly machine learning-based and “narrow” – powerful as they are by today's standards, they're limited to performing a few, narro Read more…

By Doug Black

The Deep500 – Researchers Tackle an HPC Benchmark for Deep Learning

January 7, 2019

How do you know if an HPC system, particularly a larger-scale system, is well-suited for deep learning workloads? Today, that’s not an easy question to answer Read more…

By John Russell

HPC Reflections and (Mostly Hopeful) Predictions

December 19, 2018

So much ‘spaghetti’ gets tossed on walls by the technology community (vendors and researchers) to see what sticks that it is often difficult to peer through Read more…

By John Russell

Arm Unveils Neoverse N1 Platform with up to 128-Cores

February 20, 2019

Following on its Neoverse roadmap announcement last October, Arm today revealed its next-gen Neoverse microarchitecture with compute and throughput-optimized si Read more…

By Tiffany Trader

Move Over Lustre & Spectrum Scale – Here Comes BeeGFS?

November 26, 2018

Is BeeGFS – the parallel file system with European roots – on a path to compete with Lustre and Spectrum Scale worldwide in HPC environments? Frank Herold Read more…

By John Russell

France to Deploy AI-Focused Supercomputer: Jean Zay

January 22, 2019

HPE announced today that it won the contract to build a supercomputer that will drive France’s AI and HPC efforts. The computer will be part of GENCI, the Fre Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This