Ayar Labs to Demo Photonics Chiplet in FPGA Package at Hot Chips

By Tiffany Trader

August 19, 2019

Silicon startup Ayar Labs continues to gain momentum with its DARPA-backed optical chiplet technology that puts advanced electronics and optics on the same chip using standard CMOS fabrication. At Hot Chips 31 in Stanford, Calif., this week, Ayar Labs will show results of the work it’s been doing with DARPA and Intel on an FPGA chiplet integration platform targeting radar applications.

Ayar Labs, which has offices in Emeryville and Santa Clara, Calif., has just taped out its 11th generation “AL11” electrical-to-optical I/O chiplet, the basis for its commercial product, called TeraPHY, which the company says will enable optical I/O bandwidth capability in excess of a Terabit/sec at 10X lower power than today’s Gigabit/sec I/O solutions.

TeraPHY will be integrated with other silicon technology partners’ designs into multi-chip module (MCM) computing products. In a stroke of good timing for Ayar Labs–which expects early system deployments and prototypes in “reasonable volume” in 2020–there’s something of an MCM/chiplet revolution afoot. Chipmakers, such as Intel and AMD, are turning to the modular packaging approach to sidestep Moore’s law (or maybe Gordon Moore actually foresaw this).

Ayar Labs has been working closely with Intel’s Programmable Solutions Group as part of the DARPA CHIPS (Common Heterogeneous Integration and IP Reuse Strategies) project. At Hot Chips this week, Ayar Labs will be showing its 10th gen chiplet installed in an Intel FPGA package with an Intel Stratix 10 die; and it will also announce its support for the Advanced Interface Bus (AIB) protocol.

ERI Summit (July 16, 2019)

Intel’s Senior Principal Engineer Sergey Shumarayev previewed the new technology at DARPA’s ERI Summit last month. The Ayar Labs chiplet connects to the Intel Stratix 10 FPGA die via the AIB interface using Intel’s EMIB packaging. The Jariet Technologies’ millimeter wave chiplet gets information in through RF, passes that information to the FPGA for pre-processing and that gets shipped out optically via the Ayar Labs chiplet.

The University of Michigan chiplet functions as an AI accelerator. Ayar Labs is not directly involved in the University of Michigan research project, but Ayar Labs Chief Scientist and Co-Founder Mark Wade noted the AI chiplet illustrates the versatility of this type of integration.

“This chiplet package ecosystem allows someone to put in an AI accelerator chip and interface to the FPGA, and then also on the other side interface to our high performance optical IO chip; there’s a kind of transformative joint solution coming together that can be enabled by this emerging ecosystem,” said Wade, who will be presenting at Hot Chips on Tuesday.

Ayar Labs has been working with Intel to evolve AIB — the wide, parallel PHY-level interface that Intel open sourced to help spur chiplet adoption. Freely available on GitHub, AIB is being driven as an open industry standard for enabling low-latency, power-efficient direct chiplet-to-chiplet communication.

Ayar Labs also continues to work on support for serial interfaces, like SerDes — an approach of course that has a lot of momentum behind it. Wade said Ayar’s roadmap now includes a provision for both wide parallel implementations, such as AIB, as well as serial implementations, but he says they are getting the most traction with the wide parallel approach.

Ayar Labs CEO Charlie Wuischpard, who joined the company in November 2018, after heading HPC at Intel for a number of years, told HPCwire that while the FPGA solution being demoed this week is emerging as an Intel technology, the opportunity extends beyond working with a single vendor.

“The fact that the government is standardizing on an electrical interface makes this really suitable for anybody else to pick up and run with,” he said. “We’ve been getting a ton of interest from across the industry as there’s this greater recognition that optical I/O creates whole new sets of opportunities.

Example of a possible heterogeneous system in package (SiP) that combines sensors, proprietary ASIC, FPGA, CPU, Memory, and I/O using AIB as the chiplet interface. (Source: Intel Corp.)

“One of the really interesting things about our technology is not just the innovative optical chiplet that goes in the package, but once you’re relieved of the constraints that are currently in place in system designs around the power and packaging and cooling to drive the massive signals, and that’s the bandwidth over copper, then you can create brand new system architectures,” said Wuischpard.

The FPGA demonstration targets phased array radar applications, which is an important market target for Ayar Labs along with 5G, but Hugo Saleh, vice president of marketing and business development, who also made the jump from Intel, told HPCwire they anticipate another killer app: high-end HPC and AI. Picture, for example, an Nvidia DGX-2 type-box, but instead of the NVLinks, there would be 16 GPUs or AI chips connected by optical interfaces, and then multiple boxes could be connected using optical instead of switched fabrics.

Potential use cases extend to any server with a PCIe card in it. “We’ve got a prototype design where essentially you could take a GPU or an FPGA or an ASIC and build a system the size of a whole rack or maybe multiple racks all optically connected that far expands beyond the 16 GPUs currently supported [in the DGX2],” said Saleh.

The glueless, point-to-point interconnect approach applies for standard two-socket servers, he added. Ayar Labs foresees using optical to create supernodes where every element of the system can talk to every other element of the system, versus going up through a switch.

“We’ve reviewed this with several of the large hyperscale providers and the day we have this capability, they want to start testing it,” said Wuischpard. “We have to be able to first produce our single instance of optically connected FPGA as a first step, but we very rapidly want to move to these new system designs that use optical I/O in a new way.”

AI applications and graph analytics are clamoring for this kind of memory semantic type architecture, and government agencies are looking to leverage the technology for advanced threat intelligence, for identifying patterns in vast and disparate data sets and for tracking objects—like airplanes and drones-in real-time.

Ayar Labs has in its pipeline a graph analytics design win that is using four Ayar Labs chiplets per socket with each one of those chiplets putting out 1.3 Terabits per second of data. “That’s an aggregate of over 5 Terabits per second out of package and the only way they can do this is with optical,” Saleh stated.

In the lab: co-packaged Stratix 10 FPGA and TeraPHY chiplets with fiber connected (courtesy: Ayar Labs)

Ayar Labs works closely with GlobalFoundries, a trusted foundry for the U.S. government. The TeraPHY chips are being manufactured using GloFo’s 45 nanometer RF SOI (Radio Frequency Silicon on Insulator) process, currently at its East Fishkill, NY, facility.

Each optoelectronic chip spans an area of approximately five by eight millimeters. Wade said the sizing takes into account how the chips line up with the Intel FPGA. The Intel chip is a full reticle FPGA die, about 25 millimeters on one edge, and the entire east-west side of that edge has the AIB interface on it. That allows up to three Ayar dies to abut each of those edges, to connect up to the AIB interface and escape all that bandwidth.

“This chiplet kind of revolution that is happening right now really allows a mix and match type of configurability that was not there before,” said Wade. “So it allows you to essentially customize at the package level a chip solution that can be targeted to specific applications. In some cases, you might use an FPGA that doesn’t need every slot populated with an optical I/O chip. So maybe you have a memory chip, maybe you have an AI accelerator chip, maybe you have one of our chiplets. And the beauty is you can mix and match all these things together in a heterogeneous way that previously really was not possible. So this emergence of the chiplet ecosystem is really a big development enabling our type of solution.”

Ayar Labs was founded in 2015 by a group of researchers from MIT, UC Berkeley, and CU Boulder who were part of a 10-year research collaboration funded by DARPA. The company has gone through several funding rounds with an investor roster that includes Playground Global, Intel Capital, Global Foundries, the Founders Fund, Silicon Valley Bank and others. It has also been the recipient of several recent DARPA grants and is involved in the CHIPS and PIPES projects.

Feature image caption: a cameo shot in the lab of Ayar Labs’ TeraPHY chiplet co-packaged with a Stratix 10 FPGA (courtesy: Ayar Labs)

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Army Seeks AI Ground Truth

April 3, 2020

Deep neural networks are being mustered by U.S. military researchers to marshal new technology forces on the Internet of Battlefield Things. U.S. Army and industry researchers said this week they have developed a “c Read more…

By George Leopold

Piz Daint Tackles Marsquakes

April 3, 2020

Even as researchers use supercomputers to probe the mysteries of earthquakes here on Earth, others are setting their sights on quakes just a little farther away. Researchers at ETH Zürich in Switzerland have applied sup Read more…

By Oliver Peckham

HPC Career Notes: April 2020 Edition

April 2, 2020

In this monthly feature, we’ll keep you up-to-date on the latest career developments for individuals in the high-performance computing community. Whether it’s a promotion, new company hire, or even an accolade, we’ Read more…

By Mariana Iriarte

AMD Epyc CPUs Now on Bare Metal IBM Cloud Servers

April 1, 2020

AMD’s expanding presence in the datacenter and cloud computing markets took a step forward with today’s announcement that its 7nm 2nd Gen Epyc 7642 CPUs are now available on IBM Cloud bare metal servers. AMD, whose Read more…

By Doug Black

Supercomputer Testing Probes Viral Transmission in Airplanes

April 1, 2020

It might be a long time before the general public is flying again, but the question remains: how high-risk is air travel in terms of viral infection? In an article for the Texas Advanced Computing Center (TACC), Faith Si Read more…

By Staff report

AWS Solution Channel

Amazon FSx for Lustre Update: Persistent Storage for Long-Term, High-Performance Workloads

Last year I wrote about Amazon FSx for Lustre and told you how our customers can use it to create pebibyte-scale, highly parallel POSIX-compliant file systems that serve thousands of simultaneous clients driving millions of IOPS (Input/Output Operations per Second) with sub-millisecond latency. Read more…

ECP Milestone Report Details Progress and Directions

April 1, 2020

The Exascale Computing Project (ECP) milestone report issued last week presents a good snapshot of progress in preparing applications for exascale computing. There are roughly 30 ECP application development (AD) subproj Read more…

By John Russell

ECP Milestone Report Details Progress and Directions

April 1, 2020

The Exascale Computing Project (ECP) milestone report issued last week presents a good snapshot of progress in preparing applications for exascale computing. Th Read more…

By John Russell

Pandemic ‘Wipes Out’ 2020 HPC Market Growth, Flat to 12% Drop Expected

March 31, 2020

As the world battles the still accelerating novel coronavirus, the HPC community has mounted a forceful response to the pandemic on many fronts. But these efforts won't inoculate the HPC industry from the economic effects of COVID-19. Market watcher Intersect360 Research has revised its 2020 forecast for HPC products and services, projecting... Read more…

By Tiffany Trader

LLNL Leverages Supercomputing to Identify COVID-19 Antibody Candidates

March 30, 2020

As COVID-19 sweeps the globe to devastating effect, supercomputers around the world are spinning up to fight back by working on diagnosis, epidemiology, treatme Read more…

By Staff report

Weather at Exascale: Load Balancing for Heterogeneous Systems

March 30, 2020

The first months of 2020 were dominated by weather and climate supercomputing news, with major announcements coming from the UK, the European Centre for Medium- Read more…

By Oliver Peckham

Q&A Part Two: ORNL’s Pooser on Progress in Quantum Communication

March 30, 2020

Quantum computing seems to get more than its fair share of attention compared to quantum communication. That’s despite the fact that quantum networking may be Read more…

By John Russell

DoE Expands on Role of COVID-19 Supercomputing Consortium

March 25, 2020

After announcing the launch of the COVID-19 High Performance Computing Consortium on Sunday, the Department of Energy yesterday provided more details on its sco Read more…

By John Russell

[email protected] Rallies a Legion of Computers Against the Coronavirus

March 24, 2020

Last week, we highlighted [email protected], a massive, crowdsourced computer network that has turned its resources against the coronavirus pandemic sweeping the globe – but [email protected] isn’t the only game in town. The internet is buzzing with crowdsourced computing... Read more…

By Oliver Peckham

Conversation: ANL’s Rick Stevens on DoE’s AI for Science Project

March 23, 2020

With release of the Department of Energy’s AI for Science report in late February, the effort to build a national AI program, modeled loosely on the U.S. Exascale Initiative, enters a new phase. Project leaders have already had early discussions with Congress... Read more…

By John Russell

[email protected] Turns Its Massive Crowdsourced Computer Network Against COVID-19

March 16, 2020

For gamers, fighting against a global crisis is usually pure fantasy – but now, it’s looking more like a reality. As supercomputers around the world spin up Read more…

By Oliver Peckham

Julia Programming’s Dramatic Rise in HPC and Elsewhere

January 14, 2020

Back in 2012 a paper by four computer scientists including Alan Edelman of MIT introduced Julia, A Fast Dynamic Language for Technical Computing. At the time, t Read more…

By John Russell

Global Supercomputing Is Mobilizing Against COVID-19

March 12, 2020

Tech has been taking some heavy losses from the coronavirus pandemic. Global supply chains have been disrupted, virtually every major tech conference taking place over the next few months has been canceled... Read more…

By Oliver Peckham

[email protected] Rallies a Legion of Computers Against the Coronavirus

March 24, 2020

Last week, we highlighted [email protected], a massive, crowdsourced computer network that has turned its resources against the coronavirus pandemic sweeping the globe – but [email protected] isn’t the only game in town. The internet is buzzing with crowdsourced computing... Read more…

By Oliver Peckham

DoE Expands on Role of COVID-19 Supercomputing Consortium

March 25, 2020

After announcing the launch of the COVID-19 High Performance Computing Consortium on Sunday, the Department of Energy yesterday provided more details on its sco Read more…

By John Russell

Steve Scott Lays Out HPE-Cray Blended Product Roadmap

March 11, 2020

Last week, the day before the El Capitan processor disclosures were made at HPE's new headquarters in San Jose, Steve Scott (CTO for HPC & AI at HPE, and former Cray CTO) was on-hand at the Rice Oil & Gas HPC conference in Houston. He was there to discuss the HPE-Cray transition and blended roadmap, as well as his favorite topic, Cray's eighth-gen networking technology, Slingshot. Read more…

By Tiffany Trader

Fujitsu A64FX Supercomputer to Be Deployed at Nagoya University This Summer

February 3, 2020

Japanese tech giant Fujitsu announced today that it will supply Nagoya University Information Technology Center with the first commercial supercomputer powered Read more…

By Tiffany Trader

Tech Conferences Are Being Canceled Due to Coronavirus

March 3, 2020

Several conferences scheduled to take place in the coming weeks, including Nvidia’s GPU Technology Conference (GTC) and the Strata Data + AI conference, have Read more…

By Alex Woodie

Leading Solution Providers

SC 2019 Virtual Booth Video Tour

AMD
AMD
ASROCK RACK
ASROCK RACK
AWS
AWS
CEJN
CJEN
CRAY
CRAY
DDN
DDN
DELL EMC
DELL EMC
IBM
IBM
MELLANOX
MELLANOX
ONE STOP SYSTEMS
ONE STOP SYSTEMS
PANASAS
PANASAS
SIX NINES IT
SIX NINES IT
VERNE GLOBAL
VERNE GLOBAL
WEKAIO
WEKAIO

Cray to Provide NOAA with Two AMD-Powered Supercomputers

February 24, 2020

The United States’ National Oceanic and Atmospheric Administration (NOAA) last week announced plans for a major refresh of its operational weather forecasting supercomputers, part of a 10-year, $505.2 million program, which will secure two HPE-Cray systems for NOAA’s National Weather Service to be fielded later this year and put into production in early 2022. Read more…

By Tiffany Trader

Exascale Watch: El Capitan Will Use AMD CPUs & GPUs to Reach 2 Exaflops

March 4, 2020

HPE and its collaborators reported today that El Capitan, the forthcoming exascale supercomputer to be sited at Lawrence Livermore National Laboratory and serve Read more…

By John Russell

Summit Supercomputer is Already Making its Mark on Science

September 20, 2018

Summit, now the fastest supercomputer in the world, is quickly making its mark in science – five of the six finalists just announced for the prestigious 2018 Read more…

By John Russell

IBM Unveils Latest Achievements in AI Hardware

December 13, 2019

“The increased capabilities of contemporary AI models provide unprecedented recognition accuracy, but often at the expense of larger computational and energet Read more…

By Oliver Peckham

TACC Supercomputers Run Simulations Illuminating COVID-19, DNA Replication

March 19, 2020

As supercomputers around the world spin up to combat the coronavirus, the Texas Advanced Computing Center (TACC) is announcing results that may help to illumina Read more…

By Staff report

IBM Debuts IC922 Power Server for AI Inferencing and Data Management

January 28, 2020

IBM today launched a Power9-based inference server – the IC922 – that features up to six Nvidia T4 GPUs, PCIe Gen 4 and OpenCAPI connectivity, and can accom Read more…

By John Russell

Summit Joins the Fight Against the Coronavirus

March 6, 2020

With the coronavirus sweeping the globe, tech conferences and supply chains are being hit hard – but now, tech is hitting back. Oak Ridge National Laboratory Read more…

By Staff report

University of Stuttgart Inaugurates ‘Hawk’ Supercomputer

February 20, 2020

This week, the new “Hawk” supercomputer was inaugurated in a ceremony at the High-Performance Computing Center of the University of Stuttgart (HLRS). Offici Read more…

By Staff report

  • arrow
  • Click Here for More Headlines
  • arrow
Do NOT follow this link or you will be banned from the site!
Share This