Tackling the Co-design 3.0 Puzzle – New Thinking Needed

By John Russell

November 9, 2016

Co-design has long been a vibrant discussion point in the HPC community. The need to coordinate development across hardware, software, and system architecture in the face of constraints from a declining Moore’s Law is a given. The question is how to do it. In this brief Q&A, Sadasivan (Sadas) Shankar, visiting lecturer at Harvard and former senior principal engineer in Intel’s Technology and Manufacturing Organization, glimpses into his invited SC16 talk, Co-Design 3.0 – Configurable Extreme Computing, Leveraging Moore’s Law for Real Applications.

HPCWire: Co-design means different things to different people and your talk is positioned as a discussion around emerging Co-design 3.0. What’s your definition of co-design and maybe talk a bit about the six scaling paradigm you suggest are relevant to co-design.

Sadasivan (Sadas) Shankar, Harvard
Sadasivan (Sadas) Shankar, Harvard

Sadas Shankar: Co-design refers to the methodology in which architecture of the computing platform, hardware, software, and applications are concurrently designed for a global optimum. In other words, it is the antithesis of the traditionally serial (or mostly serial) way of addressing the problem.

The six scaling paradigms are given below:

  1. Scale of physical and man-made entities
  2. Combinatorial Scaling
  3. Scaling of algorithms
  4. Technology scaling
  5. Economics of scaling
  6. Scaling of applications

Currently most of the thinking revolves around Moore’s law and cost (paradigms 4 and 5), with focus on big data analytics (part of paradigm 6). Till now, Moore’s law was the main driver for information technology, which in turn was driving the ecosystem. However, there are more ecosystems and collaterals changing that need to be addressed.

One specific example is can we could use the algorithms that have been developed for big data (paradigm 3 & 6) to accelerate drug development or design a new battery (paradigm 6) or solve environmental toxicity (paradigms 2 and 6). This is starting to happen, but not in a large scale. The key maybe that the current paradigm in which a given architecture-hardware combination on which software-algorithms are developed for a given application is too restricted and non-optimal. Ideally, it should be available to users to be able to customize it as they need it (See the last question).

screen-shot-2016-11-08-at-3-52-04-pmHPCWire: You use a Lego metaphor for how systems components – hardware, applications, algorithms, architecture, software – must become more easily integrated; yet doing this has proven challenging. How should differing domain expertise and different communities (academia, industry, government) be incented and organized to work together?

Shankar: Within United States, at the governmental level, Department of Energy, NSA and related organizations are the biggest consumers of computation. The hardware vendors put together components, microprocessors to certain high level specifications. The users compile software and use them depending on the applications. The business model for HPC has been like this since the advent of the mainframe. But recently, things have shifted. Silicon Valley and the entrepreneurs are starting to disrupt this model at low cost. Examples are given in the talk.

The question is can we put together a hybrid model in which an initiative in which academics, national labs and other federal agencies, and industry can come together to develop an evolving and flexible initiative. The main difference between this and traditional HPC or Computational centers is bringing in Silicon Valley start-up thinking into the mix. The ability to make Co-design an evolving effort with low cost business model and long-term sustainability is important to make this happen. For this both academics and industrial partners need to be part of this as well.

There is another problem brewing in the horizon; HPC (from the era of Thinking Machines, Cray, CDC etc.) is not considered an exciting area for students to specialize in or do research in. This means that the pipeline for HPC may dry up. Hence the need to tying Co-design with the academia in addition to just the national labs.

HPCWire: What is different about Co-Design 3.0, compared to what is already being done or why do we need it?

Shankar: Co-design 3.0 is both about a thinking shift and more distributed ecosystem in which not just the high-end users, but all get to design computing for their purposes. In order to financially and intellectually sustain and grow such a system, it needs the academic and industrial participants in addition to research labs and federal agencies. Coming back to the example of Lego blocks, players could make a truck or a car depending on the need. This ability is due to modularity, integrability and re-usability of the blocks. If I can personalize a smart phone (with “apps”), should this Co-design have “blocks” and “apps”? We don’t know yet, but should test out the different possibilities.

HPCWire: Two technologies getting a lot of attentions today are neuromorphic computing and quantum computing, each in its own way representing different computing paradigms. Looking at the implications flowing from Co-design 3.0 what’s your sense of emerging computing technologies that will be important in Post Moore’s era?

Shankar: Co-design 3.0 is meant to address these kinds of shifting paradigms. Both of these computing platforms may be optimal for specific applications such as neuromorphic computing for pattern recognition, and quantum computing for cryptography. This is in line with Co-design 3.0 thinking-one should not have to be locked into a given hardware and architecture for all applications. This is possible as long as the building “blocks” are reasonably modular enough, but yet can be assembled depending on the applications and disassembled without a large cost penalty. Although this is more difficult for quantum computing (QC) where information processing itself is based on qubits, there is still ability to develop the blocks such that part of a given larger problem can be solved in QC, while the remaining part can be solved in conventional HPC architectures. We will touch upon some of the challenges that need to be addressed by research, development, and application. Co-design 3.0 is as much a thinking paradigm as well as a framework to make it happen by some of the best minds from academia to government to Silicon Valley.

screen-shot-2016-11-08-at-2-47-50-pmHPCWire: Finally, in your abstract, you note developing a class at Harvard in which students are “taught hands-on about using extreme computing to address real applications.” Could you briefly describe the effort and how it is working?

Shankar: As we mentioned before, students seem to be losing interest in HPC as an area for future career growth. In order to get the students excited, we offered a class, possibly for the first time in US or elsewhere on “Extreme Computing for Real Applications”. We wanted the students t get excited that they could solve problems of societal importance by using hardware and software at the limits of what computing could accomplish. The course gave the students hands-on experience on 3 different applications (social networking, cancer genomics, battery modeling), on 3 different computing hardware platforms (cloud computing, cluster computing, supercomputer in Department of Energy Laboratory).

This course was taught by faculty from Harvard, in collaboration with the visitors from National Cancer Institute and Argonne National Laboratories, and had both lectures explaining the basis of theory and methods and computer lab sessions in which the students actually solved the problem. We are planning to offer the class again in Fall 2017. More write-up of the class is given in the web link below: Computing that goes to extremes

Shankar’s SC16 talk is Thursday at 10:30 am: http://sc16.supercomputing.org/presentation/?id=inv109&sess=sess261

Presenter Bio

Sadasivan (Sadas) Shankar is the first Margaret and Will Hearst Visiting Lecturer in Computational Science and Engineering at the Harvard John A. Paulson School of Engineering and Applied Sciences. In fall 2013, as the first Distinguished Scientist in Residence at the Institute of Applied Computational Sciences in Harvard, along with Dr. Tim Kaxiras, he developed and co-instructed with Dr. Brad Malone, a graduate-level class on Computational Materials Design, which covered fundamental atomic and quantum techniques and practical applications for new materials by design.

Shankar was also senior principal engineer and led materials design in the Design and Technology Group within the Intel Technology and Manufacturing Organization. Over his tenure in research and development in the semiconductor industry, he and his team have worked on several new initiatives; using modeling to optimize semiconductor processing and equipment for several technology generations, advanced process control using physics-based models, thermo-mechanical reliability of microprocessors, thermal modeling of 3D die stacking, and using thermodynamic principles to estimate energy efficiency of ideal computing architectures.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Why HPC Storage Matters More Now Than Ever: Analyst Q&A

September 17, 2021

With soaring data volumes and insatiable computing driving nearly every facet of economic, social and scientific progress, data storage is seizing the spotlight. Hyperion Research analyst and noted storage expert Mark No Read more…

GigaIO Gets $14.7M in Series B Funding to Expand Its Composable Fabric Technology to Customers

September 16, 2021

Just before the COVID-19 pandemic began in March 2020, GigaIO introduced its Universal Composable Fabric technology, which allows enterprises to bring together any HPC and AI resources and integrate them with networking, Read more…

What’s New in HPC Research: Solar Power, ExaWorks, Optane & More

September 16, 2021

In this regular feature, HPCwire highlights newly published research in the high-performance computing community and related domains. From parallel programming to exascale to quantum computing, the details are here. Read more…

Cerebras Brings Its Wafer-Scale Engine AI System to the Cloud

September 16, 2021

Five months ago, when Cerebras Systems debuted its second-generation wafer-scale silicon system (CS-2), co-founder and CEO Andrew Feldman hinted of the company’s coming cloud plans, and now those plans have come to fruition. Today, Cerebras and Cirrascale Cloud Services are launching... Read more…

AI Hardware Summit: Panel on Memory Looks Forward

September 15, 2021

What will system memory look like in five years? Good question. While Monday's panel, Designing AI Super-Chips at the Speed of Memory, at the AI Hardware Summit, tackled several topics, the panelists also took a brief glimpse into the future. Unlike compute, storage and networking, which... Read more…

AWS Solution Channel

Supporting Climate Model Simulations to Accelerate Climate Science

The Amazon Sustainability Data Initiative (ASDI), AWS is donating cloud resources, technical support, and access to scalable infrastructure and fast networking providing high performance computing (HPC) solutions to support simulations of near-term climate using the National Center for Atmospheric Research (NCAR) Community Earth System Model Version 2 (CESM2) and its Whole Atmosphere Community Climate Model (WACCM). Read more…

ECMWF Opens Bologna Datacenter in Preparation for Atos Supercomputer

September 14, 2021

In January 2020, the European Centre for Medium-Range Weather Forecasts (ECMWF) – a juggernaut in the weather forecasting scene – signed a four-year, $89-million contract with European tech firm Atos to quintuple its supercomputing capacity. With the deal approaching the two-year mark, ECMWF... Read more…

Why HPC Storage Matters More Now Than Ever: Analyst Q&A

September 17, 2021

With soaring data volumes and insatiable computing driving nearly every facet of economic, social and scientific progress, data storage is seizing the spotlight Read more…

Cerebras Brings Its Wafer-Scale Engine AI System to the Cloud

September 16, 2021

Five months ago, when Cerebras Systems debuted its second-generation wafer-scale silicon system (CS-2), co-founder and CEO Andrew Feldman hinted of the company’s coming cloud plans, and now those plans have come to fruition. Today, Cerebras and Cirrascale Cloud Services are launching... Read more…

AI Hardware Summit: Panel on Memory Looks Forward

September 15, 2021

What will system memory look like in five years? Good question. While Monday's panel, Designing AI Super-Chips at the Speed of Memory, at the AI Hardware Summit, tackled several topics, the panelists also took a brief glimpse into the future. Unlike compute, storage and networking, which... Read more…

ECMWF Opens Bologna Datacenter in Preparation for Atos Supercomputer

September 14, 2021

In January 2020, the European Centre for Medium-Range Weather Forecasts (ECMWF) – a juggernaut in the weather forecasting scene – signed a four-year, $89-million contract with European tech firm Atos to quintuple its supercomputing capacity. With the deal approaching the two-year mark, ECMWF... Read more…

Quantum Computer Market Headed to $830M in 2024

September 13, 2021

What is one to make of the quantum computing market? Energized (lots of funding) but still chaotic and advancing in unpredictable ways (e.g. competing qubit tec Read more…

Amazon, NCAR, SilverLining Team for Unprecedented Cloud Climate Simulations

September 10, 2021

Earth’s climate is, to put it mildly, not in a good place. In the wake of a damning report from the Intergovernmental Panel on Climate Change (IPCC), scientis Read more…

After Roadblocks and Renewals, EuroHPC Targets a Bigger, Quantum Future

September 9, 2021

The EuroHPC Joint Undertaking (JU) was formalized in 2018, beginning a new era of European supercomputing that began to bear fruit this year with the launch of several of the first EuroHPC systems. The undertaking, however, has not been without its speed bumps, and the Union faces an uphill... Read more…

How Argonne Is Preparing for Exascale in 2022

September 8, 2021

Additional details came to light on Argonne National Laboratory’s preparation for the 2022 Aurora exascale-class supercomputer, during the HPC User Forum, held virtually this week on account of pandemic. Exascale Computing Project director Doug Kothe reviewed some of the 'early exascale hardware' at Argonne, Oak Ridge and NERSC (Perlmutter), while Ti Leggett, Deputy Project Director & Deputy Director... Read more…

Ahead of ‘Dojo,’ Tesla Reveals Its Massive Precursor Supercomputer

June 22, 2021

In spring 2019, Tesla made cryptic reference to a project called Dojo, a “super-powerful training computer” for video data processing. Then, in summer 2020, Tesla CEO Elon Musk tweeted: “Tesla is developing a [neural network] training computer called Dojo to process truly vast amounts of video data. It’s a beast! … A truly useful exaflop at de facto FP32.” Read more…

Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

May 27, 2021

A ribbon-cutting ceremony held virtually at Berkeley Lab's National Energy Research Scientific Computing Center (NERSC) today marked the official launch of Perlmutter – aka NERSC-9 – the GPU-accelerated supercomputer built by HPE in partnership with Nvidia and AMD. Read more…

Esperanto, Silicon in Hand, Champions the Efficiency of Its 1,092-Core RISC-V Chip

August 27, 2021

Esperanto Technologies made waves last December when it announced ET-SoC-1, a new RISC-V-based chip aimed at machine learning that packed nearly 1,100 cores onto a package small enough to fit six times over on a single PCIe card. Now, Esperanto is back, silicon in-hand and taking aim... Read more…

Enter Dojo: Tesla Reveals Design for Modular Supercomputer & D1 Chip

August 20, 2021

Two months ago, Tesla revealed a massive GPU cluster that it said was “roughly the number five supercomputer in the world,” and which was just a precursor to Tesla’s real supercomputing moonshot: the long-rumored, little-detailed Dojo system. “We’ve been scaling our neural network training compute dramatically over the last few years,” said Milan Kovac, Tesla’s director of autopilot engineering. Read more…

CentOS Replacement Rocky Linux Is Now in GA and Under Independent Control

June 21, 2021

The Rocky Enterprise Software Foundation (RESF) is announcing the general availability of Rocky Linux, release 8.4, designed as a drop-in replacement for the soon-to-be discontinued CentOS. The GA release is launching six-and-a-half months after Red Hat deprecated its support for the widely popular, free CentOS server operating system. The Rocky Linux development effort... Read more…

Google Launches TPU v4 AI Chips

May 20, 2021

Google CEO Sundar Pichai spoke for only one minute and 42 seconds about the company’s latest TPU v4 Tensor Processing Units during his keynote at the Google I Read more…

Intel Completes LLVM Adoption; Will End Updates to Classic C/C++ Compilers in Future

August 10, 2021

Intel reported in a blog this week that its adoption of the open source LLVM architecture for Intel’s C/C++ compiler is complete. The transition is part of In Read more…

AMD-Xilinx Deal Gains UK, EU Approvals — China’s Decision Still Pending

July 1, 2021

AMD’s planned acquisition of FPGA maker Xilinx is now in the hands of Chinese regulators after needed antitrust approvals for the $35 billion deal were receiv Read more…

Leading Solution Providers

Contributors

Hot Chips: Here Come the DPUs and IPUs from Arm, Nvidia and Intel

August 25, 2021

The emergence of data processing units (DPU) and infrastructure processing units (IPU) as potentially important pieces in cloud and datacenter architectures was Read more…

Julia Update: Adoption Keeps Climbing; Is It a Python Challenger?

January 13, 2021

The rapid adoption of Julia, the open source, high level programing language with roots at MIT, shows no sign of slowing according to data from Julialang.org. I Read more…

10nm, 7nm, 5nm…. Should the Chip Nanometer Metric Be Replaced?

June 1, 2020

The biggest cool factor in server chips is the nanometer. AMD beating Intel to a CPU built on a 7nm process node* – with 5nm and 3nm on the way – has been i Read more…

HPE Wins $2B GreenLake HPC-as-a-Service Deal with NSA

September 1, 2021

In the heated, oft-contentious, government IT space, HPE has won a massive $2 billion contract to provide HPC and AI services to the United States’ National Security Agency (NSA). Following on the heels of the now-canceled $10 billion JEDI contract (reissued as JWCC) and a $10 billion... Read more…

Quantum Roundup: IBM, Rigetti, Phasecraft, Oxford QC, China, and More

July 13, 2021

IBM yesterday announced a proof for a quantum ML algorithm. A week ago, it unveiled a new topology for its quantum processors. Last Friday, the Technical Univer Read more…

Intel Launches 10nm ‘Ice Lake’ Datacenter CPU with Up to 40 Cores

April 6, 2021

The wait is over. Today Intel officially launched its 10nm datacenter CPU, the third-generation Intel Xeon Scalable processor, codenamed Ice Lake. With up to 40 Read more…

Frontier to Meet 20MW Exascale Power Target Set by DARPA in 2008

July 14, 2021

After more than a decade of planning, the United States’ first exascale computer, Frontier, is set to arrive at Oak Ridge National Laboratory (ORNL) later this year. Crossing this “1,000x” horizon required overcoming four major challenges: power demand, reliability, extreme parallelism and data movement. Read more…

Intel Unveils New Node Names; Sapphire Rapids Is Now an ‘Intel 7’ CPU

July 27, 2021

What's a preeminent chip company to do when its process node technology lags the competition by (roughly) one generation, but outmoded naming conventions make it seem like it's two nodes behind? For Intel, the response was to change how it refers to its nodes with the aim of better reflecting its positioning within the leadership semiconductor manufacturing space. Intel revealed its new node nomenclature, and... Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire