Back to the Future of Serial Speed?

By Bill Sembrat

October 30, 2013

For the last few decades we have had great increases in performance. Since going to “off-the-shelf components” and riding on the tails of increasing processor improvements along with ever greater number of chips and cores some have come to realize that this can’t go on forever.

We have filled up a number of ever-bigger rooms with racks and racks until we have come face to face with having to own your own power company just for the power needed to run these complexes. On another front I would have expected more concerns about the high cost of these high-end systems.

It also seems that others are raising doubts because recently there has been a lot questions, concerns, discussions, comments, articles about the problems, and enough concerns about the way forward to even provide extra funding, but few seem to looking at root issues and saying that moving forward things need to change.

We have been lucky to ride the train so far, much longer and further than those of us would have ever imaged. But, now it has become ever harder and more costly to keep the train going. So lets take a deeper look at the road traveled. About 20 years ago the HPC train switched paths from custom processors and custom systems to off-the-shelf processors and systems. It has been a good and fruitful ride.

I think it would be very wise to notice that recently most of the speed improvement has come from the parallel side. I (we) was always highly focused on the serial side. At this point maybe I should say something about myself. I was fortunate to have worked with Seymour Cray for many years. So, the we, I am referring to, is my involvement and experience with Seymour and Seymour’s machines. Seymour was never in any race and not really concerned about what someone else may be doing or not doing, but always interested in exploring and pushing serial speed on real workloads. We were mainly focused on serial speed because it kept things simple and made systems easier to use, easier to program, with less overhead and higher system efficiencies.

Few may have ever talked to Seymour about serial speed vs. parallel speed, but I can tell you that Seymour was always quite aware and disciplined himself to stay focused on serial speed improvements. He felt he could contribute more, add more value, was personally more challenging, and he very much like to work on, enjoyed working on serial improvements.

Although, he would never admit it, he also knew that he was the “king” of serial speed. Just a side comment, Seymour was also interested in exploring the far end of parallel processors and we had a running prototype parallel machine that had a design goal of 30,000,000 processors, but that is quite another story. Getting back to this topic we were really always highly focused on serial speed with the “Cray’s.” Over the last 20 odd years the current off-the-shelf path has relied on serial speed improvements but ever more increasingly on greater and greater parallel speed improvements. Parallel speed improvements has, naturally, associated with it higher overhead and power costs along with lower system efficiencies and now ever higher costs to get into the top of the list.

So to get large cost effective improvements I think that we now need to re-focus back to serial speed improvements. I believe that by addressing serial speed improvements that speed improvements of 50X+ can be achieved because we were addressing root level changes that could lead to these kinds of improvements. This quickly leads one to a startling conclusion that memory can’t keep up, does not work and becomes the big elephant in the room. So you really need to look at how memory is used and really the only way to see it is to wipe the slate clean and get rid of memory. In order to think about it you need first get rid of it and start again fresh. Very few may be up to the task of starting fresh with a blank sheet. This is a rather hard task and not as simple as one may think.

While Seymour always preferred blank paper pads with faint light blue lines and number 2 pencils, at a time, it seemed, everyone started using computers and in some cases even “Cray” Super Computers to design the “next” machine. Einstein never needed or used a computer for his theories and I would guess that Peter Higgs didn’t use one either. Giant leaps and great things seem to come from very simple root ideas. Also can-do-positive attitudes play a most, maybe the most important part, even over seemingly impossible tasks.

The memory model currently used is largely based on a 70-year-old model. Oh, if you can wake up the guys that came up with the model, that were in the farm house/barn in Princeton at the time, they would be quite amazed at the great strides and progress but in very short order they would be able to program today’s machines – so in some ways things really haven’t changed much. Other areas will need to be addressed and changed, but memory is the first and most looming problem. Because these changes are deeper root issues they should be hidden from users and even and from most of the vast layers of existing software. Funny you may think that this is new but most was tried and used years ago, but never commercialized and sometimes discarded because of lack of the-then-current available technology.

Well, yes I do believe that by addressing some deep root issues that over time large serial speed improvements can be achieved, but to use them you will quickly come to the several conclusions including that you must deal with new ways to see and use memory and all that this implies. To achieve very large improvements, I think, the focus needs to be on very several very fundamental and root changes and then apply all the parallel knowledge and improvements made over the last 20 years. Now here, I believe, may be a bigger problem. In the US we have been blessed with chip and system vendors that have been able to supply ever-increasing speeds and lots of chips and cores so we have been glued to that path but others may be unencumbered, highly motivated and more able to do something new and different.

Although they may operate under different set of rules and have additional other problems they do not have as much invested in existing ideas, enterprises, hard plant and equipment; and may be less locked in and may be more willing to change pathways. So I am concerned with our current shortsighted attitude and lack of “Americanism” in keeping the leadership local.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industy updates delivered to you every week!

Microsoft Wants to Speed Quantum Development

December 12, 2017

Quantum computing continues to make headlines in what remains of 2017 as tech giants jockey to establish a pole position in the race toward commercialization of quantum. This week, Microsoft took the next step in advanci Read more…

By Tiffany Trader

ESnet Now Moving More Than 1 Petabyte/wk

December 12, 2017

Optimizing ESnet (Energy Sciences Network), the world's fastest network for science, is an ongoing process. Recently a two-year collaboration by ESnet users – the Petascale DTN Project – achieved its ambitious goal t Read more…

HPC-as-a-Service Finds Toehold in Iceland

December 11, 2017

While high-demand workloads (e.g., bitcoin mining) can overheat data center cooling capabilities, at least one data center infrastructure provider has announced an HPC-as-a-service offering that features 100 percent fre Read more…

By Doug Black

HPE Extreme Performance Solutions

Explore the Origins of Space with COSMOS and Memory-Driven Computing

From the formation of black holes to the origins of space, data is the key to unlocking the secrets of the early universe. Read more…

HPC Iron, Soft, Data, People – It Takes an Ecosystem!

December 11, 2017

Cutting edge advanced computing hardware (aka big iron) does not stand by itself. These computers are the pinnacle of a myriad of technologies that must be carefully woven together by people to create the computational c Read more…

By Alex R. Larzelere

Microsoft Wants to Speed Quantum Development

December 12, 2017

Quantum computing continues to make headlines in what remains of 2017 as tech giants jockey to establish a pole position in the race toward commercialization of Read more…

By Tiffany Trader

HPC Iron, Soft, Data, People – It Takes an Ecosystem!

December 11, 2017

Cutting edge advanced computing hardware (aka big iron) does not stand by itself. These computers are the pinnacle of a myriad of technologies that must be care Read more…

By Alex R. Larzelere

IBM Begins Power9 Rollout with Backing from DOE, Google

December 6, 2017

After over a year of buildup, IBM is unveiling its first Power9 system based on the same architecture as the Department of Energy CORAL supercomputers, Summit a Read more…

By Tiffany Trader

Microsoft Spins Cycle Computing into Core Azure Product

December 5, 2017

Last August, cloud giant Microsoft acquired HPC cloud orchestration pioneer Cycle Computing. Since then the focus has been on integrating Cycle’s organization Read more…

By John Russell

GlobalFoundries, Ayar Labs Team Up to Commercialize Optical I/O

December 4, 2017

GlobalFoundries (GF) and Ayar Labs, a startup focused on using light, instead of electricity, to transfer data between chips, today announced they've entered in Read more…

By Tiffany Trader

HPE In-Memory Platform Comes to COSMOS

November 30, 2017

Hewlett Packard Enterprise is on a mission to accelerate space research. In August, it sent the first commercial-off-the-shelf HPC system into space for testing Read more…

By Tiffany Trader

SC17 Cluster Competition: Who Won and Why? Results Analyzed and Over-Analyzed

November 28, 2017

Everyone by now knows that Nanyang Technological University of Singapore (NTU) took home the highest LINPACK Award and the Overall Championship from the recently concluded SC17 Student Cluster Competition. We also already know how the teams did in the Highest LINPACK and Highest HPCG competitions, with Nanyang grabbing bragging rights for both benchmarks. Read more…

By Dan Olds

Perspective: What Really Happened at SC17?

November 22, 2017

SC is over. Now comes the myriad of follow-ups. Inboxes are filled with templated emails from vendors and other exhibitors hoping to win a place in the post-SC thinking of booth visitors. Attendees of tutorials, workshops and other technical sessions will be inundated with requests for feedback. Read more…

By Andrew Jones

US Coalesces Plans for First Exascale Supercomputer: Aurora in 2021

September 27, 2017

At the Advanced Scientific Computing Advisory Committee (ASCAC) meeting, in Arlington, Va., yesterday (Sept. 26), it was revealed that the "Aurora" supercompute Read more…

By Tiffany Trader

NERSC Scales Scientific Deep Learning to 15 Petaflops

August 28, 2017

A collaborative effort between Intel, NERSC and Stanford has delivered the first 15-petaflops deep learning software running on HPC platforms and is, according Read more…

By Rob Farber

Oracle Layoffs Reportedly Hit SPARC and Solaris Hard

September 7, 2017

Oracle’s latest layoffs have many wondering if this is the end of the line for the SPARC processor and Solaris OS development. As reported by multiple sources Read more…

By John Russell

AMD Showcases Growing Portfolio of EPYC and Radeon-based Systems at SC17

November 13, 2017

AMD’s charge back into HPC and the datacenter is on full display at SC17. Having launched the EPYC processor line in June along with its MI25 GPU the focus he Read more…

By John Russell

Nvidia Responds to Google TPU Benchmarking

April 10, 2017

Nvidia highlights strengths of its newest GPU silicon in response to Google's report on the performance and energy advantages of its custom tensor processor. Read more…

By Tiffany Trader

Japan Unveils Quantum Neural Network

November 22, 2017

The U.S. and China are leading the race toward productive quantum computing, but it's early enough that ultimate leadership is still something of an open questi Read more…

By Tiffany Trader

GlobalFoundries Puts Wind in AMD’s Sails with 12nm FinFET

September 24, 2017

From its annual tech conference last week (Sept. 20), where GlobalFoundries welcomed more than 600 semiconductor professionals (reaching the Santa Clara venue Read more…

By Tiffany Trader

Google Releases Deeplearn.js to Further Democratize Machine Learning

August 17, 2017

Spreading the use of machine learning tools is one of the goals of Google’s PAIR (People + AI Research) initiative, which was introduced in early July. Last w Read more…

By John Russell

Leading Solution Providers

Amazon Debuts New AMD-based GPU Instances for Graphics Acceleration

September 12, 2017

Last week Amazon Web Services (AWS) streaming service, AppStream 2.0, introduced a new GPU instance called Graphics Design intended to accelerate graphics. The Read more…

By John Russell

Perspective: What Really Happened at SC17?

November 22, 2017

SC is over. Now comes the myriad of follow-ups. Inboxes are filled with templated emails from vendors and other exhibitors hoping to win a place in the post-SC thinking of booth visitors. Attendees of tutorials, workshops and other technical sessions will be inundated with requests for feedback. Read more…

By Andrew Jones

EU Funds 20 Million Euro ARM+FPGA Exascale Project

September 7, 2017

At the Barcelona Supercomputer Centre on Wednesday (Sept. 6), 16 partners gathered to launch the EuroEXA project, which invests €20 million over three-and-a-half years into exascale-focused research and development. Led by the Horizon 2020 program, EuroEXA picks up the banner of a triad of partner projects — ExaNeSt, EcoScale and ExaNoDe — building on their work... Read more…

By Tiffany Trader

Delays, Smoke, Records & Markets – A Candid Conversation with Cray CEO Peter Ungaro

October 5, 2017

Earlier this month, Tom Tabor, publisher of HPCwire and I had a very personal conversation with Cray CEO Peter Ungaro. Cray has been on something of a Cinderell Read more…

By Tiffany Trader & Tom Tabor

Tensors Come of Age: Why the AI Revolution Will Help HPC

November 13, 2017

Thirty years ago, parallel computing was coming of age. A bitter battle began between stalwart vector computing supporters and advocates of various approaches to parallel computing. IBM skeptic Alan Karp, reacting to announcements of nCUBE’s 1024-microprocessor system and Thinking Machines’ 65,536-element array, made a public $100 wager that no one could get a parallel speedup of over 200 on real HPC workloads. Read more…

By John Gustafson & Lenore Mullin

Flipping the Flops and Reading the Top500 Tea Leaves

November 13, 2017

The 50th edition of the Top500 list, the biannual publication of the world’s fastest supercomputers based on public Linpack benchmarking results, was released Read more…

By Tiffany Trader

IBM Begins Power9 Rollout with Backing from DOE, Google

December 6, 2017

After over a year of buildup, IBM is unveiling its first Power9 system based on the same architecture as the Department of Energy CORAL supercomputers, Summit a Read more…

By Tiffany Trader

Intel Launches Software Tools to Ease FPGA Programming

September 5, 2017

Field Programmable Gate Arrays (FPGAs) have a reputation for being difficult to program, requiring expertise in specialty languages, like Verilog or VHDL. Easin Read more…

By Tiffany Trader

  • arrow
  • Click Here for More Headlines
  • arrow
Share This