The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing
December 04, 2008
Planning to Scale in a World with Less "Embarrassing Parallelism"
The need to write scalable applications has been important for programmers in the HPC community for years. Now, the proliferation of multi-core processors is making scalability a top priority for millions of programmers. Previously, HPC programs that scaled very well were called "embarrassingly parallel," but it is inevitable that we will increasingly settle for "good enough" parallelism. Any scaling, and a design to keep scaling, will be of paramount importance. Efficient designs will be relatively unimportant if a program cannot scale into the future. "Forward scaling" will become a key part of good program design. Less "embarrassing" parallelism will become the norm for most programmers.
The move to multi-core processors means that all programmers will need to incorporate new techniques to write scalable applications and adopt a new mindset to achieve further gains in application performance. In the past, a poorly written sequential program would still generally speed up as processor clock rates increased. However, we are now faced with the reality that a poorly written concurrent program will generally not speed up as more processor cores become available. Programmers today need to "think parallel" and write parallel programs to harvest the potential of multiple processor cores.
Thinking and writing parallel are not easy feats. As computer science professor Andrew S. Tanenbaum recently observed at the USENIX '08 conference, "Sequential programming is really hard, and parallel programming is a step beyond that." Obviously, we would prefer to not do these difficult tasks over and over again as future-generation architectures are released. Forward scaling offers a means of avoiding much of that repetitive work.
Free lunch, not free beer
As the writer and programmer Herb Sutter announced a few years ago, "The free lunch is over." Programmers can no longer count on the acceleration of their applications along with a continuous rise in clock rates. Is there a new free lunch on the horizon? Clearly programmers want a way to maximize the long-term returns for the time and effort they invest today.
Some have suggested that scaling is the new free lunch. Scaling provides a way to avoid rewriting code and redoing difficult tasks with the release of each new architecture. When our applications are successful at scaling, our software will run well on today's dual- and quad-core processors while also capitalizing on tomorrow's many-core architectures.
Is scaling the new free lunch? It's probably not free in the sense of "free beer" (as Richard Stallman might say). Scaling takes some work. The goal is to make investments today that will continue to pay off in the future.
Forward scaling defined
Forward scaling is an approach to designing software that will deliver outstanding performance on today's multi-core processors and will scale that performance on tomorrow's many-core architectures. The objective of forward scaling is to find techniques that will help avoid a complete code rewrite with the addition of more processor cores. Is this possible? In many cases, it appears so. There is no perfect solution but the options available today can assist a great deal.
Page: 1 of 3(Digg, Technorati, more)
PGI Accelerator™ Fortran 95/03 and C99 compilers for x64+NVIDIA
Accelerate applications on x64+GPU platforms by adding OpenMP-like compiler directives to existing Fortran and C programs. Available now for Linux, MacOS and Windows. Download a free 15 day trial.
Platform HPC Workgroup Manager
Platform HPC Workgroup Manager integrates all the cluster productivity tools you need to deploy, run and manage your HPC environment.
Mar 19 | OfficialWire | New super to support intelligence work Down Under. Read more...
Mar 18 | ChannelWeb | Westmere parts already showing up in HPC machines. Read more...
Mar 17 | The Register | But what about the tier ones? Read more...
Mar 17 | Cadalyst Magazine | A new generation of workstations is changing the nature of technical computing. Read more...
Mar 17 | Linux Magazine | Latest iteration of Sun Grid Engine able to tap into Cloud. Read more...
Jan 12 | | In-depth look at vSMP Foundation server virtualization technology, technical implementation, use cases and capabilities. The technical whitepaper provides an architectural overview and details on the three vSMP Foundation products: vSMP Foundation for SMP, vSMP Foundation for Cluster and vSMP Foundation for Cloud.
Jan 18 | | This white paper discusses Gore’s copper cable assemblies, and how they continue to exceed the standards for providing reliable, cost-effective solutions for high-performance computer applications.
Join this online panel discussion for live Q&A with leading industry experts, analysts, and end-users to discuss the latest innovations, best practices, barriers to implementation, and measurable benefits of server virtualization with a particular focus on today's real world solutions.
Learn about scalable fault-tolerant architectures and examples of energy efficient and scalable supercomputing clusters using dual QDR InfiniBand to combine capacity computing with network failover capabilities with the help of programming languages such as MPI and a robust Linux cluster management package.
LIVE@SCO9: The IBM team discusses new innovations in hardware, software and services that help clients better understand their workloads and get insight from their R&D efforts. Technology demonstrations include the soon-to-be-released Power7 HPC processor, the DCS990 system with 2.4 petabytes of storage, the xCAT management tool, secure HPC cloud computing and more. Winners of two HPCwire Readers' and Editors’ Choice Awards! Take the IBM virtual tour at SC09 or more information go online to: http://www-03.ibm.com/systems/deepcomputing/sc09.html