The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing
April 06, 2007
I just returned from two small conferences, CGO (Code Generation and Optimization, http://www.cgo.org/) and PPoPP (Principles and Practice of Parallel Programming, http://www.ppopp.org/). One of the key themes in both conferences, perhaps the dominating theme, was that multicore chips are here, are mainstream, and we'd better figure out how to use them.
One of the PPoPP attendees, Prof. Rudolf Eigenmann (Purdue Univ.) issued an indictment, saying that we in the parallel programming research community should be ashamed of ourselves. Single-processor systems have run out of steam, something the parallel programming community has been predicting since I was a college student. Now is the time to step up and reap the benefits of all our past work. We've had 30 years to study this problem and come up with a solution, but what's the end result? Surprise! We still have no well-accepted method to generate parallel applications.
We had a similar discussion at one of the workshops before CGO, where someone said that the programming problem would finally get solved because "now we're motivated!" This implies either that the past thirty years of investigation was being done by the wrong people, or that perhaps now with more people looking at the problem, someone will randomly stumble on a good solution. Science at its best?
Dr. Andrew Chien (Intel), one of the PPoPP keynote speakers, took issue with Eigenmann's criticism. Chien said that in fact we've had a great deal of success in parallel programming: just look at all the massively parallel systems and the applications that run on them. However, halfway through his talk was the slide "Wanted: Breakthrough Innovations in Parallel Programming." I asked how he could claim past success, then state that breakthrough innovations are needed; it sounded like a typical manager: "good job, now get back to work." He replied that in the past, parallel programming meant high performance. Now, parallel programming means spreadsheets, games, email, and applications on your laptop. It's a different target environment, with a different class of programmer, and different expectations.
So parallel programming is hard. Hey, sequential programming is hard too. Adding parallelism just makes it harder. Two of the major problems are expressing the parallelism and synchronizing or communicating between the parallel threads or activities.
An approach to the synchronization problem that is gaining lots of traction is transactional memory, or TM. This takes a page from the successful database community, which has long had to deal with many processes simultaneously accessing and modifying a shared database. Rather than letting a process lock the database, the process executes a transaction; the transaction may involve adding, removing, or modifying relations in the database. While in the middle of a transaction, no other process can see the modifications; at the end, the process commits the results. The database then atomically exposes all the updates to the other processes. In particular, this lets multiple processes modify disjoint parts of the database in parallel without conflict. If two processes try to update the same data at the same time, the first commit will succeed, and the second will fail. The application then has to restart its transaction from the start, since it may have made some decisions based on values that are now stale.
Moving transactions to the parallel programming world means rather than updating shared data in a critical section, the model is to enter a transaction, perform the updates, then commit the changes. The implementation must buffer the modifications until the commit, then atomically commit all the modifications at once. As in a database, if some other parallel thread had made changes to the same shared data, the commit will fail, and the transaction must restart and retry. The expected behavior is that most transactions will succeed, so the retry overhead is quite low.
However, it's still open as to how to implement the transaction buffering and commit. Modified caches could be used to buffer stores, unless you run out of associativity on the cache line. A separate transactional store queue could be designed, but the size of the queue would limit the size of the transaction.
There are other problems as well. Imagine two transactions, A and B, reading and modifying the same shared data. Suppose transaction B finishes and commits while A is still working. If transaction A then reads some data that was modified by B, A's commit is likely to fail (in fact, may be guaranteed to fail), since some of the data it read was before B's commit and some was read after. In fact, the inconsistency may cause A to generate a fault (suppose B allocated or freed a pointer), or loop infinitely; one speaker termed A a "zombie transaction," the walking dead. It can be important to detect and kill zombies before they get to the commit state, to avoid spurious faults.
In managed software environments (think Java or C#), these problems can be handled in software, and transactions are likely to be successful there. However, it remains unclear how long it will be before transactions can migrate into HPC.
Page: 1 of 3(Digg, Technorati, more)
PGI Accelerator™ Fortran 95/03 and C99 compilers for x64+NVIDIA
Accelerate applications on x64+GPU platforms by adding OpenMP-like compiler directives to existing Fortran and C programs. Available now for Linux, MacOS and Windows. Download a free 15 day trial.
Platform HPC Workgroup Manager
Platform HPC Workgroup Manager integrates all the cluster productivity tools you need to deploy, run and manage your HPC environment.
Mar 18 | ChannelWeb | Westmere parts already showing up in HPC machines. Read more...
Mar 17 | The Register | But what about the tier ones? Read more...
Mar 17 | Cadalyst Magazine | A new generation of workstations is changing the nature of technical computing. Read more...
Mar 17 | Linux Magazine | Latest iteration of Sun Grid Engine able to tap into Cloud. Read more...
Mar 16 | Bio-IT World | Biotech firm builds genetic models from patient data. Read more...
Jan 12 | | In-depth look at vSMP Foundation server virtualization technology, technical implementation, use cases and capabilities. The technical whitepaper provides an architectural overview and details on the three vSMP Foundation products: vSMP Foundation for SMP, vSMP Foundation for Cluster and vSMP Foundation for Cloud.
Jan 18 | | This white paper discusses Gore’s copper cable assemblies, and how they continue to exceed the standards for providing reliable, cost-effective solutions for high-performance computer applications.
Join this online panel discussion for live Q&A with leading industry experts, analysts, and end-users to discuss the latest innovations, best practices, barriers to implementation, and measurable benefits of server virtualization with a particular focus on today's real world solutions.
Learn about scalable fault-tolerant architectures and examples of energy efficient and scalable supercomputing clusters using dual QDR InfiniBand to combine capacity computing with network failover capabilities with the help of programming languages such as MPI and a robust Linux cluster management package.
LIVE@SCO9: The IBM team discusses new innovations in hardware, software and services that help clients better understand their workloads and get insight from their R&D efforts. Technology demonstrations include the soon-to-be-released Power7 HPC processor, the DCS990 system with 2.4 petabytes of storage, the xCAT management tool, secure HPC cloud computing and more. Winners of two HPCwire Readers' and Editors’ Choice Awards! Take the IBM virtual tour at SC09 or more information go online to: http://www-03.ibm.com/systems/deepcomputing/sc09.html