The Cluster Challenge, now in its second year as part of the technical program at the annual Supercomputing conference, is billed as an event to showcase the computational power that even inexpensive clusters have and, according to the Web site, the ability of potential users around the world “to harness open source software to solve interesting and important problems.” The competition pits teams of undergraduates against one another to see who can build and configure a cluster that accomplishes the most work, using real computational codes in the least amount of time.

But according to Brent Gorda, deputy for Advanced Technology at Lawrence Livermore National Lab and co-chair of the Cluster Challenge, there is a larger mission being served. “We started based on the assertion that clusters have arrived, that they are accessible to any size of business interested in having access to them,” says Gorda. “To that end we structured the contest so that teams would be made up entirely of undergraduate students. This sends the message that a company can find talent at their local college or university to work with this technology.”

To further cement the Challenge as an example for industry, vendors are invited to partner with the Challenge teams and support the event. A team consists of up to six undergraduate students, a supervisor, and an optional vendor partner. The supervisor is responsible for the team but cannot provide technical assistance (though he or she is encouraged to keep the pizza and coke flowing at all times). The vendor partners are encouraged to contribute training, financial support, and equipment. I’m sure a t-shirt or two would be welcomed as well.

Six teams participated in the competition last year — The University of Colorado, Purdue University, Indiana University, University of Alberta, Stony Brook University, and National Tsing Hua University in Taiwan — and were supported by a range of vendors supplying both hardware and software.

So what kind of clusters are we talking about here? In order to level the playing field, there are tight physical constraints on the systems. The cluster compute hardware (processors, switch, storage, etc.) must fit into a single 42U rack powered by two 120-volt, 13-amp circuits. According to Gorda, even with this physical limitation the systems built last year were substantial, “At the event one team achieved an amazing 420 GFlops of performance on HPL (Linpack). That score would have put them in the Top500 in 2003, only four years earlier.” For even more perspective, a system like this would have been number one in 1995 and still in the top 20 as late as 1998.

This is a powerful snapshot of the impact of the democratization of supercomputing. As Gorda puts it, “Half a dozen undergraduates have the ability to stand up systems that only the national laboratories could afford nine years ago. The science we were doing at that time and on those systems was significant (including the ASCI efforts). Now this can literally be done in a garage by a hobbyist.”

The Cluster Challenge also has a key role to play in recruiting and training the next generation of HPC professionals, and SC is already seeing the fruits of last year’s competition: four of the six teams from last year’s Challenge are giving presentations this month, and three of the colleges represented in the competition are now teaching cluster-oriented classes. Last year’s team from Taiwan is working with SC to replicate the event in Taiwan, extending the educational reach around the globe.

The winning team in last year’s Challenge was out of the University of Alberta, supported by vendor partner SGI. This team included five U of Alberta undergrads, one high school student, faculty coach Paul Lu, and a variety of other coaches and contributors. Lu described an intensive effort by the team in preparing for the competition, including work to understand the applications used in the workload (POP, GAMESS, and others) and the factors influencing their performance. Cluster work began in earnest when the SGI hardware arrived in August, accompanied by twice-weekly team meetings, lots of sub-team meetings in between, and weekend build sessions. In a testament to the level of effort invested by these teams of non-experts, despite all the work they did to get ready for their winning performance, there were still things left undone. According to Lu, the team only did “…about 50% of what we wanted to do in preparation, but we did accomplish a lot in the run up.”

What was the key to Alberta’s victory? Paul Lu points to the deep commitment of team, “As a coach, I learned that you cannot really motivate a team that does not already want to win. The student team members were motivated and talented.” Lu also credits their vendor partner, “…SGI was great with engineering support (when needed) and, for example, in providing some last minute, expensive memory upgrades when we determined it could make a big difference with the GAMESS application.”

Lu’s final recommendation to others considering the Challenge? Do it! “I would encourage any school to send a team. In most undergraduate programs, there are so few opportunities to see something end-to-end like taking commercial hardware, setting it up, installing and learning to run several real applications, visualize the output, and then deal with ‘real world’ problems like power budgets, power outages, and working under pressure.”

If you are interested in taking part in this year’s Cluster Challenge, you still have time to get in on this incredible learning experience. You can get more information about the various Challenges sponsored at SC08 — including very detailed information about the Cluster Challenge — at the SC08 challenges Web site (http://sc08.supercomputing.org/?pg=challenges.html). The deadline for entry into this year’s competition is July 31. And if you are a company interested in sponsoring a team, or you are part of a team in search of a vendor, get in touch with the Challenge chairs by sending an email to [email protected].

SC08, the international conference for high performance computing, networking, storage and analysis, will celebrate 20 years of unleashing the power of HPC this year, following the traditions set by the first SC Conference in 1988. Sponsored by the ACM and IEEE Computer Society, SC08 will showcase how high performance computing, networking, storage and analysis lead to advances in research, education and commerce. This premiere international conference includes technical and education programs, workshops, tutorials, an exhibit area, demonstrations and hands-on learning. For more information, visit http://sc08.supercomputing.org/.