Like every technology-based sector, high performance computing takes its biggest leaps by the force of disruptive innovation, a term coined by the man who will keynote this year’s Supercomputing Conference (SC10) in New Orleans. Clayton M. Christensen doesn’t know a whole lot about supercomputing, but he knows a great deal about the forces that drive it.

For the past 15 years, Christensen, a professor at the Harvard Business School, has been studying how technological innovation works, how it can drive some businesses to succeed, and how it can cause others to fail spectacularly. Today he is considered one of the leading experts on innovation. At SC10, he will attempt to impart some of this wisdom to the HPC faithful.

Not a techno-geek by any means, Christensen’s focus is on the businesses end of disruptive innovation. In 1997 he penned his first book on the subject, The Innovator’s Dilemma, wherein he describes the challenges of managing innovation. Since then he’s developed a set of well-respected theories on innovation and has published a number of other books that explore different aspects of the subject. HPCwire recently got the opportunity to speak with Christensen to ask him about his work and how his theories can apply to the high performance computing industry.

From Christensen’s perspective, disruptive innovation is not a technical idea, it encompasses a business model that is at the heart of how technology is delivered to the marketplace. In a nutshell, disruptive innovation represents a new value to the marketplace, and it usually emerges as a simpler and less expensive alternative to established technologies. But it is not a market-specific concept. The way Christensen has done his research is by studying how the innovation process works in a generic sense, not by studying an industry, like high performance computing, and then developing a theory that is specifically applicable to it.

According to Christensen, there’s a basic problem the way world is designed; data is only available from what happened in the past. And it’s convincingly available only about the distant past. So when managers make predictions about the future using historical data, it tends to be very unreliable.

So how is one to predict the future? The answer is theory, says the Harvard professor. “A really good theory gets down to the fundamental insight on why the world works the way it does,” explains Christensen. “You guys are scientists and engineers and use theories all of the time in the technical dimensions. But now there is a set of theories about the business side that are very valuable.”

The group Christensen works with at Harvard has spent years developing business management models that can help predict which kind of product, service or company is likely to be successful and which will likely fail. Some of his students have had some remarkable success applying this framework to real-life situations. For example, one of Christensen’s student successfully predicted the demise of Google’s Wave communication platform, an all-encompassing web-based communication tool that the search giant put on the shelf after just four months of user trial.

The HPC business, of course, lives and breathes in a world of disruptive technologies. From the “Attack of the Killer Micros” that all but wiped out custom processor-based supercomputing in the 1990s, to today’s emergence of general-purpose GPU computing, HPC seems especially prone to being reshaped by simpler technologies from below.

Which may explain why even established HPC players like IBM, Cray, and HP often struggle to make their supercomputing businesses profitable. The challenge for the industry leaders is that they need sustaining technologies to maintain their business model, says Christensen. Disruptive technologies are not good fits for market leaders, since these companies tend to cater to customers high up the food chain. In other words, the IBMs of the world need to continually create higher value products to feed their best clients. Alternatively, they can acquire other companies whose products match their existing customer base.

Christensen’s theories actually predict this type of business interaction quite well. For example, in the 1960s, X-ray technology was the only device that let doctors people peer inside the body. But in 1971, a British company called EMI launched computed tomography (CT), a high end technology which delivered superior imaging technology since it revealed soft tissues as well. Within a year the leaders of the X-ray technology — GE, Siemens and Phillips — developed better CT technology than EMI and eventually drove them out of business.

The next medical imaging technology was Magnetic Resonance Imaging (MRI), which turned out to be any even better way to look at certain structures inside the body. But again, the early developers of MRI technology were overtaken by GE, Siemens, and Phillips. For both CT and MRI devices, the established companies found they could sell them for even better profits than X-ray machines.

On the other hand, when ultrasound technology was developed, that was a different story. Ultrasound didn’t produce crystal clear images, but the devices were inexpensive and simple to operate. Therefore it could be purchased and used as standard equipment for doctors’ offices. GE, Siemens and Phillips bypassed the ultrasound market because the financial incentives were wrong for their business structure. So a whole new set of vendors emerged for ultrasound products. It was a true disruptive innovation.

If Christensen models had been applied to startups like ClearSpeed or SiCortex, they might have revealed the technologies they developed, as good as they were, did not fit the disruptive profile at all and also did not offer a sustaining technology for larger vendors. His theories might also have predicted the recent rash of HPC software tool acquisitions of Cilk Arts, Interactive Supercomputing, RapidMind, TotalView Technologies, Visual Numerics, and Acumem. All of these tool companies had sustaining technologies of value to the larger buyers, in this case, Intel, Microsoft, and Rogue Wave Software.

So what’s the next big disruptive technology? Christensen thinks it could very well be cloud computing. According to him, the cloud is setting itself up the be a countervailing force that will cut across the mainframe and high-end computing. As such, it has the potential to usurp the established business model of HPC. “The supercomputer leaders should watch out,” he warns.