This week saw the inaugural meeting of HPC Horizons, a new community of HPC users, vendors, and policymakers dedicated to collaborative discussion of forward-looking topics that push the boundaries of High Productivity Computing. The two-day conference had 125 attendees and featured speakers that represented both traditional and emerging HPC applications. Tabor Communications, the parent company of HPCwire and Tabor Research and the founders of HPC Horizons, billed the event as a prelude to ongoing online discussions among a wider member audience.

The presentations, panel discussions, and community action groups provided a compelling view of future trends in HPC that will be important across both new and established HPC application spaces. Tabor Research observed many common threads and provides the following analysis.

Data Ingest

Massive ingest of data as a limiting factor to HPC scalability was mentioned by several speakers, including a keynote address by Dr. J. Craig Venter, well known throughout the industry for his work in mapping the human genome. The Venter Institute is now investigating microbial life forms, which represent over half the biomass on earth and may hold the key to finding sustainable bio-energy sources or understanding the synthesis of life. Dr. Venter noted that a major challenge currently facing biologists is to gather as much genetic information from the microbial world as possible. A barrel of seawater can yield thousands of new microbial species to analyze and categorize.

Deborah Gracio of Pacific Northwest National Laboratories also discussed data-intensive and data-streaming applications. She noted that advances in high-throughput sensors can easily overwhelm data storage capabilities, giving the example of a proteomics mass spectrometer that was run at about 1 percent of capability because it would take over all of PNNL's storage within two days if run at full capacity. She noted that data filtering and analysis needs to be done close to the sensor, and the multithreaded architectures work well for problems with large irregular memory access.

This level of data influx -- massive amounts of data points for analysis coming from disperse points around the periphery of a system -- is mirrored in other types of emerging applications, such as surveillance, online gaming, logistics, virtual reality networks, or trading analysis. Many of these applications involve real-time or near real-time analysis requirements, and they involve a wide range of data or file types. To address the challenge, many users and vendors suggested the need to move computation closer to the data source.

Predictive Networks

In an opening keynote, Jaron Lanier of UC Berkeley discussed latency as applied to virtual reality systems. In a compelling analogy, he pointed out that the human brain has relatively poor latency in communication between different sections, and he posited that the reason the brain is such a fast computer is due to its predictive capabilities, with each section predicting the information it will receive from other sections ahead of data arrival and then adjust to any variances with the actual information as it comes in.

As the discussions moved toward other latency-sensitive applications, the development of predictive networks was a consistent theme. Several users suggested the need to compute ahead on likely dimensions in order to hide latencies and allow applications to run well at scale.

Distributed computation can also be used to reduce latency issues. For example, two separate views of the flight of a ball can be computed from the initial position and trajectory. Computed independently, these become two halves of a predictive system.

This concept also manifested in a few speeches that suggested the need for improvisational systems, which could be applied to either the computer systems or the human decision-making systems surrounding them. (Tom Lange of Procter & Gamble commented, "We don't sell soap; we sell clean.") By extension, systems that are predictive also need to sustain themselves when unexpected results occur.

Productivity

For a long time, the HPC community has discussed the need to measure productivity in ways that go beyond price/performance ratios. John Hurley of Boeing noted the difference between performance and productivity systems, with performance systems being typified by such factors as: application specificity to architecture, integrated systems, restricted access, cost no object, specialized technology, and tools-driven. In contrast, he described productivity systems as having data-driven architectures that run broader classes of applications with shared access, focusing on both cost and ROI, general-purpose products.

Lange also challenged the gathering with a number of controversial questions. For example, he asked the user community, "Are bigger problems worth it?" and "If software is not commercially affordable, are you willing to write your own?" He asked ISVs, "Is nature against us, and are problems just too hard to parallelize?" and "Are R&D expenditures balanced between application improvements and user interface improvements?" Finally, he asked software and hardware vendors, "Do you view national labs as potential partners or potential competitors?"

These observations raise several important issues concerning productivity, but perhaps none more important than these: How is productivity measured, and what does success look like?

Tabor Research is attempting to address these issues by offering users an interactive, online productivity analysis tool. With this tool, users will fill out profiles and questionnaires, including an allocation of the time and effort spent by different people within their organizations across various phases and tasks. The tool will then provide a peer comparison of how time is being spent and combine that data with each user's profile and what they believe to be important metrics of productivity. Each user then receives concrete recommendations that help focus incremental HPC investment. Tabor Research announced plans to introduce the tool to the Horizons group and the HPC community at large in April.

Conclusions

Several participants noted to Tabor Research that the HPC Horizons conference was unique in considering HPC as a business area as well as a technology area. One noted that it brought people together from wide variety of fields and helped identify the areas commonality between seeming disparate applications areas.

At the "horizon of HPC," new application spaces are emerging that will drive requirements in new dimensions. Many of these are similar to how traditional HPC is evolving. Tabor Research believes these trends will be significant forces in determining the shape of next-generation HPC architectures.