The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing
March 14, 2008
This week saw the inaugural meeting of HPC Horizons, a new community of HPC users, vendors, and policymakers dedicated to collaborative discussion of forward-looking topics that push the boundaries of High Productivity Computing. The two-day conference had 125 attendees and featured speakers that represented both traditional and emerging HPC applications. Tabor Communications, the parent company of HPCwire and Tabor Research and the founders of HPC Horizons, billed the event as a prelude to ongoing online discussions among a wider member audience.
The presentations, panel discussions, and community action groups provided a compelling view of future trends in HPC that will be important across both new and established HPC application spaces. Tabor Research observed many common threads and provides the following analysis.
Data Ingest
Massive ingest of data as a limiting factor to HPC scalability was mentioned by several speakers, including a keynote address by Dr. J. Craig Venter, well known throughout the industry for his work in mapping the human genome. The Venter Institute is now investigating microbial life forms, which represent over half the biomass on earth and may hold the key to finding sustainable bio-energy sources or understanding the synthesis of life. Dr. Venter noted that a major challenge currently facing biologists is to gather as much genetic information from the microbial world as possible. A barrel of seawater can yield thousands of new microbial species to analyze and categorize.
Deborah Gracio of Pacific Northwest National Laboratories also discussed data-intensive and data-streaming applications. She noted that advances in high-throughput sensors can easily overwhelm data storage capabilities, giving the example of a proteomics mass spectrometer that was run at about 1 percent of capability because it would take over all of PNNL's storage within two days if run at full capacity. She noted that data filtering and analysis needs to be done close to the sensor, and the multithreaded architectures work well for problems with large irregular memory access.
This level of data influx -- massive amounts of data points for analysis coming from disperse points around the periphery of a system -- is mirrored in other types of emerging applications, such as surveillance, online gaming, logistics, virtual reality networks, or trading analysis. Many of these applications involve real-time or near real-time analysis requirements, and they involve a wide range of data or file types. To address the challenge, many users and vendors suggested the need to move computation closer to the data source.
Predictive Networks
In an opening keynote, Jaron Lanier of UC Berkeley discussed latency as applied to virtual reality systems. In a compelling analogy, he pointed out that the human brain has relatively poor latency in communication between different sections, and he posited that the reason the brain is such a fast computer is due to its predictive capabilities, with each section predicting the information it will receive from other sections ahead of data arrival and then adjust to any variances with the actual information as it comes in.
As the discussions moved toward other latency-sensitive applications, the development of predictive networks was a consistent theme. Several users suggested the need to compute ahead on likely dimensions in order to hide latencies and allow applications to run well at scale.
Distributed computation can also be used to reduce latency issues. For example, two separate views of the flight of a ball can be computed from the initial position and trajectory. Computed independently, these become two halves of a predictive system.
This concept also manifested in a few speeches that suggested the need for improvisational systems, which could be applied to either the computer systems or the human decision-making systems surrounding them. (Tom Lange of Procter & Gamble commented, "We don't sell soap; we sell clean.") By extension, systems that are predictive also need to sustain themselves when unexpected results occur.
Productivity
For a long time, the HPC community has discussed the need to measure productivity in ways that go beyond price/performance ratios. John Hurley of Boeing noted the difference between performance and productivity systems, with performance systems being typified by such factors as: application specificity to architecture, integrated systems, restricted access, cost no object, specialized technology, and tools-driven. In contrast, he described productivity systems as having data-driven architectures that run broader classes of applications with shared access, focusing on both cost and ROI, general-purpose products.
Lange also challenged the gathering with a number of controversial questions. For example, he asked the user community, "Are bigger problems worth it?" and "If software is not commercially affordable, are you willing to write your own?" He asked ISVs, "Is nature against us, and are problems just too hard to parallelize?" and "Are R&D expenditures balanced between application improvements and user interface improvements?" Finally, he asked software and hardware vendors, "Do you view national labs as potential partners or potential competitors?"
These observations raise several important issues concerning productivity, but perhaps none more important than these: How is productivity measured, and what does success look like?
Tabor Research is attempting to address these issues by offering users an interactive, online productivity analysis tool. With this tool, users will fill out profiles and questionnaires, including an allocation of the time and effort spent by different people within their organizations across various phases and tasks. The tool will then provide a peer comparison of how time is being spent and combine that data with each user's profile and what they believe to be important metrics of productivity. Each user then receives concrete recommendations that help focus incremental HPC investment. Tabor Research announced plans to introduce the tool to the Horizons group and the HPC community at large in April.
Conclusions
Several participants noted to Tabor Research that the HPC Horizons conference was unique in considering HPC as a business area as well as a technology area. One noted that it brought people together from wide variety of fields and helped identify the areas commonality between seeming disparate applications areas.
At the "horizon of HPC," new application spaces are emerging that will drive requirements in new dimensions. Many of these are similar to how traditional HPC is evolving. Tabor Research believes these trends will be significant forces in determining the shape of next-generation HPC architectures.
Cray at SC08 – Celebrating Innovation
Visit us at booth #532 and see the latest technology from Cray, including the new Cray XT5 system with ECOphlex technology and the recently introduced Cray CX1 desk side supercomputer.
Visit IBM at SC08 - Experience the latest breakthroughs in High Performance Computing
As the world's leading provider of high performance computing solutions, IBM will showcase Exascale Stream Processing, Cloud Computing, Blue Brain, Interactive Ray Tracing along with many other exciting demos.
Harness the power of Sun to solve your most complex problems
Beat your competition by getting to market first, running more simulations, and solving complex problems with Sun HPC Systems. Sun HPC: Open, Simple, Reliable.
Last week, San-Francisco-based Complete Genomics came out of stealth mode to become the first provider of large-scale human genome sequencing services. HPCwire recently asked company representatives a few questions about their new offering.
Read More...
Intel has acquired the assets of NetEffect, an Austin-based company that makes iWARP-capable adapters. Intel will inherit NetEffect's product portfolio, which includes 1 and 10 GbE accelerated adapters, 10 GbE adapters for blade configurations as well as a 10 GbE ASIC.
Read More...
Woven Systems has added a new 10 Gigabit Ethernet top-of-rack switch to its product lineup. The TRX 200 is aimed at high performance datacenter environments requiring a scalable Ethernet fabric.
Read More...
Oct 15 | Linux Magazine | Today machines manage what we cannot. Are we dependent upon results or processes we do not understand? Read more...
Oct 15 | International Science Grid This Week | Exa-scale computing is probably years away. But GPUs and volunteer grids may provide a shortcut. Read more...
Oct 14 | Texas Advanced Computing Center | TACC has unveiled a new visualization laboratory capable of reproducing terascale data sets with exceptional clarity and resolution. Read more...
Oct 13 | Computerworld | Microsoft will have to overcome Windows' historical baggage if its new HPC Server 2008 offering is to be acceptable to users. Read more...
Oct 13 | Knoxville News Sentinel | Oak Ridge National Laboratory has petaflop computing in sight as it upgrades its 'Jaguar' supercomputer. Read more...
Sep 04 | | Disk drives are approximately 250 times denser today than a decade ago. This is good news for users who are creating, manipulating and storing more data than ever before. It gives them an opportunity to derive more value from their stored data and lowers the capital acquisition and operating expense associated with that data.
Sep 05 | | The excellent scalability features of Linux, in addition to robust security and performance makes it an excellent choice for server systems, especially in the high performance computing area.
BlueArc's Titan architecture represents an evolutionary step in file servers by creating a hardware-based file system that can scale bandwidth, IOPS, and overall data capacity well beyond conventional software-based devices. With its ability to virtualize a massive storage pool of up to four usable petabytes of tiered storage, Titan can scale with growing data requirements, offering a competitive advantage for businesses, researchers, or other enterprises seeking to better manage data growth while still ensuring optimal performance.
Get updates and insights on the High Productivity Computing industry delivered driectly to your inbox.