The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing
March 14, 2008
HPC Horizons Community Takes HPC to the Edge
This week saw the inaugural meeting of HPC Horizons, a new community of HPC users, vendors, and policymakers dedicated to collaborative discussion of forward-looking topics that push the boundaries of High Productivity Computing. The two-day conference had 125 attendees and featured speakers that represented both traditional and emerging HPC applications. Tabor Communications, the parent company of HPCwire and Tabor Research and the founders of HPC Horizons, billed the event as a prelude to ongoing online discussions among a wider member audience.
The presentations, panel discussions, and community action groups provided a compelling view of future trends in HPC that will be important across both new and established HPC application spaces. Tabor Research observed many common threads and provides the following analysis.
Data Ingest
Massive ingest of data as a limiting factor to HPC scalability was mentioned by several speakers, including a keynote address by Dr. J. Craig Venter, well known throughout the industry for his work in mapping the human genome. The Venter Institute is now investigating microbial life forms, which represent over half the biomass on earth and may hold the key to finding sustainable bio-energy sources or understanding the synthesis of life. Dr. Venter noted that a major challenge currently facing biologists is to gather as much genetic information from the microbial world as possible. A barrel of seawater can yield thousands of new microbial species to analyze and categorize.
Deborah Gracio of Pacific Northwest National Laboratories also discussed data-intensive and data-streaming applications. She noted that advances in high-throughput sensors can easily overwhelm data storage capabilities, giving the example of a proteomics mass spectrometer that was run at about 1 percent of capability because it would take over all of PNNL's storage within two days if run at full capacity. She noted that data filtering and analysis needs to be done close to the sensor, and the multithreaded architectures work well for problems with large irregular memory access.
This level of data influx -- massive amounts of data points for analysis coming from disperse points around the periphery of a system -- is mirrored in other types of emerging applications, such as surveillance, online gaming, logistics, virtual reality networks, or trading analysis. Many of these applications involve real-time or near real-time analysis requirements, and they involve a wide range of data or file types. To address the challenge, many users and vendors suggested the need to move computation closer to the data source.
Predictive Networks
In an opening keynote, Jaron Lanier of UC Berkeley discussed latency as applied to virtual reality systems. In a compelling analogy, he pointed out that the human brain has relatively poor latency in communication between different sections, and he posited that the reason the brain is such a fast computer is due to its predictive capabilities, with each section predicting the information it will receive from other sections ahead of data arrival and then adjust to any variances with the actual information as it comes in.
As the discussions moved toward other latency-sensitive applications, the development of predictive networks was a consistent theme. Several users suggested the need to compute ahead on likely dimensions in order to hide latencies and allow applications to run well at scale.
Distributed computation can also be used to reduce latency issues. For example, two separate views of the flight of a ball can be computed from the initial position and trajectory. Computed independently, these become two halves of a predictive system.
This concept also manifested in a few speeches that suggested the need for improvisational systems, which could be applied to either the computer systems or the human decision-making systems surrounding them. (Tom Lange of Procter & Gamble commented, "We don't sell soap; we sell clean.") By extension, systems that are predictive also need to sustain themselves when unexpected results occur.
Productivity
For a long time, the HPC community has discussed the need to measure productivity in ways that go beyond price/performance ratios. John Hurley of Boeing noted the difference between performance and productivity systems, with performance systems being typified by such factors as: application specificity to architecture, integrated systems, restricted access, cost no object, specialized technology, and tools-driven. In contrast, he described productivity systems as having data-driven architectures that run broader classes of applications with shared access, focusing on both cost and ROI, general-purpose products.
Lange also challenged the gathering with a number of controversial questions. For example, he asked the user community, "Are bigger problems worth it?" and "If software is not commercially affordable, are you willing to write your own?" He asked ISVs, "Is nature against us, and are problems just too hard to parallelize?" and "Are R&D expenditures balanced between application improvements and user interface improvements?" Finally, he asked software and hardware vendors, "Do you view national labs as potential partners or potential competitors?"
These observations raise several important issues concerning productivity, but perhaps none more important than these: How is productivity measured, and what does success look like?
Tabor Research is attempting to address these issues by offering users an interactive, online productivity analysis tool. With this tool, users will fill out profiles and questionnaires, including an allocation of the time and effort spent by different people within their organizations across various phases and tasks. The tool will then provide a peer comparison of how time is being spent and combine that data with each user's profile and what they believe to be important metrics of productivity. Each user then receives concrete recommendations that help focus incremental HPC investment. Tabor Research announced plans to introduce the tool to the Horizons group and the HPC community at large in April.
Conclusions
Several participants noted to Tabor Research that the HPC Horizons conference was unique in considering HPC as a business area as well as a technology area. One noted that it brought people together from wide variety of fields and helped identify the areas commonality between seeming disparate applications areas.
At the "horizon of HPC," new application spaces are emerging that will drive requirements in new dimensions. Many of these are similar to how traditional HPC is evolving. Tabor Research believes these trends will be significant forces in determining the shape of next-generation HPC architectures.
Sponsored Links
Interview: Appro CEO Shares HPC Vision
Appro CEO Daniel Kim provides a glimpse into Appro's vision and opportunities for its supercomputer and high-performance cluster solutions.
RSS Feeds
Feeds by Topic
- Applications
- Developer Tools
- Interconnects
- Middleware
- Networks
- Processors
- Storage
- Systems
- Visualization
Feeds by Industry
Feeds by Content Type
Feature Articles
Computed Tomography Software Taps Into NVIDIA GPUs
Minnesota-based North Star Imaging, a firm that specializes in industrial X-rays for nondestructive testing and analysis, is employing NVIDIA GPUs to accelerate 3D renderings in their CT (computed tomography) software. Julien Noel, the company's CT product manager, says the exceptional computational power afforded by CUDA and Tesla hardware is increasing customer productivity and transforming their workflow.
Read More...
The Next Big Thing in Humanities, Arts and Social Science Computing: 18thConnect
For the humanities scholar who may have only recently mastered library and archival finding aids beyond the archaic card catalog, the possibility of retrieving source materials at the flash of a keystroke (well maybe a few...) is very heady stuff.
Read More...
HPC Clouds -- Alto Cirrus or Cumulonimbus
The "cloud" model of exporting user workload and services to remote, distributed and virtual environments is emerging as a powerful computing paradigm. Yet, one domain that challenges this model in its characteristics and needs is high performance computing.
Read More...
Top Headlines
Dawning 6000 to Use Chinese-Made Loongson Processor
Nov 28 | People's Daily Online | Currently under development, the Dawning 6000 HPC system will be based on the Chinese-made "Loongson" microprocessor. Read more...
Can Supercomputers Help Save the Economy?
Nov 27 | Computerworld | The use of supercomputers to increase the industrial might of the U.S. has amounted to little more than an asterisk from a financial standpoint in both the federal budget and the economy as a whole. Read more...
IBM to Establish 'Collaboratory' in Dublin
Nov 26 | Science Business | IBM is getting ready to set up a supercomputing research “collaboratory” in Dublin, Ireland. Read more...
Texan Prof Sees Big Future for Graphene Storage
Nov 25 | The Register | A Rice University professor believes that his proposed graphene arrays could be many times denser and faster than existing storage tech, and they'd be more reliable too. Read more...
Super Micro Computer: A One-Man, or at Least One-Family, Powerhouse
Nov 24 | The New York Times | Server maker Super Micro Computer lives by two principles: give customers what they want, and do it as fast as humanly possible. Read more...
Multimedia
Video White Paper: Architecting a Better Network Storage Solution
BlueArc's Titan architecture represents an evolutionary step in file servers by creating a hardware-based file system that can scale bandwidth, IOPS, and overall data capacity well beyond conventional software-based devices. With its ability to virtualize a massive storage pool of up to four usable petabytes of tiered storage, Titan can scale with growing data requirements, offering a competitive advantage for businesses, researchers, or other enterprises seeking to better manage data growth while still ensuring optimal performance.
Newsletters
Stay informed! Subscribe to HPCwire email Newsletters.
Get updates and insights on the High Productivity Computing industry delivered driectly to your inbox.
HPC Job Bank
-
Sr. Systems Developer/Engineer
Ohio State University -
HPC System Administration Specialist
Argonne National Laboratory - Visit the HPCwire Job Bank
Featured Events
-
February 24-27, 2009
TradeTech Architecture 2009
London England
-
April 13-16, 2009
Symposium 2009: LEAN, CLEAN, and GREEN
New York , NY
USA