HPCwire: The Next Challenge in High Performance Computing

The Leading Source for Global News and Information Covering the Ecosystem of High Productivity Computing

Leading HPC
Solution Providers

HPCwire >> Off the Wire

February 08, 2008

The Next Challenge in High Performance Computing

-- Applications and Application Enablement

by Ashwini K. Nanda

Widening gap between platform affordability, and HPC software and applications

The availability of commodity high performance processors, fast and reliable commodity interconnection networks, modular system building blocks, and open system software has made high performance computer systems more affordable, easier to build, and leveled the playing field in HPC. Once an exclusive domain of powerful players such as national labs and large corporations, in recent times HPC has been making steady inroads into the computing radars of medium and small companies across the globe.

The need for HPC has pervaded all walks of life, from manufacturing to medicine, science and technology, and from defense to education, entertainment and national security. The emergence of commodity HPC platforms such as clusters of blades, clusters of desk top/rack mount servers, clusters of PlayStations and PC's makes the expectations from, and the interest in, the promise of HPC more intense. Take the financial institutions as an example. Until recently, a few tens or hundreds of servers working overnight to compute futures and options pricing for a large firm was considered acceptable. But with the expanding possibilities, the institutions now expect to run these analytics in real time to provide more value to their customers, asking for orders of magnitude higher compute capacity. In order to provide quicker diagnostics and better patient care, doctors wish to get rid of the hours/days of wait between an MRI/fMRI/PET scan and an off-line image analysis and provide real-time image analysis using significant amount of compute power. In the entertainment world, gamers are not satisfied with the improved graphics and life-like experience made available by today's advanced game consoles. Game providers wish to build massive compute infrastructures to simulate virtual worlds with millions of avatars to take the user experience to the next level. The list keeps growing.

However, the development of HPC applications has not kept pace with the perceived needs. Despite the increasing demand for more compute power to solve real life problems, the ever-widening gap in applications and software availability to harness the power of these platforms has been a serious bottleneck in the widespread adoption of HPC.

Factors impacting successful HPC application development

Successful HPC applications development rests on several factors including availability of parallel programming tools and libraries, sustained access to HPC platform facilities, and availability of highly skilled parallel programmers with domain expertise in the relevant scientific areas.

The developers of desk top applications and traditional server applications such as database and web based applications have plenty of mature development tools at their disposal (e.g. Eclipse). Large HPC developers (such as fortune 500 companies or National labs) use custom built in-house tools that are not available to the broader parallel application developers. Tools like TotalView scale to handle today's large clusters, however it needs to expand to support heterogeneous technologies (e.g. Cell, GPU and accelerator based solutions). Lack of an integrated, broad-based HPC development and debugging tool set is a significant barrier to entry for small and new HPC enterprises aiming to develop parallel applications.

Stringent export licensing requirements around scalable HPC systems in technology sourcing countries, have, to a large extent, made access to these platforms by scientific and technical professionals in new technology powerhouse economies such as India and China difficult. Commodity cluster technology is starting to break that barrier, but the licensing constraints remain a deterrent to wider HPC application development. For data-intensive and search-oriented HPC workloads, the cloud computing initiatives pioneered by Amazon, Google, Yahoo!, IBM and Microsoft hold the promise of convenient platform access across geographical boundaries using the utility computing model.

Since HPC and parallel programming had primarily been the playground of the privileged few until recently, organized efforts to train and develop broad-based parallel programming skills have been few and low-key at best, further isolating the demand for parallel applications from the domain experts in various fields.

The emergence of multiple hardware platforms, especially multicore and heterogenous processors from manufacturers such as Intel, IBM, AMD and NVIDIA brings additional challenges to parallel processing tools and library development. Despite pioneering work from IBM, Intel, RapidMind, several universities and the open source community, parallelizing and vectorizing compilers have a long way to go before becoming mainstream.

Difficulty in HPC application deployment

Besides development difficulties, deployment of HPC applications faces many other obstacles, including efficient scaling of parallel applications to larger clusters, and reliability of scalable HPC applications. Scaling needs support of efficient communication and networking software libraries, development and tuning of efficient parallel algorithms and libraries for various problem domains. Reliability becomes an increasing concern as system size grows, especially when using commodity servers. While recent efforts by SGI, EverGrid, Scali and others to address these issues are in the right direction, much remains to be done before HPC users feel comfortable with application level check pointing and error recovery.

A dream environment for many HPC users would be an integrated platform with the right "middleware," consisting of a set of development and debugging tools, parallel libraries, a lightweight HPC-friendly operating system, scaling and reliability support, and in many cases, inclusive of the application itself. While companies like Linux NetworX, IBM, SGI and HP have taken initiatives in this direction, a seamless integrated HPC solution remains a distant dream for many.

Bringing compute power, scientific computing skills, and HPC investment together to address some of these gaps

In an effort to close some of these gaps, recently the Computational Research Laboratories (CRL) Ltd, a start-up company funded by the Tata Group in Pune, India, developed the Eka supercomputer by integrating leading edge commodity cluster technology components from several vendors: quad-core Cloverton processors and high speed fibre optic cables from Intel, high performance/high density blade technology and Linux cluster software stack from HP, DDR Infiniband switches from Mellanox and Voltaire, and in-house developed dense datacenter technology. Eka was ranked the 4th fastest supercomputer in the world and the fastest supercomputer in Asia (over 120 TFLOPs sustained LINPACK performance), according to the Top500 list announced at SC07 in Reno, Nev.

During the announcement of Eka, Mr. Ratan Tata, Chairman of the Tata Group said: "High performance computing solutions like Eka will be playing an ever-expanding role in the scientific and technological space and will increasingly enhance the quality of life for our global society." Mr. S. Ramadorai, Chairman of CRL (also the CEO and MD of India's largest IT company, Tata Consultancy Services) said, "CRL's supercomputer, Eka, has put India at the forefront of high performance and supercomputing technology globally. This facility fills a gap in the availability of large HPC system in India, which has a latent hunger for huge compute power, and vast scientific and technical skill base to leverage this compute power in developing HPC applications."

Today roughly 70 scientists and engineers at CRL (about 25 percent Ph.D.s) are busy transforming its supercomputing facility into an HPC solutions and services hub, targeting to make it accessible to customers anywhere in the world, including India. CRL's customers can choose from a variety of services and solution offerings. The offerings range from providing compute time for a fee to consolidated applications research and development services, and from turn-key system integration services to domain-specific, complete end-to-end HPC solutions.

The CRL team was able to integrate Asia's fastest supercomputer from commodity technology in record time, bringing together several worldwide technology partners in a seamless fashion, and is now well equipped to replicate this success in customers' premises for small to medium and very large HPC systems. The team, consisting of a healthy mix of domain experts, computer architects, hardware and software engineers and parallel programming experts, along with CRL's application-specific web portals, aspires to offer a one-stop shop for parallel application consultancy, development, optimization and performance tuning for HPC users.

The team is developing parallel applications and libraries in several key domains, including financial modelling, life sciences, molecular simulation, aerodynamics, crash simulation, digital media animation and rendering. Third party application porting and scaling with the help of partners is in progress in areas such as seismic modelling, geophysical signal processing, automotive design, weather prediction, medical imaging, nanotechnology, personalized drug discovery, real time rendering, and virtual worlds, among others. Working with partners, the team is also developing efficient parallel communication software and error recovery ability for scalable HPC applications.

Armed with a combination of partnerships with global ISV's and end-users, along with in-house parallel application and computer systems expertise, CRL is strongly placed to meet the comprehensive solutions and services needs of HPC customers. In addition to these strengths, CRL also has established strategic partnership with group company Tata Consulting Services (TCS) in providing many of the solutions and services to our global customers.

Backed with substantial investment by India's most well known industrial house, the Tata Group, and the know-how to build world class supercomputers, CRL is zeroing in on leveraging abundantly available scientific and technical brain power in India to address many of the challenges outlined in this article toward making HPC meet its full potential in serving society's needs.

About the author

Dr. Ashwini Nanda joined the Computational Research Laboratories (CRL), Pune, India, in May 2007. Prior to joining CRL, Dr. Nanda worked at the IBM TJ Watson Research Center in NY. Most recently at IBM, he pioneered the Cell processor based real time, digital media and HPC systems technology including three generations of blade cluster products (QS20/QS21/QS22) and PetaFLOP system designs based on Cell blades (such as the LANL Roadrunner machine).

About CRL Limited

Computational Research Laboratories was incorporated as a fully-owned subsidiary of Tata Sons with a mandate to achieve global leadership in the area of high-performance computing systems. With an elite team of 70 researchers and scientists covering application software, system architecture, system software and hardware design, CRL not only builds world-class and globally competitive supercomputer systems but also delivers application-level scalability. For more information on CRL, please visit www.crlindia.com.

About Tata Consultancy Services Ltd (TCS)

Tata Consultancy Services is an IT services, business solutions and outsourcing organization that delivers real results to global businesses, ensuring a level of certainty no other firm can match. TCS offers a consulting-led, integrated portfolio of IT and IT-enabled services delivered through its unique Global Network Delivery Model, recognized as the benchmark of excellence in software development.

A part of the Tata Group, India's largest industrial conglomerate, TCS has over 100,000 of the world's best trained IT consultants in 47 countries. The Company generated consolidated revenues of US $4.3 billion for fiscal year ended 31 March, 2007 and is listed on the National Stock Exchange and Bombay Stock Exchange in India. For more information, visit us at www.tcs.com.

Article Tools

Share & Save Options

Discussion

There are 0 discussion items posted.

Join the Discussion

You must be a registered user in order to comment on HPCWire stories.

Members enjoy:

	Access to Special Content, including Podcasts, Webcasts, and Whitepapers and more!
	Ability to take an active role in the discussions that are shaping the HPC technology discipline!
	Special offers available only to HPCWire members!

Become a Registered User Today!

Registered Users Log in Here to Comment

Email Address:	Password (case sensitive)

Remember me	Forgot Password?

RSS Feeds

Feeds by Topic

Feeds by Industry

Feeds by Content Type

Feature Articles

Cray Unveils New Cooling Technology For the Petascale Era

Cray, known for its power and packaging prowess since 1976, when Seymour Cray bent the Cray-1 into a "C" shape, is unveiling a petascale-era cooling technology it says is more than 10 times as efficient as same-size water coils. Cray CTO Steve Scott discusses this innovation and the company that was green before green was cool.
Read More...

HPC Simulations Help Researchers Develop Artificial Sniffer Technology

From law enforcement to cancer detection, the uncanny ability of dogs to sniff out the earliest signs of danger and disease is receiving ever-increasing attention by researchers. Despite tremendous advances in law enforcement technology, there is still no man-made tool that can detect the presence of explosives quite like a canine's sophisticated sniffing system.
Read More...

Will High-Speed Copper Interconnects Dim Optical's Prospects?

The prospects for optical cable interconnects never looked so good. With single and double data rate InfiniBand firmly entrenched in HPC clusters and quad data rate deployments just around the corner, optical cable makers are looking to displace bulky copper cabling in the next generation of high performance systems.
Read More...

Read more HPCwire features...

Top Headlines

Heard at Hot Interconnects: Multicore SoCs May Require Optical Interconnect

Aug 27 | EDN | The lead-off session at the Hot Interconnects conference at Stanford examined the growing challenge of chip interconnect technology as today's multicore processors evolve into chips with tens or hundreds of cores. Read more...

Data Centers: Focusing on Sustainability

Aug 27 | Dr. Dobb's Portal | Dallas Thornton, director of Cyberinfrastructure Services for the San Diego Supercomputer Center, offers his thoughts on how energy concerns and other datacenter issues are affecting the industry. Read more...

China Aims for Petaflop Computer in 2010

Aug 27 | InfoWorld | China still lags behind its international rivals in chip development but is stepping up investment in its homegrown Godson microprocessor. Read more...

Really Big Things

Aug 27 | Linux Magazine | How does one manage really big clusters? Perhaps nature can give us a clue. Read more...

Fujitsu Readies Eight-Core Sparc64 Chip

Aug 27 | Computerworld | Fujitsu is developing an eight-core version of its Sparc64 processor, which should give a performance boost to the Sparc Enterprise Servers that Fujitsu jointly develops with Sun Microsystems. Read more...

Read more headlines...

Featured Whitepapers

Foundry High Performance Computing Application Note

Aug 01 | | The intent of this document is to provide an overview of Foundry Networks products that support high end HPCC configurations.

Improving Performance and Manageability for Seismic Processing and Imaging Applications with Parallel Storage

Jun 05 | | As pressure increases on the upstream seismic processing community to deliver ever-higher levels of productivity and efficiency, a new generation of storage solutions will be required that allow the maximum utilisation of high-performance computing (HPC) Linux cluster resources, together with the minimum of management overhead.

View the White Paper Library

Multimedia

Video White Paper: Architecting a Better Network Storage Solution

BlueArc's Titan architecture represents an evolutionary step in file servers by creating a hardware-based file system that can scale bandwidth, IOPS, and overall data capacity well beyond conventional software-based devices. With its ability to virtualize a massive storage pool of up to four usable petabytes of tiered storage, Titan can scale with growing data requirements, offering a competitive advantage for businesses, researchers, or other enterprises seeking to better manage data growth while still ensuring optimal performance.

Podcast: Interview with Ben Bennett of ClearSpeed Technology

Today, HPC organizations are requiring substantially more floating point performance to solve real-world problems. In this podcast, Ben Bennett, ClearSpeed General Manager, discusses how acceleration technology can improve the overall performance of standard x86-based systems...