Linux Widens HPC Goalposts

By Meike Chabowski

May 4, 2015

It is well known that the term “high performance computing” (HPC) originally describes the use of parallel processing for running advanced application programs efficiently, reliably and quickly. The term applies especially to systems that function above a teraflop or 10^12 floating-point operations per second, and is also often used as a synonym for supercomputing. Technically a supercomputer is a system that performs at or near the currently highest operational rate for computers. To increase systems performance, over time the industry has moved from uni-processor to SMP to distributed-memory clusters, and finally to multicore and manycore chips.

However, for a growing number of users and vendors, HPC today refers not to cores, cycles, or FLOPS but to discovery, efficiency, or time to market. Some years ago, IDC came up with the interpretation of HPC to High Productivity Computing, highlighting the idea that HPC provides a more effective and scalable productivity to customers, and this term fits really well for most commercial customers.

But how did we get here?

A Faster Pace of Innovation

A few years back, UNIX variants such as AIX, HP-UX, Tru64 UNIX, Solaris, etc., ruled – and building supercomputers out of clustering independent, commodity-class machines was a controversial idea as recently as 15 years ago. The historic cost of HPC limited their use to market segments that could afford them. But the evolution of both lower cost hardware and Linux has dramatically reduced the cost of these systems, while computing power has increased on a scale of one thousand times in just a few years. This combination has allowed many companies to use the power of supercomputers in the form of an HPC Linux cluster on commodity hardware.

Time is money

Compared to before, more business are now using computers not only to manage their businesses but also as part of their delivery (animations, stock exchange, analysts, finance, weather forecasting) or to support the creation of products (oil research, car crash tests, drug research). Being able to get this done faster speeds up time to market and making it more accurate gives more buffer. Thus it’s a competitive advantage to run it as fast as possible – which can be done on an HPC system. To give an example on upgrading a cluster, a team recently built a new HPC system for a healthcare client. The new cluster was 10x faster than the old one, half the price, and one quarter the size.

Big Data or HPC?

Another important category is “ultra-scale business computing.” Commercial companies with data-intensive tasks are adopting HPC. For example, just take a look at companies like Google, Amazon, Facebook or eBay; most of these companies don’t have traditional HPC workloads, but at the end of the day these companies use HPC technologies to deal with all the data processing for their “big data” and to run them at extreme scale.

Building Bridges

There is also a growing number of mid-market companies adopting HPC due to changing business needs and the availability of economical solutions. Quite a few organizations in the supercomputing area start building bridges to the SME industry by offering these companies access not only to their supercomputers but also to their own expertise in high performance computing, and co-operations between industry and government/academia are continuously growing – two examples are Pittsburgh Supercomputing Center, and the Irish Centre of High-End Computing.

Linux is your friend

Enterprises that are long-time UNIX users for their HPC business workloads need to understand that they fully can rely on Linux, that Linux did reach UNIX parity and in many segments even surpassed UNIX with regard to availability, scalability and performance.

Windows isn’t the only option

Businesses that are used to using Windows need to “have the heart” to check out alternatives – and dual-boot HPC systems might be a first step. As Linux and Windows seem to become the two dominant platforms of the future in the enterprise, there will be an increasing need for these operating systems and the tools that manage them to work well together. Systems that lack well-developed interoperability capabilities can cause inefficiencies throughout the enterprise.

For example, limited interoperability between Linux and Windows environments, in both physical and virtual instances, can lead to server sprawl. It can also lead to redundant management tools and inefficient use of IT staff. This translates as well for HPC; it seems to be logical that the two major platforms used in the HPC market will be Linux (primarily) and Windows – and they also need to interoperate well in this area.

Thanks to its speedy adoption of technical innovations and improvements – or even better, as Linux very often is “spearheading” technical innovations, Linux will further play a significant role in the new HPC market dynamics, where HPC turns more and more into “High Productivity Computing.” For a non-HPC environment to move to HPC clusters, the speed improvement is somewhere in the range of the node numbers. So moving from a single workstation to a 200 node cluster (about 5 racks) gives you around 200x the performance than before.

As technology continues to evolve, supercomputing will continue to become an essential technology across more industries and within High Productivity Computing. In fact, the U.S. Department of Energy is currently working on developing the ultimate supercomputer to provide exascale (1,000 petaflops per second of sustained performance) for industries that may adopt HPC in coming decade. As open source and Linux continue their evolution, High Productivity Computing will be found in most data-driven industries as the key to success.

About the Author

Meike Chabowski is product marketing manager for Enterprise Linux Servers at SUSE.

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from its predecessors, including the red-hot H100 and A100 GPUs. Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. While Nvidia may not spring to mind when thinking of the quant Read more…

2024 Winter Classic: Meet the HPE Mentors

March 18, 2024

The latest installment of the 2024 Winter Classic Studio Update Show features our interview with the HPE mentor team who introduced our student teams to the joys (and potential sorrows) of the HPL (LINPACK) and accompany Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the field was normalized for boys in 1969 when the Apollo 11 missi Read more…

Apple Buys DarwinAI Deepening its AI Push According to Report

March 14, 2024

Apple has purchased Canadian AI startup DarwinAI according to a Bloomberg report today. Apparently the deal was done early this year but still hasn’t been publicly announced according to the report. Apple is preparing Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimization algorithms to iteratively refine their parameters until Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the fi Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimizat Read more…

PASQAL Issues Roadmap to 10,000 Qubits in 2026 and Fault Tolerance in 2028

March 13, 2024

Paris-based PASQAL, a developer of neutral atom-based quantum computers, yesterday issued a roadmap for delivering systems with 10,000 physical qubits in 2026 a Read more…

India Is an AI Powerhouse Waiting to Happen, but Challenges Await

March 12, 2024

The Indian government is pushing full speed ahead to make the country an attractive technology base, especially in the hot fields of AI and semiconductors, but Read more…

Charles Tahan Exits National Quantum Coordination Office

March 12, 2024

(March 1, 2024) My first official day at the White House Office of Science and Technology Policy (OSTP) was June 15, 2020, during the depths of the COVID-19 loc Read more…

AI Bias In the Spotlight On International Women’s Day

March 11, 2024

What impact does AI bias have on women and girls? What can people do to increase female participation in the AI field? These are some of the questions the tech Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Analyst Panel Says Take the Quantum Computing Plunge Now…

November 27, 2023

Should you start exploring quantum computing? Yes, said a panel of analysts convened at Tabor Communications HPC and AI on Wall Street conference earlier this y Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Leading Solution Providers

Contributors

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Training of 1-Trillion Parameter Scientific AI Begins

November 13, 2023

A US national lab has started training a massive AI brain that could ultimately become the must-have computing resource for scientific researchers. Argonne N Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire