IDC OPINION

Hewlett Packard Enterprise (HPE) emerged from 2015 as the clear revenue leader in the expanding worldwide market for high-performance computing (HPC) servers, with a 35.9% share of this $11.4 billion market and an even larger 41.1% share of the $3.3 billion supercomputers segment for HPC server systems selling for more than $500,000 each, including the largest share (31%) of the systems on the November 2015 Top500 list. In both cases, HPE’s share was more than double the share of the company’s nearest rival. HPE captured the leadership position with a strategy combining a long-term corporate commitment to HPC, adequate scale to address all segments of the market, a strong embrace of open standards, expanded HPC hardware and software ecosystem innovation (thanks in no small part to HP Labs), and strong and deepening expertise in HPC-reliant domains. Now, HPE is upping the ante with its comprehensive portfolio of HPE Apollo purpose-built platforms and related solutions aimed squarely at the rapid convergence of Big Compute, Big Data, and the Internet of Things (IoT) — including industry-specific requirements. Initial turnkey solutions are aimed at the economically important financial services, oil and gas (energy), healthcare, life science, and manufacturing sectors. This white paper examines HPE’s rise to HPC market leadership against the background of escalating user requirements that present new opportunities and challenges for vendors.

HPC/HPDA MARKET DYNAMICS: RAPID GROWTH AND CHANGE

The global market for HPC server systems has been one of the fastest-growing IT markets. HPC server revenue more than doubled from $4.8 billion in 2001 to $11.4 billion in 2015 and is en route to an IDC-predicted value of $15.1 billion in 2020 (a CAGR of 5.9% versus 3.6% for the worldwide server market as a whole). When software, storage, and service are added to the mix, the 2020 forecast for the HPC market effectively doubles.

During the past 15 years, clusters and other server systems based on x86 architecture processors have been primarily responsible for propelling rapid growth of the global HPC server market. In the five-year period from 2009 to 2014, the number of x86 processors sold annually into the HPC market grew 78.7%, from 1.7 million to 3.0 million. In 2014, x86-based systems accounted for 93.1% of worldwide HPC server revenue. The majority of these x86 processors have come from Intel. Further:

• Historically, HPC market growth has been driven by expanding use at established sites, targeting higher performance and density. This has been heavily augmented by successive waves of new users, each presenting vendors with new requirements and challenges.
• The new edition of “A Strategy for American Innovation,” made public by President Obama on October 21, 2015, named HPC as one of the top investment priorities for growing the

U.S. economy. A key role for HPC cited in “A Strategy for American Innovation” is to address “the rise of extremely large data sets and attendant computational challenges.”

Big Compute

HPC began in the 1960s as a niche market for government- and university-based researchers, primarily for compute-intensive floating point modeling and simulation (M&S) of physical and quasi-physical phenomena. But in less than a decade, HPC M&S began penetrating tier 1 commercial firms as a game changer for accelerating product development and competitiveness. In a pioneering IDC study for the Washington, D.C.–based Council on Competitiveness, 97% of companies that had adopted HPC said they could no longer compete or survive without it.

The arrival of standards-based, commercial-grade clusters in 2001–2002 made HPC M&S affordable for many small and medium-sized enterprises (SMEs) and start-ups. In 2015, 113,000 HPC systems were sold at prices below $100,000.

The spread of HPC into private sector firms of all sizes has brought with it a need for vendors to begin developing industry-specific, and even workload-specific, solutions. Today, the ability to create

purpose-built solutions for economically important HPC domains and use cases is becoming crucial for large-scale vendors that want to address all segments of the HPC market.

Big Data

Historically, the HPC market has included Big Data workloads of two main types. First, some M&S jobs have been data intensive (i.e., they have involved much more data processing than computation). Second, a few HPC domains — notably the intelligence community and the financial services industry (FSI) — have long relied heavily on integer-based analytics (as opposed to floating point–based M&S). The back offices (“quants”) of large investment banks began using HPC in the late 1980s, especially for pricing exotic instruments, portfolio optimization, and firmwide risk management (high-frequency trading [HFT] was recently added to this mix). Today, a growing number of HPC sites have both M&S and analytics workloads.

Much newer is the trend for large commercial firms to move up to HPC to tackle mission-critical analytics challenges that are too complex and time critical for the firms’ enterprise server technology to handle alone. In these cases, HPC servers are typically inserted directly into the live data pipeline. The drivers here are competitive forces and the opportunity to save money (PayPal has saved over $700 million by migrating to HPC).

In 2012, IDC launched the High-Performance Data Analysis (HPDA) service, which tracks both historical data-intensive computing and newer advanced analytics in the commercial sector. Economically important use cases include fraud and anomaly detection, business intelligence, affinity marketing, and personalized (“precision”) medicine. Arguably, no IT market has seen a more powerful Big Data explosion than the HPC/HPDA market. An important consequence of this explosion is the need for users to adopt advanced data analytics methods (Hadoop, Spark, etc.). Even more important is the need to elevate storage capacities and capabilities, such as by adding object storage and software-defined storage, to enable effective scaling.

Internet of Things

HPC will almost certainly perform two key functions in the nascent Internet of Things market:

• HPC systems will provide dense nodes needed for local, exceedingly data-intensive, real-time use cases such as managing urban traffic with a mix of human-driven, driverless, and semi-driverless vehicles. Urban traffic management is already an important HPC application around the world, and major automakers are using HPC heavily to develop tomorrow’s driverless vehicles, along with the real-time IoT infrastructure that will be needed to support their use.

• HPC will be needed for the functional and wellness management of large portions of the global IoT network, such as national-level IoT networks (China’s national IoT plan already calls for HPC management). HPC will also be needed for IoT data tracking and aggregation, especially in network-edge environments.

TECHNICAL AND OTHER CHALLENGES

Today, HPC system developers and users face an array of interrelated challenges, including:

• Developing software capable of efficiently exploiting HPC hardware systems
• Burgeoning system sizes and complexity
• Heterogeneity (CPUs, accelerators)
• New environments (e.g., public clouds)
• Reliability/resiliency requirements
• A mix of compute- and data-intensive workloads
• Energy efficiency
• An influx of small and medium-sized businesses (SMBs) and other commercial users that want “ease of everything”
• The movement from synchronous applications to asynchronous workflows
• A serious shortage of qualified job applicants, especially programmers, systems administrators, and people able to bridge the gap between HPC and domain science/engineering/analytics

Together, these challenges represent a daunting agenda for future system development.

HPE‘S STRATEGY FOR MARKET LEADERSHIP

As noted previously, HPE exited 2015 as the clear revenue leader in the expanding worldwide market for all HPC servers, with a 35.9% share of this $11.4 billion market. Also in 2015, HPE captured an even larger share of 41.1% of the $3.3 billion supercomputers segment for HPC server systems selling for more than $500,000 each.

HPE has been ramping up efforts to tackle HPC system and storage challenges head-on. For example, HPE is addressing the Big Compute–Big Data convergence with an expanding lineup of server and storage offerings for the related HPC, deep/machine learning, and nascent Internet of Things markets. The offerings include tailored solutions for financial services, oil and gas, life sciences, and manufacturing.

Concerted R&D Initiatives for HPC

HPE leverages the company’s overall R&D on behalf of the HPC market and also invests substantial R&D funding specifically for HPC. Metagoals are to democratize HPC and make it easier to access, use, and maintain.

Domain Expertise and Centers of Excellence

• A prerequisite for these purpose-built solutions is HPE’s deep, expanding domain expertise. The company has been investing in industry consortiums and collaborating with leading academic institutions and other vendors to develop benchmarks and best practices at the intersection of HPC and Big Data.

• HPE and Intel jointly created two HPC centers of excellence (CoEs), one in Grenoble, France, and the other in Houston, Texas. The companies have staffed the centers with deep engineering expertise, vertical industry knowledge, and expertise in performance optimization and code modernization. The centers focus on benchmarking and proof-of-concept work.
• HPE and Intel created another CoE at Teratec in Bruyères-le-Châtel, France. Teratec is “a European pole of competence in high-performance digital simulation” and brings together over 80 companies, laboratories and research centers, universities, and engineering schools. The CoE’s roles are to showcase new technologies, conduct proof of concepts and performance benchmarks, and develop educational white papers and reference architectures for HPC and Big Data analytics solutions.

• HPE is actively engaged in the National Strategic Computing Initiative (NSCI) and is helping provide exascale computing capabilities to the United States to shape the future of the country’s technology direction. To date, HPE has participated in NSCI activities including a congressional panel sponsored by ITIF, OSTP workshops, and meetings with NSCI stakeholders.

Expanding Product Portfolio for Big Compute, Big Data, and IoT

HPE’s new HPC solutions feature innovations in systems design, workload optimization, density optimization, open source software, and visualization (including software-defined visualization). Together, the innovations aim to accelerate time to value in machine learning/deep learning, energy exploration, mechanical design, financial trading and regulatory compliance, and other major HPC domains.

HPE’s HPC solutions are designed to be customer centric and technology agnostic. The limited scope of this white paper does not permit fuller descriptions, but a mere list of the product portfolio should give some feel of the breadth and depth of HPE’s product set, which consists of hardware and software platforms, horizontal solutions, and more. The product portfolio includes the following offerings:

• The HPE Apollo 6500 deep learning platform, which has 4U chassis with two server nodes that can hold up to 8 NVIDIA GPU cards or Intel Xeon Phi cards per node.
• HPE Apollo 4520 System, a high-density, high-scalability, and high-resiliency storage server, which can be outfitted with Intel Enterprise Edition for Lustre* software or open source Lustre as well as other parallel file systems.
• HPE Trade and Match Server, optimized for superior HFT performance
• HPE Risk Compliant Archive for regulatory compliance management in financial services
• HPE Moonshot Trader Workstation, designed to maximize trader experience and productivity while lowering TCO
• Apollo 2000 System, which is highly scalable for HPC workloads, such as ANSYS for CAE, or traditional IT workloads
• Apollo 6000 System, designed for HPC workloads at rack scale
• The new HPE Edgeline IoT Systems, resulting from an HPE-Intel partnership to help deliver proven open solutions for the IoT market (The HPE Edgeline IoT Systems 10 and 20, as well as the HPE Edgeline 1000 and 4000 systems, sit at the network edge and are designed to enable customers to securely aggregate and analyze data in real time and control devices and things.)
• HPE Apollo 8000, a warm water–cooled supercomputer that delivers over 250 peak teraflops per square foot while targeting high efficiency and minimal energy consumption
• The HPE Big Data portfolio, which includes the purpose-built HPE Apollo 4510 System that is tailored for object storage at petabyte scale; HPE Apollo 4530 and 4200 Systems, aimed at Hadoop analytics and other Big Data analytics; and HPE Integrity Superdome X System, designed for workloads benefiting from in-memory computing and real-time analytics

In addition to these purpose-built platforms, HPE has a comprehensive portfolio of general-purpose compute platforms in the company’s HPE ProLiant racks and towers and HPE BladeSystem series. HPE’s comprehensive software portfolio includes HPE’s Core HPC Software Stack, Insight CMU, Cluster Test, and HPE OneView.

HPE’s overall market leadership is fueled by close collaboration and deep relationships with the company’s technology partners. The HPE portfolio builds on industry-leading technologies from partners including (but not limited to) AMD, Intel, Mellanox, NVIDIA, Seagate, ISVs, and the open source community.

OPPORTUNITIES AND CHALLENGES

Opportunities

• Exploit IDC’s forecast market growth, HPE’s market leader position, and the escalating convergence of Big Compute, Big Data, and IoT. Being the OEM revenue leader in a robustly growing market presents opportunities for further growth, especially as HPC competencies increasingly drive the convergence of advanced computation and advanced analytics and become indispensable for more large commercial firms and the nascent IoT market.
• Maintain HPE’s exceptional customer loyalty. IDC studies have shown that with regard to customer loyalty, most HPC vendors’ ratings are bunched closely together but HPE rises above the crowd to form a class of its own. HPE has an opportunity to bank on this valuable asset as the company works to expand its HPC market leadership.
• Promote HPE’s R&D innovation for HPC more assertively. In IDC’s opinion, HPE has been overly modest about its R&D investments and innovations benefiting the HPC community. The company has an opportunity to tell this story more assertively in order to receive proper recognition for these contributions.

Challenges

• Ensure that HPE’s senior management publicly and consistently expresses the company’s long- term commitment to the HPC market. HPE’s senior management has taken important steps to demonstrate this commitment, such as creating a dedicated business unit focused on HPC, Big Data, and IoT with a strong leadership team; forging an HPC alliance with Intel; engaging deeply in the NSCI; and funding several new, long-term R&D initiatives and go-to-market programs.
• Extend HPE’s HPC customer base beyond the current large, loyal contingent. Note that HPE has picked up a lot of former IBM customers during the IBM-Lenovo transition.
• Grasp and exploit the dynamics of the convergence of Big Compute, Big Data, and IoT.

CONCLUSION

HPE is now the clear revenue leader in the fast-growing worldwide HPC server market, valued at

$11.4 billion in 2015, and in the $3.3 billion supercomputers segment of the market. The company’s winning strategy started with a long-term commitment to the HPC market, both in its own right and as an effective lever for opening up major new opportunities for Big Compute, Big Data, and IoT, all of which face crucial challenges that need to be addressed by the global HPC community. HPE’s product portfolio is comprehensive enough to cover the vast majority of HPC user/buyer requirements — and the newly announced products move that collection substantially forward. HPE has also been growing its domain-specific solutions for economically important vertical segments, along with the domain expertise that enables peer-to-peer sales and support.

Finally, HPE has been ramping up R&D for its HPC strategy, from new technology initiatives to centers of excellence where promising new technologies and ideas can be evaluated and benchmarked.

IDC predicts that by 2020, the global market for HPC servers will exceed $15 billion and the whole

HPC market — servers, storage, software, and service — will be worth about double that amount.

We believe that HPE’s existing leadership status, combined with the company’s long-term commitment and strategy for this market, positions HPE well to exploit our forecast growth for the interrelated

HPC Big Compute, Big Data, and IoT markets. For HPE, as for any major vendor, it will be challenging to anticipate and respond effectively to the increasingly complex dynamics of the HPC market, but the company so far has shown an impressive ability to do this.

About IDC

International Data Corporation (IDC) is the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications and consumer technology markets. IDC helps IT professionals, business executives, and the investment community make fact- based decisions on technology purchases and business strategy. More than 1,100 IDC analysts provide global, regional, and local expertise on technology and industry opportunities and trends in over 110 countries worldwide. For 50 years, IDC has provided strategic insights to help our clients achieve their key business objectives. IDC is a subsidiary of IDG, the world’s leading technology media, research, and events company.

Global Headquarters
5 Speen Street
Framingham, MA 01701
USA
508.872.8200
Twitter: @IDC
idc-community.com www.idc.com

Copyright Notice

External Publication of IDC Information and Data — Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason.

Copyright 2016 IDC. Reproduction without written permission is completely forbidden.