HPE today launched a new family of storage solutions bundled with IBM’s Spectrum Scale Erasure Code Edition parallel file system (description below) and featuring license changes so it’s not necessary to license storage capacity separately by terabyte or by storage drive (SSD or HDD). Broadly, the new line spearheads HPE’s push into enterprise markets where NAS solutions have dominated and where the Lustre parallel file system has faltered. HPE’s E1000 Lustre-based solution remains its top-end product targeting supercomputing.

Writing today in an HPE blog, perhaps a little tongue-in-cheek, Uli Plechschmidt, worldwide product marketing, HPC Storage, said, “An HPE storage system with embedded IBM Spectrum Scale. Really? Yes. Really! With the new HPE Parallel File System Storage product, HPE is now introducing unique parallel storage for customers that use clusters of HPE Apollo Systems or HPE ProLiant DL rack servers with HPE InfiniBand HDR/Ethernet 200Gb adapters for modeling and simulation, artificial intelligence (AI), machine learning (ML), deep learning (DL), and high- performance analytics workloads.”

Attractive market opportunities often produce unexpected alliances. Analysts suggest dramatic data growth and surging AI compute demand are creating an opportunity for scalable, high-performance parallel file systems in the enterprise for HPC/AI workloads. HPE is a dominant player in HPC/AI server sales. IBM’s Spectrum Scale (formerly GPFS) has long had many attractive enterprise features – snapshots, compression, data replication, data encryption, policy-based data movement, among others.

Seen through this lens the HPE-IBM storage pairing seems opportune. Two other factors are contributing. There’s a general consensus that Lustre, while extremely powerful, is more challenging to deploy and support and lacking in enterprise features. It should also be noted that HPC storage provider DDN bought Lustre developer and support company Whamcloud from Intel in 2018 and has since dropped its IBM Spectrum Scale/GPFS-based products from its offerings. DDN is actively working to make Lustre more enterprise friendly.

“IBM has remained strong in storage and software for HPC, but these have been held back by weakness in the IBM Power server business. This agreement with HPE pairs IBM’s well-liked Spectrum Scale capabilities with the highest-volume server portfolio,” said Addison Snell, CEO, Intersect360 Research. “This will be a shot in the arm for GPFS, with another route to commercial markets. Previously HPE resold a lot of other storage partners’ solutions, such as DDN, Qumulo, Scality, or Weka, so this may be a concern for them. A lot will come down to how sales reps engage with it in the field.”

The stakes are growing in HPC storage, which market analyst Hyperion Research forecasts will grow with a 57% higher CAGR than HPC compute.

Mark Nossokoff, lead storage analyst, Hyperion told HPCwire, “As enterprise datacenters are embracing AI to address their business analytics and business intelligence applications, they are adopting HPC infrastructure to be able to handle the data-intensive workloads. In addition to IBM Spectrum Scale ECE matching Lustre with the performance characteristics to satisfy the HPC-enabled AI demands, it also currently provides more of the required enterprise features (e.g., availability, non-disruptive hardware & software upgrades, data replication, end-to-end encryption) than Lustre generally provides today.”

“Scale is also a factor. Lustre-based systems tend to be architected and configured for the high-end of the HPC storage market. As such entry-level solutions provide capacity beyond the amount enterprise datacenters may require and may be priced higher than budgets allow. Providing an HPC storage solution with smaller entry-level building blocks at correspondingly lower entry price points allows HPE’s Spectrum Scale ECE-based products to be considered in markets where traditional Lustre-based solutions would not,” said Nossokoff.

Plechschmidt took direct aim at NAS in his blog, “Unlike NFS-based Network Attached Storage (NAS) or Scale-out NAS, there are (almost) no limitations regarding the scale of storage performance or storage capacity in the same file system. NFS-based storage is great for classic enterprise file serving (e.g. home folders of employees on a shared file server). But when it comes to feeding modern CPU/GPU compute nodes with data at sufficient speeds to ensure a high utilization of this expensive resource—then NFS no longer stands for Network File System but instead, it’s Not For Speed.”

Citing an IDC forecast that the price per terabyte of SSDs in 2024 will still be seven times higher than the price for terabyte of HDDs, HPE reported it believed hybrid file systems will become the norm for the foreseeable future. “Both our parallel storage systems enable customers to have two different media pools in the same file system: NVMe Flash pool(s) to drive the required performance (throughput in gigabyte per second/IOPs); and HDD pool(s) to provide most of the cost-effective storage capacity,” wrote Plechschmidt.

General availability for the new systems will start in May according to HPE. Pricing wasn’t disclosed. HPE reports its E1000 ClusterStor system, generally available since August 2020 and tapped for two of the forthcoming U.S. exascale systems, has already achieved the two exabyte milestone.

Link to blog: https://community.hpe.com/t5/Advantage-EX/New-HPE-storage-system-with-embedded-IBM-Spectrum-Scale-4-ways/ba-p/7126022#

Description of IBM Spectrum Scale Erasure Code Edition

IBM Spectrum Scale Erasure Code Edition provides IBM Spectrum Scale RAID as software, allowing customers to create IBM Spectrum Scale clusters that use scale-out storage on any hardware that meets the minimum hardware requirements.

All of the benefits of IBM Spectrum Scale and IBM Spectrum Scale RAID can be realized using your own commodity hardware.

For example, IBM Spectrum Scale Erasure Code Edition provides:

• Reed-Solomon highly fault tolerant declustered Erasure Coding, protecting against individual drive failures as well as node failures.
• Disk Hospital to identify issues before they become disasters.
• End-to-end checksum to identify and correct errors introduced by network and/or media

IBM Spectrum Scale Erasure Code Edition uses the same software and most of the same concepts that are used in the Elastic Storage Server (ESS). Elastic Storage Server (ESS) is a solution consisting of two I/O (storage) servers and between one and several JBOD disk enclosures, with each storage device (pdisk) attached to both servers. In Elastic Storage Server (ESS), there are two recovery groups (RGs). Each RG takes half of each enclosure among all enclosures. Under normal conditions, each I/O server supports one of the two RGs. If either I/O server fails, the remaining I/O server takes over and supports both RGs.