Last November, the high-performance computing world gathered in Denver for Supercomputing ‘19, where HPC enthusiasts were dazzled by innovations with bleeding-edge technology pushing performance and capacity metrics to ever greater heights. In addition to being the premier site for the unveiling of new technologies, SC19 was a great forum to view trends, as the volume and clout of the attendees provided a significant sample size to track the industry as a whole. These trends move quickly – sometimes faster than the required technological innovation. As such, SC19 was the perfect venue to watch for technologies and trends coming together. In Denver, we saw a great example of this culmination in IBM Spectrum Scale Erasure Code Edition (ECE).

Trends in storage: flashiness without the frills

While the fundamental paradigms of high-performance storage remain constant, external market forces are making previously pricey storage architectures viable. There will always be a place for spinning disk, but the precipitous decline in price for flash storage (particularly NVMe) has prompted new interest in higher-performance storage componentry. The maxim “flash is the new disk, disk is the new tape” is at least a decade old, and many in the industry are somewhat skeptical of the claim. Nevertheless, falling prices on flash leads to wider availability, and wider availability of flash means some re-examination of what exactly is “hot” or “cold.”

Another shift playing out is the packaging of the storage, i.e. the enclosures in which drives are housed. While storage enclosures and appliances remain common, the storage-rich server is catching on. Chief among the numerous benefits are the granularity (build up your storage infrastructure in smaller increments than those you get from an appliance), commodification (it’s all just x86 – nothing messy), and cost. Certainly, moving from a traditional SAN or NAS to internal storage isn’t the right play for everyone. Implementation is key, as storage pools are limited to the quantities available in each individual node. Additionally, purpose-built storage controllers and appliances provide robust, high-availability and survivability features, and storage-rich servers leveraging commodity hardware may be somewhat lacking in comparison.

IBM Spectrum Scale ECE: a new twist on a classic

For an article about new developments in HPC, a focus on IBM Spectrum Scale, formerly known as GPFS (and a mainstay of HPC storage for more than two decades!), may seem unusual. The fundamental differentiator that Spectrum Scale offers, including  Spectrum Scale RAID (formerly GPFS Native RAID, or GNR ,has been around for a while. Spectrum Scale RAID is a patented form of de-clustered RAID; that is, data is striped across disks in a uniform distribution, so that if a disk fails, the impact is mitigated across the disk array, and the severity of that disk rebuild (in terms of read and write cost) is lessened.

Spectrum Scale RAID is a powerful feature, and ECE broadens the applicability of the patented approach. Traditionally, Spectrum Scale RAID was used within the context of a disk array; de-clustered disks would be contained within a single external storage device. However, ECE removes this requirement. RAID groups implemented by ECE may now be shared across servers connected via a high-speed fabric. This network RAID shifts the paradigm: storage provisioned by Spectrum Scale truly becomes scale-out.

ECE also mitigates the important drawbacks to commodity storage servers. With the high availability and fast rebuild times from Spectrum Scale RAID, the trade-off in reliability that would come from choosing commodity servers over traditional storage appliances becomes irrelevant. Spectrum Scale also offers storage pooling, so the storage limitations of an individual node need not constrain the overall storage configuration.

Spectrum Scale and Lenovo

Lenovo works alongside IBM on solutions and offerings around IBM Spectrum Scale. Lenovo’s award-winning Distributed Storage Solution for IBM Spectrum Scale (DSS-G) is our strategic high-performance storage offering. The two companies developed an integrated solution around IBM Spectrum Scale, with a team of dedicated hardware, software and firmware architects and engineers optimizing network fabric settings, Spectrum Scale parameters, and many other variables.

Juxtaposing the feature set of ECE with the industry direction in HPC storage hardware, the synergy is clear. The past trade-offs or compromises required by server-based storage have been alleviated.  With ECE, scale-out storage on commodity hardware has never been faster, more reliable, or more affordable.