As do all of us in the storage industry, I love reading what Chris Mellor writes. Last week he described a “street fight” between NetApp and Pure Storage, with each trying to articulate that the other is a legacy storage vendor.

I even retweeted his story with our own quick summary:

I then decided to spend some time expanding upon the message from my tweet with more details, specifically, how to spot storage vendors selling legacy architectures. I feel strongly about this subject, as I started a company to run away from these older limitations and tradeoffs that were minted in the `90s. First, however, I am compelled to acknowledge that every new storage vendor gets to stand on the shoulders of the giants of the industry that developed the prior innovation, and for this I have tremendous respect for the legacies built by both NetApp and Pure Storage.

Now, let’s turn to my Twitter post, examine the five points in that tweet, and elaborate on what makes a vendor a legacy vendor. (Expansion on this subject requires abundantly more than 280 characters!) Keep in mind that vendors don’t have to hit all points to be considered legacy vendors in my eyes, but obviously the more points they “tick,” the less relevant they are in their abilities to solve the problems of tomorrow.

Point #1: Selling Systems Built with a Proprietary “Tin”

Surprisingly, the term “tin” came out in other blogs to which Mellor referred and was not brought up by me.

Storage systems started as highly engineered hardware-based solutions in the `80s and `90s of the last century. To advance the innovations in resiliency, density, and performance, storage system designers back then were forced to engineer special buses, memory, and resilient enclosures. Then in the early 2000s there was a strong movement toward COTS (Commodity off-the-shelf) hardware-based solutions that were driven by software. I was a successful part of that movement at XIV (acquired by IBM), where we were the first to build a Tier-1 block\SAN solution with no custom-engineered hardware. Although we were not designing any hardware, all of the solutions back then still relied on specialized components from obscure/niche “shelves”; even these COTS solutions were not the standard servers you could just buy from server vendors, such as Hewlett Packard Enterprise or Supermicro. We relied on specialty vendors like Xyratex or others.

Fast forward to today. With the advancement in flash and NVMe technology, networking architectures, and server platforms, it is possible to build best-of-breed storage solutions using totally standard off-the-shelf components that are widely available from pretty much any server vendor.

Summary #1: If you are still using a storage solution that is available only in a customized proprietary hardware form factor from your storage vendor, and you cannot run it on the server vendor that you like doing business with, it’s a clear indication you’re not using a solution based on current design principles, and you’re buying a legacy storage.

This brings me to the second point from my tweet–about the cloud.

Point #2: Cloud Offering Is Different & Compromised Compared to Proprietary “Tin” → Hybrid Cloud is Limited

Legacy vendors that have solutions based on their own proprietary “tins” with hardware dependency cannot run the same software on public cloud infrastructures. They have to develop a different solution. Sure, they may give it the same name, sneaking “Cloud” in the title, but it is a totally different solution from that which runs on the custom “tin.” In fact, they even can run as a managed service on the proprietary tin inside the public cloud data center, but that is not what the cloud is about.

If they want to run on cloud infrastructures, then they must either port a subset of their solutions to run on the cloud, while changing some design elements, or create completely new products to run on the cloud. Either way, the functionality of legacy vendors’ products on the cloud is not the same as that which customers enjoy on premises (on the private cloud). The integrations need to be different, which means that scalability, resiliency, and performance of the solution on the public cloud is different.

Many of the organizations we are working with are transitioning toward a hybrid cloud model, running the bulk of their workloads on their on-premises private clouds, while having the ability to burst to the public cloud for elasticity or for DR (Disaster Recovery). This allows organizations to leverage the compelling economics of on-premises storage–and the compelling economics of cloud-DR (don’t pay unless a drill or actual disaster) and bursting using spot instances.

At Weka we were able to show that we got an enterprise-grade shared-file storage solution on AWS to the top 3 positions of the IO500 table, where the first two solutions are not even commercial offerings in GA.

Summary #2: If your current storage vendor does not support exactly the same specification for features, CLI, and performance, and if it doesn’t scale the same on-premises and in the cloud, these are clear indications that you’re using a legacy vendor.

Let’s keep going.

Point #3: Limited Scale and Support for Mixed Workloads → Tons of Silos

Modern design principles allow the creation of highly distributed storage systems that can grow to hundreds of petabytes and even exabytes. Also, it is possible to come up with IO stacks that allow running diverse applications on the same system.

Legacy solutions were created with design principles, limitations, and tradeoffs that prevented coming up with large scale–in terms of capacity or performance scaling. Moreover, this design forced tuning parameters (this was also referenced in the blogs!) that limit each system performance envelope to certain types of IOs. When data sets were relatively small and storage systems were single purpose, this innovation worked, but today’s workloads are exploring data in the petascale, and IO patterns are unknown.

You shouldn’t have to consider what your system limitations might be if you want to tune for small IOs and low latency (high IOPS number) versus large IOs with high throughput or many metadata operations. You should be able to run them all over the same data on a single scalable system built on modern hardware designs. Period.

Another important requirement, for both on-premises and the cloud, is having the ability to expand a system on demand while it is online. Otherwise, you’re stuck with a horrible sizing effort each time you start a new project and buy a system. Good up-to-date storage systems must have the ability to expand, to add more capacity or performance while in heavy production usage.

Summary #3: Go back and think about your datacenter architecture decisions.  If you must deploy more storage systems than functionally required by physical separation of resources, you’re using a legacy storage vendor. If you must make sizing decisions that lock you in for the life of that system, you’re also using a legacy storage vendor.

There’s more….

Point #4: Limited Aggregate Performance & Single Client Performance → Unfit for GPU

About a decade ago it became obvious that Moore’s law ceased to exist. The enterprise world had to solve larger, more complex problems, which led to extreme scale-out, and due to limitations of legacy storage vendors it also led to the proliferation of “Big Data” that tried to circumvent those limitations.

In the last few years GPUs have entered the data centers, and they provide significantly more compute capacity than was possible using even very large CPU compute farms.

A single, modern GPU-filled server today can reach to about 5 PetaFLOPS. This performance is equivalent to the world’s top supercomputer of about a decade ago. These supercomputers that filled rooms a decade ago had IO systems that could reach dozens of GB/s aggregate throughput to get them rolling. Now that this compute capacity fits in a single box, it still needs that kind of IO throughput, and when leveraging multiple GPU-filled servers the aggregate throughput needs to be even greater.

Future-looking companies are migrating their workloads to GPUs. To be effective, each GPU-based server needs to access data that historically would have equaled the aggregate throughput of dozens or hundreds of legacy CPU servers connected to different storage systems. This trend started with AI\ML, but now we see many examples across pharmaceutical companies, financial organizations, and retail establishments where customers are replacing large-scale, “Big Data,” open-source solutions that fill entire data centers with a single machine or a few racks of infrastructure.

In order to leverage and unleash the power of the GPU platform, storage systems need to “up their game” in terms of aggregate overall performance and, even more importantly, single-client performance; otherwise, they face the risk of wasting the valuable compute cycles of these dense compute systems.

Summary #4: If the storage system you’re using now has the same single-client performance limitations that existed about a decade ago, you’re using a legacy storage vendor. If the system you’re using now has an aggregate throughput\IOPS number that has not increased dramatically when compared with numbers a decade ago, you’re using a legacy storage vendor.

Let explore the final point from my tweet.

Point #5: Data Backup and DR Are Performed by Others–or They’re Afterthoughts

Data Protection (backup\archive\etc.) is a huge responsibility for any storage professional. Back in the time when storage systems were limited in scale and workload, mobility to the cloud was unheard of, backup was a simple procedure, which led to the proliferation of “secondary storage” backup vendors.

With data capacities growing exponentially, businesses demand a sound DR strategy, a cloud bursting strategy, and also backup and archive on-premises and on the cloud. Viewing these solutions as discrete and solving each one individually with separate products is expensive and wasteful, however.  When your data grows to the petabyte scale you don’t want to store too many copies of it to satisfy separate uses (three for archive, another one for DR, etc.) Also, you do not want to integrate several different products to do that.

Now, if we considered the “bread and butter” features of storage resiliency and integrity, here are some questions to ask, the answers of which could indicate that you’re using a legacy vendor from these perspectives:

• Does your storage system crawl to its knees when it is 75% full?
• Does storage system performance crawl to a halt during rebuilds, or as long some hardware component is still faulty?
• Do you still have to build 100% of the failed storage media even when it had little data?

Summary #5: If your primary storage product, your backup product, and your cloud product are different entities, you are dealing with a legacy storage vendor. If your storage vendor forces you to treat backup and DR differently and store the data twice, you’re using a legacy vendor. If your storage vendor forces you to go to a third-party solution in order to get sound backup or archive strategy, you’re also using a legacy vendor. If the performance of the storage system drops significantly during rebuild, you’re using a legacy vendor. If you are still rebuilding blocks and not files, you are using a legacy vendor. If you don’t have end-to-end data integrity protection to the client (each block has a checksum that is calculated at the client and verified on each step of the way to ensure no bit-rot), you’re using a legacy storage vendor.

There’s one more point to make.

A Bonus: Point #6

I can go on longer about what makes a good storage solution, and this is probably a good subject for the next blog.  I have many positive aspects to share about up-to-date storage solutions that have different parallels from those described here—that is, other than those regarding the characteristics of legacy storage vendors.

Speaking of which, I’m going to add another identification cue to this guide even though it was not part of my original tweet. Another strong indication that you’re buying from a legacy storage vendor is this: if that vendor has many offerings, and you have to use different products to achieve your goal, this is an indication that what you’re currently trying to achieve was not the original design point of the solution offered to you.

Summary #6: If your storage vendor has many products, each with slightly different tradeoffs, and you have to use a different mix of them as solutions to different projects, you’re using a legacy vendor.

Conclusion

It’s important to reiterate that all new storage vendors get to build on the great technologies of the legacy vendors. I respect all of Weka’s competitors in the marketplace and acknowledge that the majority of the storage solutions that are sold today are solid–and obviously still have a place in the data center of today. As CIOs start to think about the datacenters of the future and what objectives they will have to meet in years to come, they are well served to question whether the solution that took them to 2020 will meet the needs of the future. Each generation produces a giant leap in performance and scale.

By the way, if you’re still curious as to my opinion about the original matter, both NetApp and Pure are obviously selling legacy storage products by the standards described here. Clearly, both are legacy storage vendors.