This week HPC storage vendor Panasas announced that they are making their parallel file system client software, called DirectFLOW, publicly available via open source. Their goal is to accelerate the adoption of the Parallel Network File System (pNFS) standard, which is to be released as an extension to the Network File System version 4 protocol, and will be technically known as NSF version 4.1.

HPCwire spoke with Larry Jones, Vice President of Marketing for Panasas, to get the details of the announcement.

According to Jones, pNFS is the first performance upgrade to NFS in two decades, since the protocol was developed by Sun Microsystems. Jones explained that with NFS, you have a tier of servers between the clients and the storage systems and all of your data is read in or read out through those servers. With pNFS, the servers get pulled off to the side; they still control access to the storage systems to make sure that the data is properly protected, but the clients get both direct and parellel access to that storage system — that's what accounts for performance boost.

There exists a huge imbalance between the compute side and the I/O architectures that pNFS resolves as it allows for parallel and direct access from the client to the storage device.

Jones explained that “NFS with its serial one-lane-in, one-late-out architecture doesn't meet the needs of the HPC community and increasingly the enterprise market” and the solution is pNFS.

Increased use of parallelism from cluster and multi-core processor deployments is driving the need for parallel storage to improve application performance. pNFS offers increased scalability, performance and manageability. In addition, users will have the comfort of not being locked in a by a single vendor.

Jones explained that the increased manageability is especially a boon for HPC customers. pNFS allows a single global namespace instead of having a mount table with 25 NFS servers on it. A single mount point allows for very scalable storage system, making it easier to maintain performance without all the load-balancing issues.

Although the same advantages are currently available in proprietary parallel systems like IBM's GPFS, IBRIX's Fusion Parallel File System, and Panasas' own PanFS, these systems are incompatible with each other. pNFS offers a single standard that should encourage customer deployments of parallel storage solutions.

Panasas has taken a three-step strategy to promote adoption of parallel storage: to create great products that show the potential for parallel storage, work those into standards so the industry can evolve, and, for the end users to benefit, work with the ISV community so that they can develop their software accordingly.

Panasas believes that in order for there to be a big market for parallel storage there needs to be a standard. They hope the company's efforts will energize the market. Jones said that Panasas would “like to see the whole market evolve, so that we can move from that single-lane NFS to parallel NFS and do it across the entire industry.”

To help accomplish this goal, Panasas has fostered dedicated development resources. They have recently opened an R&D center in TelAviv, Israel, where they have engineers specifically working on the Linux implementation of pNFS, i.e., inserting pNFS into the Linux kernel. There is also a group at the University of Michigan, called The Center for Information Technology Integration (CITI), working towards the same goal. Jones expressed confidence that the pNSF standard will be included in future Linux distributions, perhaps by next year sometime.

Panasas' founder and CTO, Dr. Garth Gibson, a couple of their engineers, and Peter Corbett of Network Appliance wrote the first draft of the pNFS spec back in 2004. They based the architecture on Panasas' DirectFlow protocol. Since then, Panasas has been evolving their software in concert with the standard. Other vendors working on the pNFS standard include Network Appliance, IBM, EMC and Sun.

The Internet Engineering Task Force (IETF) NFSv4 subcommittee is reviewing pNFS as part of NFS version 4.1 and is expected to be finished in September. As soon as that happens, Panasas customers will be able to upgrade their software to the standard. Support is expected on Linux, Windows, and leading Unix versions such as AIX and Solaris. Ultimately, Jones hopes that users will be able to get the technology from their Linux distribution (SUSE, Red Hat, etc.).

Jones concluded by saying: “We think that parallel is the wave of the future. NFS hasn't had a performance kicker in twenty years and it's about time it got one. We believe that parallel is the way to go and we've been leaders in that.”