New OpenSFS Rep on the Future of Lustre

By Richard L. Brandt

March 12, 2013

Tommy Minyard has been selected as this year’s Community Representative Director at Open Scalable File Systems (OpenSFS). Minyard will continue to hold down his “day job” as Director of Advanced Computing Systems (ACS) at the Texas Advanced Computing Center (TACC), where his group is responsible for operating and maintaining TACC’s production systems and infrastructure. He replaces 2012 Community Representative Director Stephen Simms, manager of the High Performance File Systems at Indiana University.

Minyard’s current projects at TACC include world-class science through leadership in high performance computing, HPC research using clusters, system performance measurement and benchmarking, and fault-tolerance for large scale cluster environments. He earned a PhD in Aerospace Engineering from the University of Texas at Austin.

HPCwire asked Minyard about his new role at OpenSFS and the future of Lustre.

HPCwire: How would you describe the status of Lustre today?

Minyard: Lustre has become a very stable and robust filesystem. It provides an extremely scalable platform for building HPC systems at any scale. We have run many versions of Lustre here at TACC and each release has improved in stability. Each release has added features to make installation and administration of Lustre much easier.

Also, with the establishment of the OpenSFS and EOFS community organizations, the overall Lustre community remains vibrant and active with growing international participation.

HPCwire: What opportunities do you see ahead for Lustre, and why?

Minyard: One key opportunity for Lustre is in the new push addressing big data. We are starting to see Lustre being used in large scale data analytics and as a possible alternative/compliment to Hadoop for MapReduce types of problems. With its scalable architecture and upcoming feature enhancements, I expect to see Lustre used in many more non-traditional HPC environments that have big data problems to solve.

HPCwire: What are the most important issues the Lustre community needs to deal with in the near term?

Minyard: One is continued development of features and ensuring that a single source tree for Lustre is maintained. Many parties contribute to Lustre and each has their own development priorities, which could lead to bifurcation of the Lustre source tree. OpenSFS is funding Lustre development and source-tree maintenance activities, along with coordinating with EOFS, to avoid this situation as new features are implemented and bugs are fixed.

HPCwire: Long term?

Minyard: Longer term, I think the most important issue for Lustre will be growing and sustaining the community development efforts initiated by OpenSFS and EOFS. I am glad to see vendors willing to support and offer Lustre-based solutions. However, it will be the funding from these two organizations that will continue development of new Lustre features for the community in order to keep it a viable open source package. Without increasing and broadening community support, these organizations could have trouble continuing to fund the very expensive software development required to keep Lustre at the forefront as new hardware and software technologies become available.

HPCwire: What kinds of features or capabilities would you most like to see added to Lustre? How will these capabilities improve Lustre?

Minyard: Some of the features that I would like to see added to Lustre are currently on the development roadmap – primarily, the implementation of distributed namespace (DNE) and the Hierarchical Storage Management (HSM) features slated for the 2.4 and 2.5 releases. DNE will resolve one of the primary meta-data bottlenecks present in current Lustre releases. HSM will allow Lustre to be used in a wider range of systems, including tape archival libraries.

HPCwire: Do you have any particular initiatives you’d like to implement this year?

Minyard: One of my primary initiatives for this year will be to expand the Lustre community and encourage more institutions to join the OpenSFS organization. The Lustre community has always been active on the various mailing lists, but I would like to see more of them participate and attend the annual Lustre User Group meeting. I look forward to LUG 2013 in April, where we will hear about recent developments from existing and new Lustre users, will see presentations from the vendors and will discuss the roadmap for upcoming releases.

HPCwire: In the last couple years, we’ve seen Intel buy Whamcloud and more recently Xyratex buy Oracle’s Lustre assets. What does this mean for the Lustre community going forward?

Minyard: My hope is that these two acquisitions will enhance the long-term viability of Lustre by ensuring that the filesystem remains supported and commercially viable, yet still available to the community as an open-source package.

Intel’s purchase of Whamcloud shows me that Intel considers Lustre to be critical on today’s current HPC platforms as well as on future exascale systems. I was also pleased to see Xyratex acquire the Lustre rights from Oracle, as it was my understanding that Oracle had stopped active development of future releases. I think Xyratex will ensure that the Lustre trademark and brand are maintained for the community.

HPCwire: What are the long-term prospects of the technology for future exascale systems?

Minyard: Lustre will be a key piece of software for deployment of exascale systems. Lustre has already proven itself to be extremely scalable. It can keep up with the growing system size from terascale to petascale now, and has already demonstrated a bandwidth capability exceeding 1TB/sec. With the planned features that will be implemented in the next few years, Lustre should evolve in such a way that it may be one of the only filesystems to work reliably on exascale systems.

Related Articles:

HPC Veteran Joins OpenSFS as Community Representative Director

Xyratex Captures Oracle’s Lustre

OpenSFS Increases Investment in Open Source for HPC

Subscribe to HPCwire's Weekly Update!

Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week!

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from its predecessors, including the red-hot H100 and A100 GPUs. Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. While Nvidia may not spring to mind when thinking of the quant Read more…

2024 Winter Classic: Meet the HPE Mentors

March 18, 2024

The latest installment of the 2024 Winter Classic Studio Update Show features our interview with the HPE mentor team who introduced our student teams to the joys (and potential sorrows) of the HPL (LINPACK) and accompany Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the field was normalized for boys in 1969 when the Apollo 11 missi Read more…

Apple Buys DarwinAI Deepening its AI Push According to Report

March 14, 2024

Apple has purchased Canadian AI startup DarwinAI according to a Bloomberg report today. Apparently the deal was done early this year but still hasn’t been publicly announced according to the report. Apple is preparing Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimization algorithms to iteratively refine their parameters until Read more…

Nvidia’s New Blackwell GPU Can Train AI Models with Trillions of Parameters

March 18, 2024

Nvidia's latest and fastest GPU, code-named Blackwell, is here and will underpin the company's AI plans this year. The chip offers performance improvements from Read more…

Nvidia Showcases Quantum Cloud, Expanding Quantum Portfolio at GTC24

March 18, 2024

Nvidia’s barrage of quantum news at GTC24 this week includes new products, signature collaborations, and a new Nvidia Quantum Cloud for quantum developers. Wh Read more…

Houston We Have a Solution: Addressing the HPC and Tech Talent Gap

March 15, 2024

Generations of Houstonian teachers, counselors, and parents have either worked in the aerospace industry or know people who do - the prospect of entering the fi Read more…

Survey of Rapid Training Methods for Neural Networks

March 14, 2024

Artificial neural networks are computing systems with interconnected layers that process and learn from data. During training, neural networks utilize optimizat Read more…

PASQAL Issues Roadmap to 10,000 Qubits in 2026 and Fault Tolerance in 2028

March 13, 2024

Paris-based PASQAL, a developer of neutral atom-based quantum computers, yesterday issued a roadmap for delivering systems with 10,000 physical qubits in 2026 a Read more…

India Is an AI Powerhouse Waiting to Happen, but Challenges Await

March 12, 2024

The Indian government is pushing full speed ahead to make the country an attractive technology base, especially in the hot fields of AI and semiconductors, but Read more…

Charles Tahan Exits National Quantum Coordination Office

March 12, 2024

(March 1, 2024) My first official day at the White House Office of Science and Technology Policy (OSTP) was June 15, 2020, during the depths of the COVID-19 loc Read more…

AI Bias In the Spotlight On International Women’s Day

March 11, 2024

What impact does AI bias have on women and girls? What can people do to increase female participation in the AI field? These are some of the questions the tech Read more…

Alibaba Shuts Down its Quantum Computing Effort

November 30, 2023

In case you missed it, China’s e-commerce giant Alibaba has shut down its quantum computing research effort. It’s not entirely clear what drove the change. Read more…

Nvidia H100: Are 550,000 GPUs Enough for This Year?

August 17, 2023

The GPU Squeeze continues to place a premium on Nvidia H100 GPUs. In a recent Financial Times article, Nvidia reports that it expects to ship 550,000 of its lat Read more…

Analyst Panel Says Take the Quantum Computing Plunge Now…

November 27, 2023

Should you start exploring quantum computing? Yes, said a panel of analysts convened at Tabor Communications HPC and AI on Wall Street conference earlier this y Read more…

Shutterstock 1285747942

AMD’s Horsepower-packed MI300X GPU Beats Nvidia’s Upcoming H200

December 7, 2023

AMD and Nvidia are locked in an AI performance battle – much like the gaming GPU performance clash the companies have waged for decades. AMD has claimed it Read more…

DoD Takes a Long View of Quantum Computing

December 19, 2023

Given the large sums tied to expensive weapon systems – think $100-million-plus per F-35 fighter – it’s easy to forget the U.S. Department of Defense is a Read more…

Synopsys Eats Ansys: Does HPC Get Indigestion?

February 8, 2024

Recently, it was announced that Synopsys is buying HPC tool developer Ansys. Started in Pittsburgh, Pa., in 1970 as Swanson Analysis Systems, Inc. (SASI) by John Swanson (and eventually renamed), Ansys serves the CAE (Computer Aided Engineering)/multiphysics engineering simulation market. Read more…

Intel’s Server and PC Chip Development Will Blur After 2025

January 15, 2024

Intel's dealing with much more than chip rivals breathing down its neck; it is simultaneously integrating a bevy of new technologies such as chiplets, artificia Read more…

Choosing the Right GPU for LLM Inference and Training

December 11, 2023

Accelerating the training and inference processes of deep learning models is crucial for unleashing their true potential and NVIDIA GPUs have emerged as a game- Read more…

Leading Solution Providers

Contributors

Baidu Exits Quantum, Closely Following Alibaba’s Earlier Move

January 5, 2024

Reuters reported this week that Baidu, China’s giant e-commerce and services provider, is exiting the quantum computing development arena. Reuters reported � Read more…

Training of 1-Trillion Parameter Scientific AI Begins

November 13, 2023

A US national lab has started training a massive AI brain that could ultimately become the must-have computing resource for scientific researchers. Argonne N Read more…

Shutterstock 1179408610

Google Addresses the Mysteries of Its Hypercomputer 

December 28, 2023

When Google launched its Hypercomputer earlier this month (December 2023), the first reaction was, "Say what?" It turns out that the Hypercomputer is Google's t Read more…

Comparing NVIDIA A100 and NVIDIA L40S: Which GPU is Ideal for AI and Graphics-Intensive Workloads?

October 30, 2023

With long lead times for the NVIDIA H100 and A100 GPUs, many organizations are looking at the new NVIDIA L40S GPU, which it’s a new GPU optimized for AI and g Read more…

AMD MI3000A

How AMD May Get Across the CUDA Moat

October 5, 2023

When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically... Read more…

Shutterstock 1606064203

Meta’s Zuckerberg Puts Its AI Future in the Hands of 600,000 GPUs

January 25, 2024

In under two minutes, Meta's CEO, Mark Zuckerberg, laid out the company's AI plans, which included a plan to build an artificial intelligence system with the eq Read more…

Google Introduces ‘Hypercomputer’ to Its AI Infrastructure

December 11, 2023

Google ran out of monikers to describe its new AI system released on December 7. Supercomputer perhaps wasn't an apt description, so it settled on Hypercomputer Read more…

China Is All In on a RISC-V Future

January 8, 2024

The state of RISC-V in China was discussed in a recent report released by the Jamestown Foundation, a Washington, D.C.-based think tank. The report, entitled "E Read more…

  • arrow
  • Click Here for More Headlines
  • arrow
HPCwire