Dec. 20 — The Military Applications Department of the French Atomic Energy Commission (CEA/DAM) has developed since 2006 a policy engine named “Robinhood,” to deal with the growing amount of data and metadata on its HPC filesystems.
Robinhood Policy Engine is a Swiss-army knife to manage large filesystems. It integrates many data management features such as access monitoring, accounting, scheduling of massive data operations like data archiving, cleaning unused data…
With this new 2.5 version, Robinhood adds new strings to its bow: it implements disaster recovery features that allow rebuilding filesystem metadata after a major corruption. It also supports the latest new features in filesystems, by implementing an automatic scheduling of data movements for Lustre 2.5 HSM.
This tool is available as an Open Source Software on sourceforge since 2009. It has a constantly growing popularity in the world of HPC filesystems especially in sites using Lustre (see: http://robinhood.sourceforge.net).
Robinhood is also integrated by several vendors in their storage solutions.
Robinhood core: a filesystem metadata replicate that can be updated near real-time
The core component of Robinhood is a database that constitutes a filesystem metadata replicate and is optimized to ingest information at a high-rate. This database can be populated by performing filesystem namespace scanning, which is made efficient thanks to a massively multi-threaded traversal algorithm.
Even better, the database can be updated by reading incremental changes from the filesystem, which offers significant benefits:
- this is much more efficient as it removes the need for scanning the namespace, which is a long operation on large filesystems.
- contents of the database are updated near real-time which allow providing fresher information than any other statistics or monitoring tool.
In particular, this second method is possible with Lustre v2 filesystems as they implement a transactional ChangeLog mechanism to keep an external application updated about filesystem changes.
Get rid of your old-fashioned scripts…
Using this metadata replicate, robinhood allows searching for entries using various criteria much more efficiently than using traditional UNIX tools like ‘find.’
By the way, it provides clones of these commands (rbh-find, rbh-du) to allow fast queries without changing administrators habits. Another interest of those commands is to offload the filesystem from an usual metadata traffic, thus releasing resources for data production.
The SQL language of the database engine also makes it very easy to aggregate information and extract customized statistics for the millions or billions of entries in the filesystem, in a very short time. Last but not least, the database contents are used by robinhood policy modules to schedule massive actions on filesystem entries, based on customizable criteria on file attributes, path, extended attributes…
With all these features, Robinhood Policy Engine can usefully replace bunches of home-made scripts that all do the same expensive namespace traversal in order to perform their job: scan the namespace to search for entries, scan the namespace to get statistics, scan the namespace to execute actions on entries…
Robinhood Policy Engine – as its name reminds us – was first designed to massively schedule actions on filesystem entries: schedule data movements between a filesystem and a long-term storage system, perform space cleaning on a temporary filesystem when it gets full…
Its flexible policy language allows defining arbitrary file classes based on entry attributes, and applying different policies to the file classes (e.g. archive some files quickly than others, never purge some other files…). Policy actions are performed in parallel by pool of worker threads, which results in a high speed of execution.
With its policy management capabilities, its long experience in managing mass data movements, and a particular support of Lustre specific features (changelogs, striping, OST pools, fids…), Robinhood is a favorite choice as a policy engine for Lustre 2.5 HSM.
A short-term concern in Robinhood development is to make it more generic and dynamic, by implementing a framework of dynamically loaded modules. This will allow filesystem administrators to schedule any kind of actions (data integrity scrubbing, Lustre OST re-balancing…), possibly using their own criteria, not necessarily using built-in conditions.
In the mid-term, the challenge for this software is to deal with metadata parallelism. Indeed, a general trend in filesystems is to parallelize the metadata service (like in Lustre with DNE) which results in multiplying the load on a metadata auxiliary service like Robinhood. This could possibly be addressed by integrating Big Data technologies like NoSQL databases.
About the CEA
The French Alternative Energies and Atomic Energy Commission (CEA) leads research, development and innovation in four main areas: low-carbon energy sources, global defense and security, information technologies and healthcare technologies. The CEA’s leadership position in the world of research is built on a cross-disciplinary culture of engineers and researchers, ideal for creating synergy between fundamental research and technology innovation. With its 15,600 researchers and collaborators, it has internationally recognized expertise in its areas of excellence and has developed many collaborations with national and international, academic and industrial partners.