Since 1986 - Covering the Fastest Computers
in the World and the People Who Run Them
Visit additional Tabor Communication Publications
Thank you for your interest in this content. Your download is available at the link below:
Map/Reduce on Lustre®
Source: Xyratex
Release Date:October 4, 2011
This paper compares HDFS and Lustre architectural drivers and resulting system performance of Map/Reduce computations on HPC hardware. We evaluate theoretical and actual performance of Lustre and HDFS for a variety of workloads in both traditional and Map/Reduce-based applications. Further, we examine the additional benefits (cluster efficiency, flexibility, and cost) of using a general-purpose distributed file system, such as Lustre, on a Hadoop compute cluster.