Lustre has emerged as the file system of choice for many HPC users based on its ability to provide scale-out high performance storage that meets the needs of scale-out HPC applications running on SGI servers. However, because of their role in providing “Formula 1” levels of performance—Lustre file systems have earned a reputation as being a storage environment that’s best used for short term “scratch” data versus long term information management.
For several years, “storage nirvana” for Lustre users has been the potential of having a Lustre environment that can provide outstanding performance and scale to almost infinite capacity—while not requiring an almost infinite budget for storage hardware and IT resources. The good news is that the most recent releases of Lustre and SGI Data Management Fabric (DMF) are bringing nirvana within reach.
The recent release of Lustre now includes integrated capabilities for linking to hierarchical storage management (HSM) systems—and SGI’s Data Management Fabric (DMF) solution is perhaps the most feature rich and scalable HSM on the market. Combining the two platforms as a certified and supported combination from SGI makes perfect sense from both a data management and budget management perspective.
Deploying a unified “Lustre and DMF” solution allows customers to create a seemingly infinitely scalable Lustre environment where user-specific policies keep the most active data on high-performance storage while less active data is invisibly migrated in the background to cost-correct storage that can include lower cost disk, cloud-based storage and library-based tape. All of your data is online and available—all of the time.
It’s the beginning of a shinier new era for Lustre.