[slurm-users] Kinda Off-Topic: data management for Slurm clusters

[Will Dennis]

Yes, we've thought about using FS-Cache, but it doesn't help on the first read-in, and the cache eviction may affect subsequent read attempts... (different people are using different data sets, and the cache will probably not hold all of them at the same time...)


On Friday, February 22, 2019 2:11 PM, Mark Hahn said:

>how about just using cachefs (backed by a local filesystem on ssd)?
>
>https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/storage_administration_guide/ch-fscache
>
>if files are being re-read, this would be effective, fast, and convenient, and wouldn't require any staging or hooks into Slurm.
>
>regards, mark hahn