Policy: scratch space

$SCRATCH VARIABLE

During the December 3rd 2020 maintenance, a new global variable called $SCRATCH was introduced into the environment. This variable will allow scripts and jobs to point to a specific directory in scratch regardless of any changes to the name or path of the scratch filesystem. This variable currently points to /n/holyscratch01 so, for example, one could use the path $SCRATCH/jharvard_lab/jsmith in a job script. This will have the added benefit of allowing us to change scratch systems at any time without your having to modify your jobs/scripts.

SCRATCH FILESYSTEM

RC maintains a large, shared temporary scratch filesystem for general use for high input/output jobs at /n/holyscratch01 . This workspace is made available with the provision that each group use no more than 50TB, and that files older than 90 days will be deleted through a periodic purge process. This purge can run at any time, especially if scratch is getting full (df -h /n/holyscratch01 to see current total use) and is also run at the start of the month during our maintenance period.

There is no charge for scratch, but please note that it intended as volatile, temporary scratch space intended for transient data and is not backed up. If your lab group needs a directory in scratch, please contact us.

Modifying file times (via touch or other process) to avoid retention is an abuse of the filesystem and will result in administrative action from RC.  You may initially modify the file date(s) on new data so that it is not in the past, but should not modify it further. These actions may include deleting data on the scratch system owned by the offender and disabling the offending account.  If data on scratch needs to be kept longer, please contact us to discuss options.

SCRATCH QUOTAS

To view your group’s scratch quota and usage:
lfs quota -hg lab_name /n/holyscratch01

Was this article helpful?
0 out of 5 stars
5 Stars 0%
4 Stars 0%
3 Stars 0%
2 Stars 0%
1 Stars 0%
How can we improve this article?
Need help?
© The President and Fellows of Harvard College
Except where otherwise noted, this content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.