Search Docs by Keyword
Data Deletion Recommendations
Properly deleting unnecessary or duplicative datasets is an important step in maintaining a clean and organized research environment. Included below are recommendations from FAS Research Computing regarding how to safely and securely delete data in your home or lab directories. This document focuses on the deletion of research data, rather than physical records, or software environments.
These are recommendations based on our experience with data deletion. Please make sure you are aware of how the tools and commands operate.
Only perform data deletions on data approved by your applicable PI or Group Leader. Data accidentally deleted is likely not retrievable depending on where it was stored.
For further information about FASRC Storage offerings and which options come with inherent backups, please refer to Data Storage (Offerings, Workflow, Costs).
Data Retention:
Policies to review prior to data deletion:
Note: Essential research records should be retained, generally, for a period of no fewer than seven (7) years after the end of a research project or activity, according to Harvard guidelines. Further guidance about what constitutes an “essential research record” is outlined in the Records Retention documentation provided.
Preliminary Steps
- Confirm with your PI or Project Leader that folders or files can be permanently deleted. The PI or Project Leader is responsible for all research generated in their lab or group and therefore has the ultimate say regarding what data can be deleted for all lab folders.
- Validate your path with non-destructive commands like ‘ls’ or ‘find’ prior to deletion
- ls -la <path>
- Create a listing of all files that will be deleted for reference in the future. This prevents confusion about what files or folders were removed.
- tree [options] – The tree command displays the hierarchical structure of the directory, showing its subdirectories and files in a tree-like format.
Data Deletion using Open OnDemand
The Open OnDemand Files app provides a web-based graphical interface for managing files. Individual files/folders can be selected using the checkbox next to the file/folder name, and removed with the red “Delete” button. Note that you must be connected to the FASRC VPN to access Open OnDemand.
Data Deletion using CLI (Command Line Interface)
Not recommended unless you have read and fully understand how each of these commands operate.
- rm / rm -rf
- rm removes files where you have permission to do so. The two most common, but also dangerous, flags for rm are -r which makes the command recursive and -f which forces deletion without prompting. These can be combined into -rf (rm -rf) but please see the important note below. Extreme care should be used when using rm/rm -rf
- rmdir
- Removes the directory. The directory must be empty.
IMPORTANT NOTE ABOUT RM: Always use absolute paths (e.g. /n/mylab/myfolder/old-data) rather than relative paths (e.g. – old-data/ or ./* ). We also highly recommend you do not run rm -rf * as you are more likely to make a mistake. Using the absolute path ensures that you see what you’re about to do and also will receive an error message if it’s incorrect.
Post-deletion Steps
- Validate the folder or file was removed, using non-destructive commands like ‘ls’ or ‘find’
Tape
Deleting data in NESE Tape requires additional assistance from the vendor. Please email rchelp@rc.fas.harvard.edu if you would like to investigate this option.
Data with Specific Destruction Requirements
If your data is governed by a data use agreement, contractual obligations, or other requirements that mandate the data must be destroyed in a certain way, or with a certificate, please contact FASRC to discuss this, letting them know the specific folder path(s), and the related DUA(s) (e.g. DUA12-1234) or FASRC FASSE projects (e.g. piname_projectname_l3).
Contact
If you require additional assistance with the deletion or would like to ask questions, please email FASRC at rchelp@rc.fas.harvard.edu or join us for our weekly virtual Office Hours.