FASRC DOCS

Skip to main content

Search Docs by Keyword

Table of Contents

Storage Modernization Initiative: Frequently Asked Questions (FAQs)

  • Compute Storage is designed as an active storage location for data analysis, with data being readily accessed and utilized. It is a highly performant, cluster adjacent storage, optimized for high throughput data access, including AI and ML.
  • Lab Storage is a general-purpose storage for hosting raw and project data. It is not intended for heavy computational workflows but can be used as a buffer storage for lab instruments. Some lab groups may decide to utilize multiple storage offerings, including both Compute Storage and Lab Storage.
  • A full listing of the features on Compute Storage and Lab Storage including associated costs, backups, and security levels, is available on our Data Workflows webpage.
All data currently residing in a FASSE storage folder will be moving to our new FASSE storage. Information about the new features, costs, and security level is provided on our our Data Workflows webpage.
The full Storage Transition schedule is available on our Storage Modernization Initiative webpage. Specific timelines and dates will be communicated directly to affected labs prior to the migration. Netscratch, holylabs, and home folders will not be affected by this migration.
All active group members will receive the email communications about upcoming storage migrations. PIs can review the list of group members in the Coldfront Allocation dashboard. To login to Coldfront, please use your FAS RC username and password. If you have difficulties with your password, you can reset it. We recommend you access the Coldfront dashboard while on the FAS RC VPN.
The full list of features for Compute Storage is available on the Data Workflows webpage. Compute Storage comes with weekly snapshots every 2 weeks. No disaster recovery option is provided.
  • It is important to have a post-processing plan for Compute Storage including the option to move post-processed data to a location with disaster recovery, if needed.
The full list of features for Lab Storage is available on our Data Workflows webpage. Lab Storage will include daily backups for 7 days and weekly backups for 4 weeks.
We will provide you with the folder path once the new folder is ready to use. If the migration has completed, you can follow steps in our Getting Started with FAS RC Storage documentation to locate the folder.
Please email rchelp@rc.fas.harvard.edu to request to be removed from a lab group.
Please email rdm@rc.fas.harvard.edu to discuss migrating the data to another storage offering.
FAS RC will migrate existing folders on each designated storage to the new Lab Storage. Prior to the cutover date, the data in your lab folder will be marked read-only. Once FAS RC has fully completed the migration, you will receive an email with the new folder path. The older version on the previous storage will then be decommissioned.
Folder paths for all groups will be changing. You may need to update your existing scripts or documentation to reflect this change. Two subdirectories will be present by default within the folder.
  • Lab: This directory is intended as the primary working directory. It is also the directory shared out via Globus. By default, folders in this subdirectory are visible to the whole lab. Individual users may update their permissions to adjust access as they like though we highly recommend keeping access open to all lab members to allow for easier collaboration and data cleanup after you leave the university.
  • Everyone: This directory is visible to any one on the HPC cluster and is intended for collaboration with other labs on the cluster. Data in this directory is by default owned by the lab who hosts the data. Note that this directory is not available on Globus and is intended only for internal sharing.
  • Directory structures may differ depending on when they were created. Some older storage folders may have a third subdirectory called Users. We have deprecated use of this folder due to issues related to data access.
Netscratch, holylabs, and home folders will not be affected by the data migrations. They will be remaining in their current location. No data from these filesystems will be migrated.
  • To request a new storage allocation, or to modify an existing storage allocation, please login to the Coldfront Allocation Dashboard. To login to Coldfront, please use your FASRC username and password. If you have difficulties with your password, you can reset it. You may also need to clear the cache on your website browser. If requesting a new storage allocation, you will need to indicate which storage offering you would like to acquire and the associated 33-digit billing code. If you do not have a FASRC Account, you will need to request one before logging into Coldfront.
  • PIs, General Managers, and Storage Managers are able to request new allocations or make changes to existing allocations. PIs can assign a General Manager or Storage Manager role to their lab using Coldfront; this will allow the lab member to add and/or modify storage allocations.
  • Lab Storage is $125/TiB per year, billed monthly.
  • For example, if you have a 20TiB storage allocation, the cost will be ($125 x 20TiB)/12 for a monthly bill of ~$208.
  • Billing is based on total allocation size, not by usage.
You can use a variable for pathnames in scripts.
  • For example set lab_storage=, then change hardcoded paths to $lab_storage/Lab/data.
To find which of your scripts have hard-coded paths, you can use the grep command recursively in a directory.
  • For example, in your home directory you might use: grep -R “isilon” /n/home00/jharvard or Grep - R “isilon” [your new lab path]/Lab/jharvard
Lab Storage will be available via SMB. Compute Storage will not be available via SMB.
Please email rdm@rc.fas.harvard.edu if you have additional questions regarding the Storage Modernization Initiative or future storage migrations.
Last Updated
  • © The President and Fellows of Harvard College.
    Except where otherwise noted, this content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.