Winter break Dec. 20th - Jan. 2nd - Harvard and FASRC closed.
Dec. maintenance deferred. Next monthly maintenance Jan. 5th (full day)

Tag: Storage service center

Starfish  – Data Management

Starfish – Data Management

To scan and manage data on different filesystems, FASRC is using Starfish. We have 2 different Starfish servers to scan our storage servers and the information is below:

Starfish Service URL Storage Servers
https://starfish01.rc.fas.harvard.edu/  holylfs04 (Tier0), holylfs05 (Tier0), holystore01
https://starfish02.rc.fas.harvard.edu/  holylfs02, holy-isilon(Tier1), bos-isilon(Tier1), boslfs02 (Tier 0)

Note: If your lab has not requested access to starfish already, please reach out to us: rchelp@rc.fas.harvard.edu

 

Starfish Features:

  • View of all your lab data folders with basic statistics per folder (usage, owner, creation, last modified, last access data)
  • View access only to your lab’s data, using a zone.
  • Export basic statistics to a CSV file to share and offline review
  • Tagging of data based on project/owners
  • Summary of usage based on file extensions, users, and tags.

 

Coldfront – Allocation management

Coldfront – Allocation management

Please check the Storage Service Center page for the dataflow to ColdFront

ColdFront is an open-source resource allocation management system designed to provide a central portal for administration, usage reporting, and allocation management of HPC resources.  FASRC adapted the software to manage allocations on the FASRC cluster.  Right now, the service has 2 major components:

Project: Lab groups and projects, if your lab has more than one group or you are part of multiple groups you will see more than one project.

Allocation: Storage Allocation on one or more storage devices.

Users need to connect to @fasrc VPN and can log in to Coldfront using FASRC credentials:

https://coldfront.rc.fas.harvard.edu/

After you log in to coldfront, you will see one or more projects on the left and one or more allocations on the right. Click, project link or Allocation active button to view details about your projects/allocation.

Project/s:

Project list all the users in your lab group. These are 2 main roles, Manager and User. Only the users with the manager role are allowed to request new allocations or request changes to the current allocation.  A user role is to view their current projects and allocations.  The allocation tab shows your current allocations and usage.

Note: If you should be authorized to request allocations, please ask the PI to log in to coldfront and promote your user as manager.

Allocations:

The page shows the total allocation size, total usage, and estimated cost per month. It also has a table to show usage per user and the information is updated once a week from the data management system (Starfish). It shows the estimated cost per month after aggregating their usage across different folders. Starfish is used to get this data and Labs can request access to starfish or its data to better understand the usage and remove any old project or user data.

Currently, allocation requests are for different storage Tiers (Tier0, Tier1, Tier3). We are still working on Tier2 to offer it as a service.

Allocation Requests:

PI or Managers of a lab project should be able to make a new allocation request or request changes to the current allocation. To update the current allocation, please add the new total size and add a comment in the justification. Please review https://www.rc.fas.harvard.edu/services/data-storage/#Offerings_Tiers_of_Service for the storage features and updates

If you have any questions. please feel free to reach us here: rchelp@rc.fas.harvard.edu

Storage Service Center

Storage Service Center

This page is to provide information on different software components deployed to help PIs, Lab managers, Data managers, and Finance managers to manage storage allocation on FASRC’s storage service center. To get more information about the storage tiers and their features please visit data-storage services and storage tiers. Please feel free to reach out to us at  rchelp@rc.fas.harvard.edu with any questions.

Starfish – Data Management

Scans the different storage servers to provide a view, usage details, metadata, and tagging based on the projects. Check here for more details about starfish and examples to query the data.

Coldfront – Lab and Allocation management:

Provide a view on PI projects and allocations. New allocation and updates to existing allocation can be requested using Coldfront. Check here for more details about Coldfront and its use.

Fiine -FAS Instrument Invoicing Environment

For lab/finance administrator to manage the expense codes per project/user and view invoices. Check here for more details about Fiine system and its use.

 

© The President and Fellows of Harvard College
Except where otherwise noted, this content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.