FASRC DOCS

Search Docs by Keyword

Table of Contents

Globus File Transfer

Globus1 is a service enabling file sharing with external collaborators without requiring them to have accounts on FAS Research Computing systems. A collaborator has to use their Globus service login and their Globus shared endpoint while FAS Research Computing user has to follow the steps described in this document to gain access to the Globus service.

Globus is a 3rd party service that FASRC and other universities use as a nexus to share and transfer data from/to their filesystems, it is not run by FASRC.

Using Globus Service To Transfer Data In or Share Data Out

  1. Familiarize yourself with the Globus How To2 documentation.
    Please note up front: If your data contains directories with hundreds or thousands of files, you will need to tar those directories up into subset files. (Type man tar at the command line to view the manual page for tar.)  Too many files in a single directory, while generally never a good idea, will cause Globus to go into a ‘endpoint is too busy’ state and your job will timeout, restart, timeout, etc.
    It is also not recommended to create a any single tar file larger than a few GB in size (and not larger than about 100GB). If your transfer restarts, it will restart at the beginning of that file.
  2. Login into the Globus web interface3, selecting Harvard University as your organization which will allow you to log in using Harvard Key, and land on File Manager page.4*
    If you are unable to complete this step, please contact HUIT to ensure your HarvardKey is current/enabled.
  3. FAS RC has two endpoints based on data centers and use-case
    1).  Holyoke endpoint is “Harvard FAS RC Holyoke” for Holyoke lab shares
    2). Boston endpoint is “Harvard FAS RC Boston” for Boston lab shares.
    Type one of these names based on the storage location. Click the endpoint and you will be asked to authenticate your FASRC account.
    Click Continue to do so.
  4. Enter your FAS RC Username and your FAS RC Verification Code5 (your OpenAuth toke code, not your password) when prompted to do so.

If you have difficulty connecting at any point in the process try again while on the RC VPN as certain steps require connectivity to our internal networks.

Globus Docs and Videos

Globus docs: http://www.globus.org | How to get started | Transfer and Sharing

Globus also provides helpful videos for common issues: https://www.globus.org/videos

Using the Harvard FAS RC Holyoke or Boston Endpoints

  • Please be aware that you can share out files or folders that you own or have explicit permission to. Please be careful about sharing too broadly.
  • To share out files in disparate directories, you should create a directory to copy those files to.
  •  The Holyoke and Boston endpoints are setup such that they will only see your lab folder’s “Lab” and “Users/$User” sub directories. The TRANSFER directory is no longer used. If you find that your lab is not seen by Globus or these sub directories don’t exist in your lab’s folder, in the Holyoke or Boston endpoints, please contact us.
  • Globus will use your RC account’s permissions the same as if you were accessing your lab storage from a node on the cluster. This dictates what Globus can see and not see when inside the Globus File manager, transferring data, or sharing data.

Sharing Data Examples

Example 1
To transfer files between the transfer folder and an external collaborator’s Globus shared endpoint, type the external collaborator’s shared endpoint name in the other Endpoint field on the File Manager page,4 select the files to transfer, specify the transfer settings and initiate the transfer by clicking on the large arrow button.6
Example 2
Share out a subfolder located in your transfer folder by creating a shared endpoint7 and authorizing external collaborator’s Globus account to access your shared endpoint. Let the collaborator know the shared endpoint’s name — the collaborator will then be able to initiate file transfers to/from your shared endpoint.
Example 3
To transfer files to/from a laptop or a desktop machine, set up a shared endpoint on it by installing Globus Connect Personal software8 and then connect to that shared endpoint the same way you would connect to an external collaborator’s shared endpoint — as described in the Example 1. Premium Globus account is not required to transfer files between an institutional (FAS RC) shared endpoint and the Globus Connect Personal software created endpoint.

Installing Globus Personal Endpoint on your computer

To share data from or to your local machine with other Globus users (not just with FASRC), you can install and run a personal endpoint on your computer.

Using Globus With Tier3 Tape

Please see https://docs.rc.fas.harvard.edu/kb/tape-globus-access/

Are there any limits on using the file transfer service?

See this Globus doc

Additional Documentation

  1. http://www.globus.org ↩
  2. http://docs.globus.org/how-to/get-started/ ↩
  3. https://docs.globus.org/faq/transfer-sharing/ ↩
  4. http://globus.org/login ↩
  5. http://docs.globus.org/how-to/get-started/#the_transfer_files_page ↩ ↩
  6. FAS RC Verification Code is the OpenAuth token generated code associated with your FAS RC user account. The same token is also used as the second factor for FAS RC VPN and FAS RC login nodes authentication. ↩
  7. http://docs.globus.org/how-to/get-started/#request_a_data_transfer ↩
  8. http://docs.globus.org/how-to/share-files/ ↩
  9. http://www.globus.org/globus-connect-personal ↩
Last Updated
  • © The President and Fellows of Harvard College
    Except where otherwise noted, this content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.