Globus File Transfer
- 1 Using Globus Service To Transfer Data In or Share Data Out
- 1.1 Globus Docs and Videos
- 1.2 Using the Harvard FAS RC Holyoke or Boston Endpoints
- 1.3 Sharing Data Examples
- 1.4 Installing Globus Personal Endpoint on your computer
- 1.5 Using Globus With Tier3 Tape
- 1.6 Are there any limits on using the file transfer service?
- 1.7 Additional Documentation
Globus1 is a service enabling file sharing with external collaborators without requiring them to have accounts on FAS Research Computing systems. A collaborator has to use their Globus service login and their Globus shared endpoint while FAS Research Computing user has to follow the steps described in this document to gain access to the Globus service.
Globus is a 3rd party service that FASRC and other universities use as a nexus to share and transfer data from/to their filesystems, it is not run by FASRC.
- Familiarize yourself with the Globus How To2 documentation.
Please note up front: If your data contains directories with hundreds or thousands of files, you will need to tar those directories up into subset files. (Type
man tarat the command line to view the manual page for tar.) Too many files in a single directory, while generally never a good idea, will cause Globus to go into a ‘endpoint is too busy’ state and your job will timeout, restart, timeout, etc.
It is also not recommended to create a any single tar file larger than a few GB in size (and not larger than about 100GB). If your transfer restarts, it will restart at the beginning of that file.
- Login into the Globus web interface3, selecting Harvard University as your organization which will allow you to log in using Harvard Key, and land on File Manager page.4*
If you are unable to complete this step, please contact HUIT to ensure your HarvardKey is current/enabled.
- FAS RC has two endpoints based on data centers and use-case
1). Holyoke endpoint is “Harvard FAS RC Holyoke” for Holyoke lab shares
2). Boston endpoint is “Harvard FAS RC Boston” for Boston lab shares.
Type one of these names based on the storage location. Click the endpoint and you will be asked to authenticate your FASRC account.
Click Continue to do so.
- Enter your FAS RC Username and your FAS RC Verification Code5 (your OpenAuth toke code, not your password) when prompted to do so.
If you have difficulty connecting at any point in the process try again while on the RC VPN as certain steps require connectivity to our internal networks.
Globus Docs and Videos
Globus also provides helpful videos for common issues: https://www.globus.org/videos
Using the Harvard FAS RC Holyoke or Boston Endpoints
- Please be aware that you can share out files or folders that you own or have explicit permission to. Please be careful about sharing too broadly.
- To share out files in disparate directories, you should create a directory to copy those files to.
- The Holyoke and Boston endpoints are setup such that they will only see your lab folder’s “Lab” and “Users/$User” sub directories. The TRANSFER directory is no longer used. If you find that your lab is not seen by Globus or these sub directories don’t exist in your lab’s folder, in the Holyoke or Boston endpoints, please contact us.
- Globus will use your RC account’s permissions the same as if you were accessing your lab storage from a node on the cluster. This dictates what Globus can see and not see when inside the Globus File manager, transferring data, or sharing data.
Sharing Data Examples
- To transfer files between the transfer folder and an external collaborator’s Globus shared endpoint, type the external collaborator’s shared endpoint name in the other Endpoint field on the File Manager page,4 select the files to transfer, specify the transfer settings and initiate the transfer by clicking on the large arrow button.6
- Share out a subfolder located in your transfer folder by creating a shared endpoint7 and authorizing external collaborator’s Globus account to access your shared endpoint. Let the collaborator know the shared endpoint’s name — the collaborator will then be able to initiate file transfers to/from your shared endpoint.
- To transfer files to/from a laptop or a desktop machine, set up a shared endpoint on it by installing Globus Connect Personal software8 and then connect to that shared endpoint the same way you would connect to an external collaborator’s shared endpoint — as described in the Example 1. Premium Globus account is not required to transfer files between an institutional (FAS RC) shared endpoint and the Globus Connect Personal software created endpoint.
Installing Globus Personal Endpoint on your computer
To share data from or to your local machine with other Globus users (not just with FASRC), you can install and run a personal endpoint on your computer.
- Install, Configure, and Uninstall Globus Connect Personal for Mac OS X
- Install, Configure, and Uninstall Globus Connect Personal for Windows
- Install, Configure, and Uninstall Globus Connect Personal for Linux
Using Globus With Tier3 Tape
(more info coming soon)
Are there any limits on using the file transfer service?
- http://www.globus.org ↩
- http://docs.globus.org/how-to/get-started/ ↩
- https://docs.globus.org/faq/transfer-sharing/ ↩
- http://globus.org/login ↩
- http://docs.globus.org/how-to/get-started/#the_transfer_files_page ↩ ↩
- FAS RC Verification Code is the OpenAuth token generated code associated with your FAS RC user account. The same token is also used as the second factor for FAS RC VPN and FAS RC login nodes authentication. ↩
- http://docs.globus.org/how-to/get-started/#request_a_data_transfer ↩
- http://docs.globus.org/how-to/share-files/ ↩
- http://www.globus.org/globus-connect-personal ↩