Installing Software Yourself

Installing Software Yourself

We have high-performance, scientific computing applications installed and available through our software module system. These include compilers, licensed software, and commonly used libraries.

If you are looking to use a “bleeding edge” version of software or something specific only to your lab, you can install applications yourself in your home directory, any other space to which you can write, or a shared lab directory. Writing and installing your own software may be part of your research, too. Below you will find several approaches to compiling and installing personal software on the cluster. If you are having problems during your installation, submit a help request on the RC Portal or stop by Office Hours.

• Installing binaries
• Installing GNU-toolchain-style apps (configure – make – make install)
• Cmake
• Spack

Making Software Available

Once you have built the software you want to use you need to make it available for use. The shell environment relies on environment variables that name where apps and libraries are located. When you install software to non-default locations, you will need to update these variables. Without doing so, you will get errors like command not found, or error while loading shared libraries. Note that if you are using Spack, Python, R, or Julia you will want to follow the guides for those codes as they have their own ways of handling software.

For example, the environment variable PATH names directories in which the apps you invoke on the command line reside. Conventionally, when you install software under a --prefix, the apps are in a sub-directory of that prefix named bin. Similarly, the variable LD_LIBRARY_PATH controls what libraries are available for dynamic dependency loading, and those libraries are often put in a directory named lib. If you installed your software to /n/holylabs/LABS/jharvard_lab/Lab/software the variables you will need will likely be:

export PATH="/n/holylabs/LABS/jharvard_lab/Lab/software/bin:$PATH"
export LD_LIBRARY_PATH="/n/holylabs/LABS/jharvard_lab/Lab/software/lib:$LD_LIBRARY_PATH"
export LIBRARY_PATH="/n/holylabs/LABS/jharvard_lab/Lab/software/lib:$LIBRARY_PATH"
export PKG_CONFIG_PATH="/n/holylabs/LABS/jharvard_lab/Lab/software/lib/pkgconfig:$PKG_CONFIG_PATH"
export CPATH="/n/holylabs/LABS/jharvard_lab/Lab/software/include:$CPATH"
export FPATH="/n/holylabs/LABS/jharvard_lab/Lab/software/include:$FPATH"
export PYTHONPATH="/n/holylabs/LABS/jharvard_lab/Lab/software/lib/python2.7/site-packages:$PYTHONPATH"
export MANPATH="/n/holylabs/LABS/jharvard_lab/Lab/software/share/man:$MANPATH"

Once you find all of these you can copy these variable definitions to the end of your ~/.bashrc file, and then the software will be available by default. Be sure to include the :$VARIABLE at the end else you will reset the variable rather than prepending. The operating system will search the list of folders defined in the variable in order to find the package or library it is looking for. Alternatively, you can save this output to a file such as ~/sw/setup.sh, and then every time you want to use this software, you run:

source ~/sw/setup.sh

This is necessary if you want to install multiple incompatible apps, or different versions of the same app, and pick and choose between them at will (you’ll have to use a different --prefix for each). You may also consider writing your own lmod modulefiles so that you can integrate your own software with our modules system.

See more on: https://github.com/fasrc/User_Codes/tree/master/Documents/Software