FASRC DOCS

Skip to main content

Search Docs by Keyword

Table of Contents

Installing Software

Overview

FASRC provides common baseline software packages via Operating System packages and our software modules system. These include compilers, licensed software, and commonly used libraries. However due to the host of different types of scientific software we cannot provide prebuilt software for every case.

To that end users will need to build and install software themselves. Users are free to install any and all software that is relevant to their research provided they comply with the Acceptable Use policy and other relevant university policies. Most software has installation guides you can follow to install, though you will want to be careful as many guides assume that the user has sudo privileges which is system administrator only. For these cases you will want to talk with the software provider about installation methods that do not involve sudo or privileged access. If privileged access is required, please submit a ticket to FASRC for assistance.

Below you will find several common approaches to compiling and installing software on the cluster. Note that these are general guides, you will want to consult the documentation for the specific software you want to install for more specific instructions.

If you are having problems during your installation, submit a ticket to rchelp@rc.fas.harvard.edu or stop by Office Hours.

Binaries

Binaries are precompiled executables that can be simply downloaded and run as is on the cluster. This saves the trouble of having to worry about compiling the software at all or including various dependencies. It should be noted that binaries are not truly portable to all systems, but rather must be built for the specific operating system (OS) and architecture that the computer is running. A Windows binary will not work on a Unix system and visa versa. Similarly a binary built for ARM will not work on x86. You will want to download the correct version for the system you are on. Some binaries can be very specific to the OS and specific processor type.

To run a binary simply download it to the cluster and set it to be executable:

chmod u+x someprogram.x
./someprogram.x

Note that the above chmod command would only make the executable work for the user in question. To add ability for the group or any one to access you would need to do g+x and o+x respectively. Since the program is not in your PATH you will need to refer to it by its relative or absolute path (see Making Software Available for how to add it to your PATH)

CMake

Cmake is a popular software compilation tool that allows the building of binaries and libraries for various operating systems and environments. Cmake works in a similar but notably different way to the GNU autotools build process. Of importance is to make sure that the version of Cmake you are using is up to date as newer versions have newer features which the software you are building may be leveraging. We provide prebuilt versions of Cmake as part of our module system and you can find what versions are available by doing:

$ module spider cmake

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  cmake: cmake/4.2.3-fasrc01
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      Cross platform build tool


    This module can be loaded directly: module load cmake/3.25.2-fasrc01

    Help:
      cmake-4.2.3-fasrc01
      Cross platform build tool

$ module load cmake/4.2.3-fasrc01

Once you have loaded the module you need for Cmake check the build/installation instructions for your code as how you use Cmake will depend on where the developer has put their Cmakelists.txt which specifies the build process. We will proceed assuming that they have placed it in the base directory of their package. If this is the case you will want to cd to that directory and mkdir build, this directory will serve as the build directory which Cmake will use for its intermediate build steps. You then:

cd build
cmake -DCMAKE_INSTALL_PREFIX=/n/holylabs/LABS/jharvard_lab/Lab/software ..
make
This will run Cmake and specify a location you want to install the resulting software to. Cmake when its done configuring then generates a
make install

It should be noted that Cmake sometimes does not respect the CMAKE_INSTALL_PREFIX flag. In this case you will need to copy the created libraries by hand, you can find the files that Cmake made in the build directory.

GNU autotools

Many apps are provided in source code form and require conversion to binary form — a process called compilation or building. Such apps are often packaged to work with the GNU toolchain (aka autotools) — you use the commands ./configure, make, and make install to build and install them. The important thing is that you specify a –prefix in order to write the software to a non-default location, i.e. a location writable by you. The examples below install software to a directory /n/holylabs/LABS/jharvard_lab/LAB/software/. Before looking at the instructions below consult the installation instructions for the software you are looking at as they will have more details about code features.

Software source is often distributed as a zipped tarball, such as APP-X.Y.Z.tar.gz. The first step is to unpack it. This can be done in whatever directory you like, even a temporary space that you later delete, since the installation step below will copy stuff out of this particular directory and into the prefix you specify:

tar -zxvf APP-X.Y.Z.tar.gz

Next, cd into the unpacked directory:

cd APP-X.Y.Z

Often the newly created directory won’t exactly match the tarball, or the files will have been dumped to the current working directory; adjust accordingly.

Next, configure the software. You will want to run ./configure --help to see if there are any addition options you want to enable. An option you will want to set is the --prefix flag telling it to install to somewhere in your lab directory:

./configure --prefix=/n/holylabs/LABS/jharvard_lab/LAB/software/

If you want to install every app in its own directory, instead of having them all share one (e.g. if you want to have different versions of the same software available), use a more specific prefix, like --prefix=/n/holylabs/LABS/jharvard_lab/LAB/software/APP-X.Y.Z

Some apps don’t actually use the full GNU toolchain, and don’t have a configure script; they have just a Makefile. In that case, you’ll have to look at the Makefile to see if it has an adjustable installation location. If not, you’ll have to manually install files.

Next, build the software:

make

This is the step that actually compiles the software. Once you work through any issues that may come up and the software compiles successfully, install it:

make install

If you get any Permission denied issues at this point, it’s possible the software did not properly use the --prefix you have it. Likewise, you may get the error No rule to make target `install'. In fact, many packages only mimic the GNU toolchain style but actually work slightly differently. In such cases, you’ll have to manually copy the build outputs to their destination.

Making Software Available

Once you have built the software you want to use you need to make it available for use. The shell environment relies on environment variables that name where apps and libraries are located. When you install software to non-default locations, you will need to update these variables. Without doing so, you will get errors like command not found, or error while loading shared libraries.

For example, the environment variable PATH names directories in which the apps you invoke on the command line reside. Conventionally, when you install software under a --prefix, the apps are in a sub-directory of that prefix named bin. Similarly, the variable LD_LIBRARY_PATH controls what libraries are available for dynamic dependency loading, and those libraries are often put in a directory named lib. If you installed your software to /n/holylabs/jharvard_lab/Lab/software the variables you will need will likely be:

export PATH="/n/holylabs/jharvard_lab/Lab/software/bin:$PATH"
export LD_LIBRARY_PATH="/n/holylabs/jharvard_lab/Lab/software/lib:$LD_LIBRARY_PATH"
export LIBRARY_PATH="/n/holylabs/jharvard_lab/Lab/software/lib:$LIBRARY_PATH"
export PKG_CONFIG_PATH="/n/holylabs/jharvard_lab/Lab/software/lib/pkgconfig:$PKG_CONFIG_PATH"
export CPATH="/n/holylabs/jharvard_lab/Lab/software/include:$CPATH"
export FPATH="/n/holylabs/jharvard_lab/Lab/software/include:$FPATH"
export PYTHONPATH="/n/holylabs/jharvard_lab/Lab/software/lib/python3.12/site-packages:$PYTHONPATH"
export MANPATH="/n/holylabs/jharvard_lab/Lab/software/share/man:$MANPATH"

Once you find all of these you can copy these variable definitions to the end of your ~/.bashrc file, and then the software will be available by default. Be sure to include the :$VARIABLE at the end else you will reset the variable rather than prepending. The operating system will search the list of folders defined in the variable in order to find the package or library it is looking for. Alternatively, you can save this output to a file such as ~/sw/setup.sh, and then every time you want to use this software, you run:

source ~/sw/setup.sh

This is necessary if you want to install multiple incompatible apps, or different versions of the same app, and pick and choose between them at will (you’ll have to use a different --prefix for each). You may also consider writing your own lmod modulefiles so that you can integrate your own software with our modules system.

Last Updated
Tags:
  • © The President and Fellows of Harvard College.
    Except where otherwise noted, this content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.