Introduction

The Message Passing Interface (MPI) library allows processes in your parallel application to communicate with one another by sending and receiving messages. There is no default MPI library in your environment when you log in to the cluster. You need to choose the desired MPI implementation for your applications. This is done by loading an appropriate MPI module. Currently the available MPI implementations on our cluster are OpenMPI and Mpich. For both implementations the MPI libraries are compiled and built with either the Intel compiler suite or the GNU compiler suite. These are organized in software modules.

Installation

MPI has many forms, we’ll list a few here, and also look at User_Codes/Parallel_Computing/MPI.

mpi4py with Python

To use mpi4py you need to load an appropriate Python software module. We have the Anaconda Python distribution from Continuum Analytics. In addition to mpi4py, it includes hundreds of the most popular packages for large-scale data processing and scientific computing.

You can load python in your user environment by running in your terminal:

module load python/2.7.14-fasrc01

For example code, see Parallel_Computing/Python/mpi4py

OpenMPI with GNU Compiler

If you want to use OpenMPI compiled with the GNU compiler you need to load appropriate compiler and MPI modules. Below are some possible combinations, check module spider MODULENAME to get a full listing of possibilities.

# GCC + OpenMPI, e.g.,
module load gcc/13.2.0-fasrc01 openmpi/5.0.2-fasrc01

# GCC + Mpich, e.g.,
module load gcc/13.2.0-fasrc01 mpich/4.2.0-fasrc01

# Intel + OpenMPI, e.g.,
module load intel/24.0.1-fasrc01 openmpi/5.0.2-fasrc01

# Intel + Mpich, e.g.,
module load intel/24.0.1-fasrc01 mpich/4.2.0-fasrc01

# Intel + IntelMPI (IntelMPI runs mpich underneath), e.g.
module load intel/24.0.1-fasrc01 intelmpi/2021.11-fasrc01

For reproducibility and consistency it is recommended to use the complete module name with the module load command, as illustrated above. Modules on the cluster get updated often so check if there are more recent ones. The modules are set up so that you can only have one MPI module loaded at a time. If you try loading a second one it will automatically unload the first. This is done to avoid dependencies collisions.

There are four ways you can set up your MPI on the cluster:

• Put the module load command in your startup files.
  Most users will find this option most convenient. You will likely only want to use a single version of MPI for all your work. This method also works with all MPI modules currently available on the cluster.

• Load the module in your current shell.
  For the current MPI versions you do not need to have the module load command in your startup files. If you submit a job the remote processes will inherit the submission shell environment and use the proper MPI library. Note this method does not work with older versions of MPI.

• Load the module in your job script.
  If you will be using different versions of MPI for different jobs, then you can put the module load command in your script. You need to ensure your script can execute the module load command properly.

• Do not use modules and set environment variables yourself.
  You obviously do not need to use modules but can hard code paths. However, these locations may change without warning so you should set them in one location only and not scatter them throughout your scripts. This option could be useful if you have a customized local build of MPI you would like to use with your applications.

Video Training

Examples

Resources