MPI IO

Introduction

MPI IO is a component of the Message Passing Interface standard that enables parallel input/output operations in distributed computing applications. It allows multiple processes to collectively access shared files, optimizing disk operations through features like collective I/O and file views that reorganize data access patterns. This capability is crucial for high-performance computing applications working with large datasets across multiple nodes, as it extends parallel processing efficiency to the I/O subsystem.

Installation

Utilizing MPI IO in your program depends on your programming language. Check your programming language’s documentation for details.

Using MPI I/O

Your programming language of choice should have an implementation for these basic functions from the complete list of functions. Each of the functions has a C and Fortran usage example in the docs, as well as some parameter information and common errors.

• MPI_Init(): Initialize the MPI environment
• MPI_File_open(): Create file handles for parallel access
• MPI_File_set_view(): Define which file portions each process accesses
• MPI_File_read_all()/MPI_File_write_all(): Perform collective I/O operations
• MPI_File_read()/MPI_File_write(): Perform independent I/O operations
• MPI_File_close(): Close file handles
• MPI_Finalize(): Clean up MPI environment

Best Practices

On Cannon <insert storage best practices for high IO jobs >.

Slurm < insert best practices for CPU and GPU >.

Examples

For associated MPI IO examples, head over to  User_Codes/Parallel_Computing/MPI_IO.

Resources

• Open-MPI.org documentation site
• OpenMPI MPI IO ( “MPIO” )