Search Docs by Keyword

Table of Contents

Parallel Computing

Description

Parallel computing is a computational approach that divides large problems into smaller tasks that can be executed simultaneously across multiple processing units. On the FASRC cluster, parallel computing enables researchers to solve complex computational problems that would be impractical or impossible on single-core systems by leveraging distributed memory architectures, shared memory systems, and specialized high-performance I/O libraries.

The FASRC cluster supports multiple parallel computing paradigms including:

  • Distributed Memory Computing: Message Passing Interface (MPI) for multi-node parallel processing
  • Shared Memory Computing: OpenMP for single-node multi-core parallelization and embarrassingly parallel job arrays
  • High-Performance I/O: Parallel HDF5, NetCDF, and ScaLAPACK for efficient data handling in large-scale computations

Usage

Parallel computing tools on the FASRC cluster are available through the module system. Load the appropriate modules based on your parallel computing paradigm.  Search for available modules with module spider.

Common Troubleshooting

  • Module Conflicts: Use module purge before loading new module combinations
  • MPI Version Mismatch: Ensure compilation and runtime use identical MPI implementations
  • Memory Issues: Monitor memory usage with sacct -j JOBID --format=MaxRSS
  • Performance Bottlenecks: Use profiling tools like Intel VTune or GNU gprof

Training

Examples

For an assortment of helpful examples used in our training videos, check out our User Codes/Parallel_Computing.

References

© The President and Fellows of Harvard College.
Except where otherwise noted, this content is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.