ScaLAPACK¶

ScaLAPACK (Scalable LAPACK) provides parallel distributed-memory routines for dense and banded linear algebra: linear systems, least squares, eigenvalue, and singular value problems.

Loading the module¶

module load ScaLAPACK/2.2.0-gompi-2023a-fb

To see all installed versions across clusters, check the software catalog or run module spider ScaLAPACK.

Info

ScaLAPACK is loaded automatically as part of the foss toolchain. With Intel toolchains, the equivalent functionality is provided by MKL (imkl). Check module list before adding a separate load.

Compiling against ScaLAPACK¶

ScaLAPACK requires MPI. Compile with the MPI wrappers and link -lscalapack:

# Fortran
mpif90 -o example example.f90 -lscalapack

# C
mpicc -o example example.c -lscalapack -llapack -lblas -lm

The module provides the BLACS communication layer automatically.

Example: solving Ax=b with PDGESV (Fortran)

This is a minimal skeleton. For a complete working example including matrix distribution and the MATINIT subroutine, see the ScaLAPACK example programs.

program scalapack_solve
  implicit none
  integer :: ictxt, nprow, npcol, myrow, mycol, info
  integer :: n, nrhs, nb, desca(9), descb(9)
  integer, allocatable :: ipiv(:)
  double precision, allocatable :: a(:,:), b(:,:)

  ! Initialize a 2x3 process grid
  nprow = 2; npcol = 3
  call sl_init(ictxt, nprow, npcol)
  call blacs_gridinfo(ictxt, nprow, npcol, myrow, mycol)
  if (myrow == -1) stop

  ! ... set up descriptors with descinit, distribute A and B ...

  call pdgesv(n, nrhs, a, 1, 1, desca, ipiv, b, 1, 1, descb, info)

  call blacs_gridexit(ictxt)
  call blacs_exit(0)
end program

Batch job example¶

ScaLAPACK programs run as MPI jobs. The total number of MPI ranks must match your BLACS process grid (e.g., 2×3 = 6 ranks):

scalapack.slurm

#!/bin/bash
#SBATCH --qos=regular
#SBATCH --job-name=scalapack
#SBATCH --nodes=1
#SBATCH --ntasks=6
#SBATCH --cpus-per-task=1
#SBATCH --mem=8G
#SBATCH --time=00:30:00
#SBATCH --output=%x.%j.out
#SBATCH --error=%x.%j.err

module load ScaLAPACK/2.2.0-gompi-2023a-fb

srun ./example

Process grid must match MPI ranks

The number of MPI ranks must equal NPROW × NPCOL defined in your code. A mismatch causes silent failures or crashes.

ScaLAPACK¶

Loading the module¶

Compiling against ScaLAPACK¶

Batch job example¶

See also¶