Some common single-node multi-threaded jobs:

  • programs that use multi-threaded linear algebra libraries (MKL, BLAS, ATLAS, etc.)
    • R on our cluster can use multiple threads for algebra if you set the environment variable export OMP_NUM_THREADS=8 (or whatever other value).
    • In case you are not sure if your program is using multi-threaded linear algebra libraries, then execute ldd <program path>. The multi-threaded programs will have at least one of the following listed (or very similar): libpthread, libblas, libgomp, libmkl, libatlas.
  • parallel make i.e. make -j (e.g. EPACTS)
  • programs that use OpenMP
  • programs that use pthreads

 If a job uses multiple threads, but you don't tell SLURM, SLURM will allocate too many jobs to that node. That will cause problems for all jobs on that node. Don't do that.

SLURM options for multi-threaded programs:

  • --cpus-per-task: the number of cores your job will use (default is 1)
  • all the options listed above.

eg, sbatch --cpus-per-task=8 --mem=32G myjob.sh will allocate 8 CPUs (cores) on a single node with 32GB of RAM (4GB per thread) for a program named myjob.sh.

Making your job use the correct number of threads:

  • When using multi-threaded linear algebra libraries, you may need to additionally restrict the number of threads using environment variables such as OMP_NUM_THREADS. Please, spend some time reading documentation of the specific library you are using to understand what environment variables need to be changed.
  • When using parallel make, set make -j <number_of_cores>.

After job finished, you can using seff <job_id> to check whether your program can make benefit from multithreads