OpenMP (Open Multi-Processing) is a standard for building parallel applications on shared-memory computers (multiprocessors). It consists primarily of a set of compiler directives, with some library routines besides.
The maximum number of threads available for OpenMP jobs on our clusters is either 4 or 16 depending on the execution host upon which the job is running. The number of threads is communicated to an OpenMP program at execution time with the environment variable
OMP_NUM_THREADS via the Grid Engine
$NSLOTS variable. Here is the line you need to add to your submissing script for an OpenMP program if your execution shell is
setenv OMP_NUM_THREADS $NSLOTS
In order to force the compiler to interpret the OpenMP directives in the source code, you need to specify appropriate flags during the compilation, otherwise a serial code will be generated.
The good introduction to OpenMP can be found here.
Thread stack size
- See: Memory Management
Hello world program
Here is an example of the simple OpenMP program written in Fortran 90, along with the instructions on how to compile, run interactively and submit this program to the cluster.
program hello implicit none include 'omp_lib.h' integer :: my_rank = 0, p = 1 !$omp parallel private( my_rank ) !$omp single write(*,'("Hello parallel world!")') !$ p = omp_get_num_threads() write(*,'("Number of threads is ", i3)') p !$omp end single !$ my_rank = omp_get_thread_num() write(*,'("Hello world from thread", i3)') my_rank !$omp end parallel write(*,'("Back to the sequential world.")') end program hello
And this is how you compile the code and run it interactively on the head node in
$ pgf90 -mp -o hello hello.f90 $ export OMP_NUM_THREADS=4 $ ./hello Hello parallel world! Number of threads is 4 Hello world from thread 3 Hello world from thread 0 Hello world from thread 1 Hello world from thread 2 Back to the sequential world. $
To submit the code
hello to the scheduler, you need to create a small submission script. With this script you instruct the scheduler where to execute the code, where to write the output, and with how many threads you want you code to be run. Here is an example of such a script called
#$ -cwd #$ -l h_rt=01:00:00 #$ -pe openmp 4 export OMP_NUM_THREADS=$NSLOTS ./hello
Finally, to submit the job, type in the command line
$ qsub submit_hello.sh