Open MPI

MPI (Message Passing Interface) is a standard application programming interface (API) for efficient interprocess communication. It is widely used in many existing research codes. It can be used to write parallel programs which span multiple computers or which reside on a single multiprocessor.

Open MPI

The default (and preferred) MPI implementation at ACEnet is Open MPI. It's free, open source, production-quality MPI-2 implementation.

Note: Do not confuse the Open MPI library with the OpenMP interface.

Resources

Open MPI web site

Frequently Asked Questions

Instructional videos and presentations

How it's spelled

It's two words (not one) — Open MPI.

It's also frequently abbreviated "OMPI" (pronounced "oom-pee").

Modulefile

openmpi

Versions

Several versions available.

Default is 1.2.9 built with PGI.

Benefits of Open MPI

• Supports all available types of interconnections (Ethernet, Myrinet, Infiniband, on-host shared memory, etc.).
• The fastest available interconnection will be used by default.
• Better performance, especially on large shared-memory nodes.
• No need to recompile your application on every cluster.
• Better integration with Sun Grid Engine.
• Better integration with parallel debuggers (see TotalView).
• User friendly, easy to use.

Commands

You can use the following standard commands:

 $ mpif90, mpif77, mpicc, mpiCC (or mpicxx or mpic++), mpirun

Getting help in command line

You can also browse man pages for the main Open MPI commands, for example:

 man mpif90
 man mpicc

Also you can get description and syntax for many MPI functions, for example:

 man MPI_Init
 man MPI_Send

Running jobs

You should use the ompi* parallel environment for Open MPI jobs.

Production jobs

Below is an example of a submission script for an MPI job. There is no need to specify the list of hosts and the number of processes for the mpirun command because Open MPI will obtain this information directly from Grid Engine.

  #$ -cwd
  #$ -pe ompi* 4
  #$ -l h_rt=01:00:00

  mpirun my_parallel_application

Interactive jobs

Below is the short reference on how to run MPI job interactively. For a complete description, please read Interactive Jobs.

On the head node (very briefly)

$ mpirun -np 2 my_parallel_application

Interactive session through Grid Engine

$ qrsh -cwd -V -l h_rt=00:10:00,test=true -pe "ompi*" 4 mpirun myapplication

If you prefer to get an interactive login shell and then run the job manually, you just need to specify the name of your favorite shell as a job name, like so

$ qrsh -cwd -V -pe "ompi*" 8 -l h_rt=00:30:00 bash

Alternative builds of Open MPI

Some users may want to use alternative builds of Open MPI to compile their applications. ACEnet provides several versions of Open MPI built with PGI, Intel and GCC compilers. In order to use them, you should load a specific modulefile. All Open MPI modulefiles are mutually exclusive. In order to minimize user's confusion and prevent version conflict, we suggest users to load an Open MPI modulefile interactively before compiling an application and then again in the submission script to run it.