MPI support in Abinit 6

Motivations

The Abinit 5 mutation took place under the constraint of keeping backward-compatibility of the build system with Abinit 4 as long as possible. While this has permitted a smoother transition to an improved development model, it has become less and less adequate with time to deal with the emerging needs of the developers. In particular, the necessity of having both a serial and a parallel version of the abinit main program built at once has been a source of endless issues, finally preventing further extension of the build system.

The Abinit developers are also now sufficiently familiar with the new build system to be willing to customize it themselves. This in turn has promoted the need to simplify the user interface of the build system to a higher priority. Quickly evolving towards a situation where only one version of abinit is built at a time has thus become essential.

Removing the support for multiparameter builds allows for:

• a drammatic overall simplication of the build system;
• an abrupt decrease in the number of MPI-related options;
• MPI build flags being included in config.h;
• the easy propagation of MPI flags to the plugins;
• the straightforward success of parallel builds (e.g. make -j4);
• running the full test suite on any variant of abinit;
• the automatic generation of configure.ac;
• the design of a test suite for the build system;
• a substantial improvement in the build-system extensibility.

Caveats

Properly setting environment variables

A noticeable change in the behaviour of the build system is that it will not try anymore to set LD_LIBRARY_PATH to have the code properly built with MPI support. It is the responsibility of the user to take care of this aspect, by making sure that the LD_LIBRARY_PATH environment variable actually points to the <mpi_prefix>/lib directory of the MPI installation they want to use.

In case of an incorrectly set build environment, the configure script will exit with a similar error message:

  checking for gcc... /home/pouillon/hpc/openmpi-1.2.4-gcc4.1/bin/mpicc
  checking for C compiler default output file name... a.out
  checking whether the C compiler works... configure: error: cannot run C compiled programs.
  If you meant to cross compile, use ‘--host’.
  See ‘config.log’ for more details.

Building Abinit in parallel (make -j)

The core of Abinit now builds satisfactorily when running a parallel make (e.g. "make -j8"). Successful tests have been conducted up to 20 processors so far. However, depending on your MPI installation, the build of the plugins might crash randomly. On dual- and quad-core machines, you may type "make mj4" instead, which will take care of building the plugins serially.

As there is virtually an infinite number of ways to install MPI on a machine, it may happen in some situations that the parallel build of the core fails as well, the compiler complaining about a "mpif.h" not found. This is a design flaw of some MPI implementations. In most cases, it is sufficient to use:

--with-mpi-includes="-I/path/to/mpi/include"

at configure time. A better solution is however to install a MPI-2-compliant implementation of MPI, such as OpenMPI or MPICH2.

Configure options

--enable-mpi

From now on the build system will only check whether the compilers are natively MPI-capable. When this option is not specified, MPI support is activated only when the Fortran compiler is able to handle MPI directives. The purpose of this option is just to force the support to be on or off.

New options

Two new options have been introduced in order to comply with a support of MPI as an external library:

• --with-mpi-includes
• --with-mpi-libs

Unchanged options

Of all the other options, the only remaining ones are:

• --enable-mpi-io
• --enable-mpi-trace
• --with-mpi-level
• --with-mpi-runner

Their meaning did not change, and they can just be used as before.

Removed options

Some options that were intensely used have been removed. The configure script will now exit with an error if you try to use them. Please also note that setting "FC=/path/to/mpi-capable-f90-compiler" is now forbidden when using the "--with-mpi-prefix=..." option. Please note that the LD_LIBRARY_PATH environment variable must point to the directory containing the MPI libraries for the configure script to work.

For parallel I/O, the --enable-mpi-io-buggy option has been replaced by a test in the configure script. Please note that this feature is still experimental. You are kindly invited to check the output of configure within the "Connectors" section and to have a look at the config.log file (look for the word "full" and the lines below).

Preprocessing options

Since MPI is now handled as all other external libraries by the build system, the associated preprocessing options have been renamed:

• from: MPI... to: HAVE_MPI...

This is necessary for a unified treatment of external libraries by the build system, which is generating many files the manual handling would be quite bothersome.

Please note that comments should be avoided after '#endif' statements. If you need however to put coments at some places, please use strict C comments exclusively, i.e. '#endif /* This is a comment. */'. Fortran comments on such lines are strictly forbidden, has they are read by the C preprocessor.

Using different builds of abinit at once

If you want to be able to use different builds of abinit at the same time, you just need to use several build directories and configure the build differently within each of them.

For instance:

  cd abinit-5.8.0
  mkdir tmp-serial
  cd tmp-serial
  ../configure \
    --program-suffix="-seq" \
    --disable-mpi
  make
  cd ..
  mkdir tmp-scalapack
  cd tmp-scalapack
  ../configure \
    --program-suffix="-mpi" \
    --enable-mpi \
    --enable-scalapack \
    --with-scalapack-libs="-lscalapack -lblacs" \
    FC=mpif90
  make

will build both a serial and a ScaLAPACK-enabled parallel version of abinit, adding corresponding suffixes when running "make install", in order to avoid name clashes.

What is particularly interesting with this new strategy is that the full test suite can now be run on binaries built from any combination of configure options, without having to do anything on the test suite side.