next up previous contents
Next: 3 Parallelism Up: 2 Installation Previous: 2.6 Running tests and   Contents

Subsections

2.7 Installation tricks and problems

2.7.1 All architectures

2.7.2 Intel Xeon Phi

For Intel Xeon CPUs with Phi coprocessor, there are three ways of compiling: "You can take advantage of the offload mode using the libxphi library. This library offloads the BLAS/MKL functions on the Xeon Phi platform hiding the latency times due to the communication. You just need to compile this library and then to link it dynamically. The library works with any version of QE. Libxphi is available from https://github.com/cdahnken/libxphi. Some documentation is available therein.

Instead, if you want to compile a native version of QE, you just need to add the -mmic flag and cross compile. If you want to use the symmetric mode, you need to compile twice: with and without the -mmic flag". "[...] everything, i.e. code+libraries, must be cross-compiled with the -mmic flag. In my opinion, it's pretty unlikely that native mode can outperform the execution on the standard Xeon cpu. I strongly suggest to use the Xeon Phi in offload mode, for now" (info by Fabio Affinito, March 2015).

2.7.3 Cray machines

For Cray XE machines:

$ module swap PrgEnv-cray PrgEnv-pgi
$ ./configure --enable-openmp --enable-parallel --with-scalapack
$ vim make.inc
then manually add -D__IOTK_WORKAROUND1 at the end of DFLAGS line.

''Now, despite what people can imagine, every CRAY machine deployed can have different environment. For example on the machine I usually use for tests [...] I do have to unload some modules to make QE running properly. On another CRAY [...] there is also Intel compiler as option and the system is slightly different compared to the other. So my recipe should work, 99% of the cases. I strongly suggest you to use PGI, also for a performance point of view.'' (Info by Filippo Spiga, Sept. 2012)

For Cray XT machines, use ./configure ARCH=crayxt4 or else configure will not recognize the Cray-specific software environment.

Older Cray machines: T3D, T3E, X1, are no longer supported.

2.7.4 IBM AIX

As of v.6.0 IBM machines with AIX are no longer supported.

2.7.5 IBM BlueGene

The current configure is tested and works on the machines at CINECA and at Jülich. For other sites, you may need something like

  ./configure ARCH=ppc64-bg BLAS_LIBS=...  LAPACK_LIBS=... \
              SCALAPACK_DIR=... BLACS_DIR=..."
where the various *_LIBS and *_DIR "suggest" where the various libraries are located.

2.7.6 Linux PC

Both AMD and Intel CPUs, 32-bit and 64-bit, are supported and work, either in 32-bit emulation and in 64-bit mode. 64-bit executables can address a much larger memory space than 32-bit executable, but there is no gain in speed. Beware: the default integer type for 64-bit machine is typically 32-bit long. You should be able to use 64-bit integers as well, but it is not guaranteed to work and will not give any advantage anyway.

Currently, configure supports Intel (ifort), NAG (nagfor), PGI (pgf90) and gfortran compilers. Support for other compilers: Pathscale (pathf95), Sun Studio (sunf95), AMD Open64 (openf95), added in the past, is still there, but it is likely obsolete. Both Intel MKL and AMD acml mathematical libraries are supported, the former much better than the latter.

It is usually convenient to create semi-statically linked executables (with only libc, libm, libpthread dynamically linked). If you want to produce a binary that runs on different machines, compile it on the oldest machine you have (i.e. the one with the oldest version of the operating system).

2.7.6.1 Linux PCs with gfortran

You need at least gfortran v.4.4 or later to properly compile QUANTUM ESPRESSO.

"There is a known incompatibility problem between the calling convention for Fortran functions that return complex values: there is the convention used by g77/f2c, where in practice the compiler converts such functions to subroutines with a further parameter for the return value; gfortran instead produces a normal function returning a complex value. If your system libraries were compiled using g77 (which may happen for system-provided libraries in not-too-recent Linux distributions), and you instead use gfortran to compile QUANTUM ESPRESSO, your code may crash or produce random results. This typically happens during calls to zdotc, which is one the most commonly used complex-returning functions of BLAS+LAPACK.

For further details see for instance this link:
http://www.macresearch.org/lapackblas-fortran-106#comment-17071
or read the man page of gfortran under the flag -ff2c.

If your code crashes during a call to zdotc, try to recompile QUANTUM ESPRESSO using the internal BLAS and LAPACK routines (using the -with-internal-blas and -with-internal-lapack parameters of the configure script) to see if the problem disappears; or, add the -ff2c flag" (info by Giovanni Pizzi, Jan. 2013).

Note that a similar problem with complex functions exists with MKL libraries as well: if you compile with gfortran, link -lmkl_gf_lp64, not -lmkl_intel_lp64, and the like for other architectures. Since v.5.1, you may use the following workaround: add preprocessing option -Dzdotc=zdotc_wrapper to DFLAGS.

2.7.6.2 Linux PCs with g95

Since v.6.1, g95 is no longer supported.

2.7.6.3 Linux PCs with Pathscale compiler

Version 3.1 and version 4 (open source!) of the Pathscale EKO compiler work (info by Cezary Sliwa, April 2011, and Carlo Nervi, June 2011). In case of mysterious errors while compiling iotk, remove all lines like:

# 1 "iotk_base.spp"
from all iotk source files.

2.7.6.4 Linux PCs with Sun Studio compiler

``The Sun Studio compiler, sunf95, is free (web site: http://developers.sun.com/sunstudio/ and comes with a set of algebra libraries that can be used in place of the slow built-in libraries. It also supports OpenMP, which g95 does not. On the other hand, it is a pain to compile MPI with it. Furthermore the most recent version has a terrible bug that totally miscompiles the iotk input/output library (you'll have to compile it with reduced optimization).'' (info by Lorenzo Paulatto, March 2010).

2.7.6.5 Linux PCs with AMD Open64 suite

The AMD Open64 compiler suite, openf95 (web site: http://developer.amd.com/cpu/open64/pages/default.aspx) can be freely downloaded from the AMD site. It is recognized by configure but little tested. It sort of works but it fails to pass several tests (info by Paolo Giannozzi, March 2010). "I have configured for Pathscale, then switched to the Open64 compiler by editing make.inc. "make pw" succeeded and pw.x did process my file, but with "make all" I get an internal compiler error [in CPV/wf.f90]" (info by Cezary Sliwa, April 2011).

2.7.6.6 Linux PCs with Intel compiler (ifort)

The Intel compiler, ifort, is available for free for personal usage (http://software.intel.com/). It produces fast executables, at least on Intel CPUs, but not all versions work as expected (see below). In case of trouble, update your version with the most recent patches, available via Intel Premier support (registration free of charge for Linux): http://software.intel.com/en-us/articles/intel-software-developer-support. Since each major release of ifort differs a lot from the previous one, compiled objects from different releases may be incompatible and should not be mixed.

If configure doesn't find the compiler, or if you get Error loading shared libraries at run time, you may have forgotten to execute the script that sets up the correct PATH and library path. Unless your system manager has done this for you, you should execute the appropriate script - located in the directory containing the compiler executable - in your initialization files. Consult the documentation provided by Intel.

The warning: feupdateenv is not implemented and will always fail, can be safely ignored. Warnings on "bad preprocessing option" when compiling iotk and complains about ``recommanded formats'' may also be ignored.

The following compiler releases are known to give segmentation faults in at least some cases of compilation of QUANTUM ESPRESSO v.6.0:

12.0.0.084 Build 20101006
12.0.1.107 Build 20101116
12.0.2.137 Build 20110112
12.0.4.191 Build 20110427
12.0.5.220 Build 20110719
16.0.1.150 Build 20151021
(Filippo Spiga, Aug. 2016)

ifort v.12: release 12.0.0 miscompiles iotk, leading to mysterious errors when reading data files. Workaround: increase the parameter BLOCKSIZE to e.g. 131072*1024 when opening files in iotk/src/iotk_files.f90 (info by Lorenzo Paulatto, Nov. 2010).

ifort v.11: Segmentation faults were reported for the combination ifort 11.0.081, MKL 10.1.1.019, OpenMP 1.3.3. The problem disappeared with ifort 11.1.056 and MKL 10.2.2.025 (Carlo Nervi, Oct. 2009).

2.7.6.7 Linux PCs with MKL libraries

On Intel CPUs it is very convenient to use Intel MKL libraries. Recent versions also contain optimized FFT routines and a FFTW interface. MKL libraries can be used also with non-Intel compilers. They work also for AMD CPU, selecting the appropriate machine-optimized libraries, but with reduced performances.

configure should recognize properly installed MKL libraries. By default the non-threaded version of MKL is linked, unless option configure -with-openmp is specified. In case of trouble, refer to the following web page to find the correct way to link MKL:
http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor/.

For parallel (MPI) execution on multiprocessor (SMP) machines, set the environment variable OMP_NUM_THREADS to 1 unless you know what you are doing. See Sec.3 for more info on this and on the difference between MPI and OpenMP parallelization.

If you get a mysterious "too many communicators" error and a subsequent crash: there is a bug in Intel MPI and MKL 2016 update 3. See this thread and the links quoted therein: http://www.mail-archive.com/pw_forum@pwscf.org/msg29684.html.

2.7.6.8 Linux PCs with ACML libraries

For AMD CPUs, especially recent ones, you may find convenient to link AMD acml libraries (can be freely downloaded from AMD web site). configure should recognize properly installed acml libraries, together with the compilers most frequently used on AMD systems: pgf90, pathscale, openf95, sunf95.


2.7.7 Linux PC clusters with MPI

PC clusters running some version of MPI are a very popular computational platform nowadays. QUANTUM ESPRESSO is known to work with at least two of the major MPI implementations (MPICH, LAM-MPI), plus with the newer MPICH2 and OpenMPI implementation. configure should automatically recognize a properly installed parallel environment and prepare for parallel compilation. Unfortunately this not always happens. In fact:

Apart from such problems, QUANTUM ESPRESSO compiles and works on all non-buggy, properly configured hardware and software combinations. In some cases you may have to recompile MPI libraries: not all MPI installations contain support for the Fortran compiler of your choice (or for any Fortran compiler at all!).

If QUANTUM ESPRESSO does not work for some reason on a PC cluster, try first if it works in serial execution. A frequent problem with parallel execution is that QUANTUM ESPRESSO does not read from standard input, due to the configuration of MPI libraries: see Sec.3.4. If you are dissatisfied with the performances in parallel execution, see Sec.3 and in particular Sec.3.4.

2.7.8 Mac OS

Mac OS-X machines (10.4 and later) with Intel CPUs are supported by configure, both with gfortran and with the Intel compiler ifort and MKL libraries. Parallel compilation with OpenMPI also works.

Gfortran information and binaries for Mac OS-X here: http://hpc.sourceforge.net/ and https://wiki.helsinki.fi/display/HUGG/Installing+the+GNU+compilers+on+Mac+OS+X.

Mysterious crashes, occurring when zdotc is called, are due to the same incompatibility of complex functions with some optimized BLAS as reported in the "Linux PCs with gfortran" paragraph. Workaround: add preprocessing option -Dzdotc=zdotc_wrapper to DFLAGS.

2.7.8.1 Detailed installation instructions for Mac OS X 10.6

(Instructions for 10.6.3 by Osman Baris Malcioglu, tested as of May 2010) Summary for the hasty:

2.7.8.2 Compilation with GNU compilers

. The following instructions use macports version of gnu compilers due to some issues in mixing gnu supplied fortran compilers with apple modified gnu compiler collection. For more information regarding macports please refer to: http://www.macports.org/

First install necessary compilers from macports

   port install gcc43
   port install g95
The apple supplied MPI environment has to be overridden since there is a new set of compilers now (and Apple provided mpif90 is just an empty placeholder since Apple does not provide fortran compilers). I have used OpenMPI for this case. Recommended minimum configuration line is:
  ./configure CC=gcc-mp-4.3 CPP=cpp-mp-4.3 CXX=g++-mp-4.3 F77=g95 FC=g95
of course, installation directory should be set accordingly if a multiple compiler environment is desired. The default installation directory of OpenMPI overwrites apple supplied MPI permanently!
Next step is QUANTUM ESPRESSO itself. Sadly, the Apple supplied optimized BLAS/LAPACK libraries tend to misbehave under different tests, and it is much safer to use internal libraries. The minimum recommended configuration line is (presuming the environment is set correctly):
  ./configure CC=gcc-mp-4.3 CXX=g++-mp-4.3 F77=g95 F90=g95 FC=g95 \
              CPP=cpp-mp-4.3 --with-internal-blas --with-internal-lapack

2.7.8.3 Compilation with Intel compilers

. Newer versions of Intel compiler (>11.1.067) support Mac OS X 10.6, and furthermore they are bundled with intel MKL. 32 bit binaries obtained using 11.1.088 are tested and no problems have been encountered so far. Sadly, as of 11.1.088 the 64 bit binary misbehave under some tests. Any attempt to compile 64 bit binary using v.< 11.1.088 will result in very strange compilation errors.

Like the previous section, I would recommend installing macports compiler suite. First, make sure that you are using the 32 bit version of the compilers, i.e.

. /opt/intel/Compiler/11.1/088/bin/ifortvars.sh ia32
. /opt/intel/Compiler/11.1/088/bin/iccvars.sh ia32
will set the environment for 32 bit compilation in my case.

Then, the MPI environment has to be set up for Intel compilers similar to previous section.

The recommended configuration line for QUANTUM ESPRESSO is:

 ./configure CC=icc CXX=icpc F77=ifort F90=ifort FC=ifort CPP=cpp-mp-4.3
MKL libraries will be detected automatically if they are in their default locations. Otherwise, mklvars32 has to be sourced before the configuration script.

Security issues: MacOs 10.6 comes with a disabled firewall. Preparing a ipfw based firewall is recommended. Open source and free GUIs such as "WaterRoof" and "NoobProof" are available that may help you in the process.


next up previous contents
Next: 3 Parallelism Up: 2 Installation Previous: 2.6 Running tests and   Contents
2017-03-03