8.3. Installation
SICOPOLIS-AD v2 requires the installation of Tapenade as well as SICOPOLIS. It is mandatory to install the external libraries such as NetCDF, LIS to access the full functionality of the code, as well as git, to be able to clone and contribute to the repository.
8.3.1. Open source AD Tool Tapenade
Tapenade is an Automatic Differentiation Engine developed at Inria at Sophia Antipolis by the Ecuador team (formerly TROPICS team). Tapenade takes as input a computer source program, plus a request for differentiation. Tapenade builds and returns the differentiated source program, that evaluates the required derivatives.
While the SICOPOLIS source files are prepared to generate adjoint sensitivities, they will not be able to do so without an operable installation of Tapenade. Fortunately the Tapenade installation procedure is straightforward.
We detail the instructions here, but the latest instructions can always be found here.
8.3.1.1. Prerequisites for Linux or Mac OS X
Before installing Tapenade, you must check that an up-to-date Java Runtime Environment is installed. Tapenade will not run with older Java Runtime Environment.
8.3.1.2. Steps for Mac OS
Tapenade 3.16 distribution does not contain a fortranParser executable for MacOS. It uses a docker image from here. You need docker on your Mac to run the Tapenade distribution with Fortran programs. Details on how to build fortranParser is here. You may also build Tapenade on your Mac from the gitlab repository.
8.3.1.3. Steps for Linux
Download tapenade_3.16.tar into your chosen installation directory install_dir.
Go to your chosen installation directory install_dir, and extract Tapenade from the tar file:
% tar xvfz tapenade_3.16.tar
On Linux, depending on your distribution, Tapenade may require you to set the shell variable
JAVA_HOME
to your java installation directory. It is oftenJAVA_HOME=/usr/java/default
. You might also need to modify thePATH
by adding the bin directory from the Tapenade installation. An example can be found here.
NOTE: Every time you wish to use the adjoint capability of SICOPOLIS-AD, you must re-source the environment. We recommend that this be done automatically in your bash or c-shell profile upon login. An example of an addition to a .bashrc
file from a Linux server is given below. Luckily, shell variable JAVA_HOME
was not required to be explicitly set for this particular Linux distribution, but might be necessary for some other distributions.
##set some env variables for SICOPOLIS tapenade
export TAPENADE_HOME="/home/shreyas/tapenade_3.16"
export PATH="$PATH:$TAPENADE_HOME/bin"
##Modules
module use /share/modulefiles/
module load java/jdk/16.0.1 # Java required by Tapenade
You should now have a working copy of Tapenade.
For more information on the tapenade command and its arguments, type:
tapenade -?
8.3.1.4. Prerequisites for Windows
NOTE: Although Tapenade can be built on Windows, SICOPOLIS requires a Unix-like system (e.g., Linux), as mentioned in “Getting started”.
Before installing Tapenade, you must check that an up-to-date Java Runtime Environment is installed. Tapenade will not run with older Java Runtime Environment. The Fortran parser of Tapenade uses cygwin.
8.3.1.5. Steps for Windows
Download tapenade_3.16.zip into your chosen installation directory install_dir.
Go to your chosen installation directory install_dir, and extract Tapenade from the zip file.
Save a copy of the
install_dir\tapenade_3.16\bin\tapenade.bat
file and modifyinstall_dir\tapenade_3.16\bin\tapenade.bat
according to your installation parameters:
replaceTAPENADE_HOME=..
byTAPENADE_HOME="install_dir"\tapenade_3.16
replaceJAVA_HOME="C:\Progra~1\Java\jdkXXXX"
by your current java directoryreplaceBROWSER="C:\Program Files\Internet Explorer\iexplore.exe"
by your current browser.
8.3.2. SICOPOLIS-AD v2
See the “Getting started” section. However, we will mention here some steps since using the Automatic Differentiation (AD) capabilities with Tapenade requires a slightly modified setup.
The satisfaction of the following prerequisites is highly recommended to access all the features of the code. Details can differ from the “Getting started” section, since there are multiple ways to do things. We detail one of them here.
8.3.2.1. GNU GCC Compiler (gfortran+gcc) or Intel Compiler (ifort+icc)
We have tested SICOPOLIS-AD v2 on gfortran/gcc v5.4.0, v7.2.0 and v8.5.0, any intermediate versions should work just as well. We have also tested it on ifort/icc v18.0.0 (however, it should be noted that we have not tested the external Lis solver with Intel compilers).
8.3.2.2. Lis (1.4.43 or newer)
Lis installation is mandatory to use shallow-shelf/shelfy-stream dynamics in simulations. Install Lis as explained in Dependencies/Lis.
The following commands might be helpful, they are written for the latest version at the time of writing:
wget https://www.ssisc.org/lis/dl/lis-2.0.30.zip
unzip lis-2.0.30.zip
cd lis-2.0.30
./configure --enable-fortran --prefix=/lis-2.0.30/installation/ --enable-shared && make && make check && make install
For AD purposes, we compile the code using the src/MakefileTapenade
makefile. This makefile requires the following environment variables set:
LISDIR
- The installation directory for the Lis version to be used. You can change this variable, either in the Makefile directly, or automatically in your bash or c-shell profile upon login (for example,.bashrc
). Examples for both are shown here.
src/MakefileTapenade
:ifndef LISDIR LISDIR=/home/shreyas/lis-2.0.30/installation endif
.bashrc
:export LISDIR="/home/shreyas/lis-2.0.30/installation"
LIBLIS
- Absolute path toliblis.so
. By default insrc/MakefileTapenade
, it is${LISDIR}/lib/liblis.so
. If you follow the original instructions to install Lis, this should work, else one can set it manually withinsrc/MakefileTapenade
.LIBLISFLAG
- Add directory using-L
to be searched for-llis
. By default insrc/MakefileTapenade
, it is-L${LISDIR}/lib -llis
. If you follow the original instructions to install Lis, this should work, else one can set it manually withinsrc/MakefileTapenade
.LISFLAG
- This flag declares directories to be searched for Lis#include
header filelisf.h
, as well as defines theBUILD_LIS
as a macro with value 1. By default insrc/MakefileTapenade
, it is-DBUILD_LIS -I${LISDIR}/include/
. If you follow the original instructions to install Lis, this should work, else one can set it manually withinsrc/MakefileTapenade
.
NOTE: Some users have reported needing to extend their LD_LIBRARY_PATH
with the location of ${LISDIR}/lib
in order to find liblis.so.0
.
8.3.2.3. NetCDF (3.6.x or newer)
NetCDF installation is mandatory since it is a powerful library with widespread use for I/O with a machine-independent data format. Install NetCDF as explained in Dependencies/NetCDF.
In some cases, for example while working on a shared server which uses a module manager or Docker container, thing have to be set up differently. src/MakefileTapenade
needs either the NETCDF_FORTRAN_DIR
macro set or both NETCDF_F90_FLAG
and LIB_NETCDF_F90_FLAG
set (see code snippet from src/MakefileTapenade
here).
ifndef NETCDF_F90_FLAG
ifndef LIB_NETCDF_F90_FLAG
ifdef NETCDF_FORTRAN_DIR
LIB_NETCDF_F90=${NETCDF_FORTRAN_DIR}/lib/libnetcdff.so
LIB_NETCDF_F90_FLAG=-L${NETCDF_FORTRAN_DIR}/lib -lnetcdff
NETCDF_F90_FLAG=-I${NETCDF_FORTRAN_DIR}/include/
endif
endif
endif
NETCDF_FORTRAN_DIR
- The installation directory for netcdf-fortran. You can change this variable, either in the Makefile directly, or automatically in your bash or c-shell profile upon login (for example,.bashrc
). Examples for both are shown here.
src/MakefileTapenade
ifndef NETCDF_FORTRAN_DIR NETCDF_FORTRAN_DIR=/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4 endif
.bashrc
export NETCDF_FORTRAN_DIR="/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4"
LIB_NETCDF_F90
- Absolute path tolibnetcdff.so
. By default insrc/MakefileTapenade
, it is{NETCDF_FORTRAN_DIR}/lib/libnetcdff.so
.LIB_NETCDF_F90_FLAG
- Add directory using-L
to be searched for-lnetcdff
. By default insrc/MakefileTapenade
, it is-L${NETCDF_FORTRAN_DIR}/lib -lnetcdff
. See some examples below where this has to be set explicitly in case of a Docker container.NETCDF_F90_FLAG
- This flag declares directories to be searched for netcdf-fortran#include
header files. By default insrc/MakefileTapenade
, it is-I${NETCDF_FORTRAN_DIR}/include/
. See some examples below where this has to be set explicitly in case of a Docker container.
For a server that uses modules, you can load the relevant modules using commands like these (can also make permanent by adding to login script like .bashrc
:
% module use /share/modulefiles/
% module load openmpi
% module load netcdf-fortran
% module load netcdf
You then have to give the NETCDF_FORTRAN_DIR
macro to src/MakefileTapenade
, either by adding to a login script or directly inside the makefile. If your system uses a module manager, you can query to find the exact directory location.
% module show netcdf-fortran
----------------------------------------------------------
/opt/ohpc/pub/moduledeps/gnu-openmpi/netcdf-fortran/4.4.4:
----------------------------------------------------------
whatis("Name: NETCDF_FORTRAN built with gnu toolchain ")
whatis("Version: 4.4.4 ")
whatis("Category: runtime library ")
whatis("Description: Fortran Libraries for the Unidata network Common Data Form ")
whatis("http://www.unidata.ucar.edu/software/netcdf/ ")
prepend_path("PATH","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/bin")
prepend_path("MANPATH","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/share/man")
prepend_path("INCLUDE","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/include")
prepend_path("LD_LIBRARY_PATH","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/lib")
setenv("NETCDF_FORTRAN_DIR","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4")
setenv("NETCDF_FORTRAN_BIN","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/bin")
setenv("NETCDF_FORTRAN_LIB","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/lib")
setenv("NETCDF_FORTRAN_INC","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/include")
help([[
This module loads the NetCDF Fortran API built with the gnu compiler toolchain.
Note that this build of NetCDF leverages the HDF I/O library and requires linkage
against hdf5 and the native C NetCDF library. Consequently, phdf5 and the standard C
version of NetCDF are loaded automatically via this module. A typical compilation
example for Fortran applications requiring NetCDF is as follows:
]])
In this case NETCDF_FORTRAN_DIR=/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4
.
For a Docker container, for example one with a centos:8
distribution, and the dnf
package manager, NetCDF is typically installed as follows:
RUN dnf install -y https://github.com/openhpc/ohpc/releases/download/v1.3.GA/ohpc-release-1.3-1.el7.x86_64.rpm
# Add some packages
RUN dnf -y install epel-release
RUN dnf -y install dnf-plugins-core
RUN dnf config-manager --set-enabled powertools
RUN dnf -y install make which git
RUN dnf -y install diffutils
RUN dnf -y install vim
RUN dnf -y install autoconf automake
RUN dnf -y install valgrind-ohpc
RUN dnf -y install gnu8-compilers-ohpc
RUN dnf -y install gsl-gnu8-ohpc hdf5-gnu8-ohpc
RUN dnf -y install openmpi-devel
RUN dnf -y install bc wget zlib-devel perl-Digest-MD5
RUN dnf -y --enablerepo=powertools install netcdf-fortran netcdf-devel # NetCDF installation
RUN dnf -y install netcdf-fortran-devel # NetCDF installation
In this case, you will find that the ./usr/lib64/gfortran/modules/netcdf.mod
exists in your docker environment. In this case, you can just directly set NETCDF_F90_FLAG=-I/usr/lib64/gfortran/modules
either the makefile or the login script (no need to set NETCDF_FORTRAN_DIR
macro).
You can also confirm that the files /usr/lib64/libnetcdff.so*
and /usr/lib64/libnetcdf.so*
exist, which means you have to set LIB_NETCDF_F90_FLAG=-L/usr/lib64 -lnetcdff
.
The instructions given in Dependencies/NetCDF, and these two cases should help cover most of the issues with the installation of NetCDF.
8.3.2.4. Downloading SICOPOLIS-AD v2
As described in the “Getting started” section. However, when using Git, the ad
branch should be cloned:
git clone --branch ad \
https://gitlab.awi.de/sicopolis/sicopolis.git
Tagged versions of SICOPOLIS-AD are also available from Zenodo.
8.3.2.5. Initial configuration
As described in the “Getting started” section.
Now you should be ready to use SICOPOLIS-AD v2, as described in Running SICOPOLIS-AD v2.