.. _ad_installation: Installation ************ SICOPOLIS-AD v2 requires the installation of Tapenade as well as SICOPOLIS. It is mandatory to install the external libraries such as NetCDF, LIS to access the full functionality of the code, as well as git, to be able to clone and contribute to the repository. .. _tapenade: Open source AD Tool Tapenade ============================ `Tapenade `__ is an Automatic Differentiation Engine developed at `Inria at Sophia Antipolis `__ by the `Ecuador team `__ (formerly `TROPICS team `__). Tapenade takes as input a computer source program, plus a request for differentiation. Tapenade builds and returns the differentiated source program, that evaluates the required derivatives. While the SICOPOLIS source files are prepared to generate adjoint sensitivities, they will not be able to do so without an operable installation of Tapenade. Fortunately the Tapenade installation procedure is straightforward. We detail the instructions here, but the latest instructions can always be found `here. `__ Prerequisites for Linux or Mac OS X ----------------------------------- Before installing Tapenade, you must check that an up-to-date Java Runtime Environment is installed. Tapenade will not run with older Java Runtime Environment. Steps for Mac OS ---------------- Tapenade 3.16 distribution does not contain a fortranParser executable for MacOS. It uses a docker image from `here `__. You need docker on your Mac to run the Tapenade distribution with Fortran programs. Details on how to build fortranParser is `here `__. You may also build Tapenade on your Mac from the `gitlab repository `__. Steps for Linux --------------- 1. Read `the Tapenade license. `__ 2. Download `tapenade_3.16.tar `__ into your chosen installation directory *install_dir*. **NOTE**: Alternatively, a Tapenade version that works correctly with SICOPOLIS-AD v2 is always available in the ``test_ad/tapenade_supported`` directory. 3. Go to your chosen installation directory *install_dir*, and extract Tapenade from the tar file:: % tar xvfz tapenade_3.16.tar 4. On Linux, depending on your distribution, Tapenade may require you to set the shell variable ``JAVA_HOME`` to your java installation directory. It is often ``JAVA_HOME=/usr/java/default``. You might also need to modify the ``PATH`` by adding the bin directory from the Tapenade installation. An example can be found :ref:`here `. .. _tapenade_bashrc_snippet: **NOTE**: Every time you wish to use the adjoint capability of SICOPOLIS-AD, you must re-source the environment. We recommend that this be done automatically in your bash or c-shell profile upon login. An example of an addition to a ``.bashrc`` file from a Linux server is given below. Luckily, shell variable ``JAVA_HOME`` was not required to be explicitly set for this particular Linux distribution, but might be necessary for some other distributions. :: ##set some env variables for SICOPOLIS tapenade export TAPENADE_HOME="/home/shreyas/tapenade_3.16" export PATH="$PATH:$TAPENADE_HOME/bin" ##Modules module use /share/modulefiles/ module load java/jdk/16.0.1 # Java required by Tapenade You should now have a working copy of Tapenade. For more information on the tapenade command and its arguments, type:: tapenade -? Prerequisites for Windows ------------------------- **NOTE**: Although Tapenade can be built on Windows, SICOPOLIS requires a Unix-like system (e.g., Linux), as mentioned in ":ref:`getting_started`". Before installing Tapenade, you must check that an up-to-date Java Runtime Environment is installed. Tapenade will not run with older Java Runtime Environment. The Fortran parser of Tapenade uses `cygwin `__. Steps for Windows ----------------- 1. Read `the Tapenade license. `__ 2. Download `tapenade_3.16.zip `__ into your chosen installation directory *install_dir*. 3. Go to your chosen installation directory *install_dir*, and extract Tapenade from the zip file. 4. Save a copy of the ``install_dir\tapenade_3.16\bin\tapenade.bat`` file and modify ``install_dir\tapenade_3.16\bin\tapenade.bat`` according to your installation parameters\: | replace ``TAPENADE_HOME=..`` by ``TAPENADE_HOME="install_dir"\tapenade_3.16`` | replace ``JAVA_HOME="C:\Progra~1\Java\jdkXXXX"`` by your current java directory | replace ``BROWSER="C:\Program Files\Internet Explorer\iexplore.exe"`` by your current browser. .. _sicopolis_ad_config: SICOPOLIS-AD v2 =============== See the ":ref:`getting_started`" section. However, we will mention here some steps since using the Automatic Differentiation (AD) capabilities with Tapenade requires a slightly modified setup. The satisfaction of the following prerequisites is highly recommended to access all the features of the code. Details can differ from the ":ref:`getting_started`" section, since there are multiple ways to do things. We detail one of them here. GNU GCC Compiler (gfortran+gcc) or Intel Compiler (ifort+icc) ------------------------------------------------------------- We have tested SICOPOLIS-AD v2 on gfortran/gcc v5.4.0, v7.2.0 and v8.5.0, any intermediate versions should work just as well. We have also tested it on ifort/icc v18.0.0 (however, it should be noted that we have not tested the external Lis solver with Intel compilers). Lis (1.4.43 or newer) --------------------- Lis installation is mandatory to use shallow-shelf/shelfy-stream dynamics in simulations. Install Lis as explained in :ref:`Dependencies/Lis `. The following commands might be helpful, they are written for the latest version at the time of writing:: wget https://www.ssisc.org/lis/dl/lis-2.0.30.zip unzip lis-2.0.30.zip cd lis-2.0.30 ./configure --enable-fortran --prefix=/lis-2.0.30/installation/ --enable-shared && make && make check && make install For AD purposes, we compile the code using the ``src/MakefileTapenade`` makefile. This makefile requires the following environment variables set\: 1. ``LISDIR`` - The installation directory for the Lis version to be used. You can change this variable, either in the Makefile directly, or automatically in your bash or c-shell profile upon login (for example, ``.bashrc``). Examples for both are shown here. ``src/MakefileTapenade``:: ifndef LISDIR LISDIR=/home/shreyas/lis-2.0.30/installation endif ``.bashrc``:: export LISDIR="/home/shreyas/lis-2.0.30/installation" 2. ``LIBLIS`` - Absolute path to ``liblis.so``. By default in ``src/MakefileTapenade``, it is ``${LISDIR}/lib/liblis.so``. If you follow the original instructions to install Lis, this should work, else one can set it manually within ``src/MakefileTapenade``. 3. ``LIBLISFLAG`` - Add directory using ``-L`` to be searched for ``-llis``. By default in ``src/MakefileTapenade``, it is ``-L${LISDIR}/lib -llis``. If you follow the original instructions to install Lis, this should work, else one can set it manually within ``src/MakefileTapenade``. 4. ``LISFLAG`` - This flag declares directories to be searched for Lis ``#include`` header file ``lisf.h``, as well as defines the ``BUILD_LIS`` as a macro with value 1. By default in ``src/MakefileTapenade``, it is ``-DBUILD_LIS -I${LISDIR}/include/``. If you follow the original instructions to install Lis, this should work, else one can set it manually within ``src/MakefileTapenade``. **NOTE**: Some users have reported needing to extend their ``LD_LIBRARY_PATH`` with the location of ``${LISDIR}/lib`` in order to find ``liblis.so.0``. NetCDF (3.6.x or newer) ----------------------- NetCDF installation is mandatory since it is a powerful library with widespread use for I/O with a machine-independent data format. Install NetCDF as explained in :ref:`Dependencies/NetCDF `. In some cases, for example while working on a shared server which uses a module manager or Docker container, thing have to be set up differently. ``src/MakefileTapenade`` needs either the ``NETCDF_FORTRAN_DIR`` macro set or both ``NETCDF_F90_FLAG`` and ``LIB_NETCDF_F90_FLAG`` set (see code snippet from ``src/MakefileTapenade`` here). :: ifndef NETCDF_F90_FLAG ifndef LIB_NETCDF_F90_FLAG ifdef NETCDF_FORTRAN_DIR LIB_NETCDF_F90=${NETCDF_FORTRAN_DIR}/lib/libnetcdff.so LIB_NETCDF_F90_FLAG=-L${NETCDF_FORTRAN_DIR}/lib -lnetcdff NETCDF_F90_FLAG=-I${NETCDF_FORTRAN_DIR}/include/ endif endif endif 1. ``NETCDF_FORTRAN_DIR`` - The installation directory for netcdf-fortran. You can change this variable, either in the Makefile directly, or automatically in your bash or c-shell profile upon login (for example, ``.bashrc``). Examples for both are shown here. ``src/MakefileTapenade`` :: ifndef NETCDF_FORTRAN_DIR NETCDF_FORTRAN_DIR=/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4 endif ``.bashrc`` :: export NETCDF_FORTRAN_DIR="/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4" 2. ``LIB_NETCDF_F90`` - Absolute path to ``libnetcdff.so``. By default in ``src/MakefileTapenade``, it is ``{NETCDF_FORTRAN_DIR}/lib/libnetcdff.so``. 3. ``LIB_NETCDF_F90_FLAG`` - Add directory using ``-L`` to be searched for ``-lnetcdff``. By default in ``src/MakefileTapenade``, it is ``-L${NETCDF_FORTRAN_DIR}/lib -lnetcdff``. See some examples below where this has to be set explicitly in case of a Docker container. 4. ``NETCDF_F90_FLAG`` - This flag declares directories to be searched for netcdf-fortran ``#include`` header files. By default in ``src/MakefileTapenade``, it is ``-I${NETCDF_FORTRAN_DIR}/include/``. See some examples below where this has to be set explicitly in case of a Docker container. For a server that uses modules, you can load the relevant modules using commands like these (can also make permanent by adding to login script like ``.bashrc``:: % module use /share/modulefiles/ % module load openmpi % module load netcdf-fortran % module load netcdf You then have to give the ``NETCDF_FORTRAN_DIR`` macro to ``src/MakefileTapenade``, either by adding to a login script or directly inside the makefile. If your system uses a module manager, you can query to find the exact directory location. :: % module show netcdf-fortran ---------------------------------------------------------- /opt/ohpc/pub/moduledeps/gnu-openmpi/netcdf-fortran/4.4.4: ---------------------------------------------------------- whatis("Name: NETCDF_FORTRAN built with gnu toolchain ") whatis("Version: 4.4.4 ") whatis("Category: runtime library ") whatis("Description: Fortran Libraries for the Unidata network Common Data Form ") whatis("http://www.unidata.ucar.edu/software/netcdf/ ") prepend_path("PATH","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/bin") prepend_path("MANPATH","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/share/man") prepend_path("INCLUDE","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/include") prepend_path("LD_LIBRARY_PATH","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/lib") setenv("NETCDF_FORTRAN_DIR","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4") setenv("NETCDF_FORTRAN_BIN","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/bin") setenv("NETCDF_FORTRAN_LIB","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/lib") setenv("NETCDF_FORTRAN_INC","/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4/include") help([[ This module loads the NetCDF Fortran API built with the gnu compiler toolchain. Note that this build of NetCDF leverages the HDF I/O library and requires linkage against hdf5 and the native C NetCDF library. Consequently, phdf5 and the standard C version of NetCDF are loaded automatically via this module. A typical compilation example for Fortran applications requiring NetCDF is as follows: ]]) In this case ``NETCDF_FORTRAN_DIR=/opt/ohpc/pub/libs/gnu/openmpi/netcdf-fortran/4.4.4``. For a Docker container, for example one with a ``centos:8`` distribution, and the ``dnf`` package manager, NetCDF is typically installed as follows:: RUN dnf install -y https://github.com/openhpc/ohpc/releases/download/v1.3.GA/ohpc-release-1.3-1.el7.x86_64.rpm # Add some packages RUN dnf -y install epel-release RUN dnf -y install dnf-plugins-core RUN dnf config-manager --set-enabled powertools RUN dnf -y install make which git RUN dnf -y install diffutils RUN dnf -y install vim RUN dnf -y install autoconf automake RUN dnf -y install valgrind-ohpc RUN dnf -y install gnu8-compilers-ohpc RUN dnf -y install gsl-gnu8-ohpc hdf5-gnu8-ohpc RUN dnf -y install openmpi-devel RUN dnf -y install bc wget zlib-devel perl-Digest-MD5 RUN dnf -y --enablerepo=powertools install netcdf-fortran netcdf-devel # NetCDF installation RUN dnf -y install netcdf-fortran-devel # NetCDF installation In this case, you will find that the ``./usr/lib64/gfortran/modules/netcdf.mod`` exists in your docker environment. In this case, you can just directly set ``NETCDF_F90_FLAG=-I/usr/lib64/gfortran/modules`` either the makefile or the login script (no need to set ``NETCDF_FORTRAN_DIR`` macro). You can also confirm that the files ``/usr/lib64/libnetcdff.so*`` and ``/usr/lib64/libnetcdf.so*`` exist, which means you have to set ``LIB_NETCDF_F90_FLAG=-L/usr/lib64 -lnetcdff``. The instructions given in :ref:`Dependencies/NetCDF `, and these two cases should help cover most of the issues with the installation of NetCDF. Downloading SICOPOLIS-AD v2 --------------------------- As described in the ":ref:`getting_started`" section. However, when using Git, the ``ad`` branch should be cloned:: git clone --branch ad \ https://gitlab.awi.de/sicopolis/sicopolis.git Tagged versions of SICOPOLIS-AD are also available from `Zenodo `__. Initial configuration --------------------- As described in the ":ref:`getting_started`" section. Now you should be ready to use SICOPOLIS-AD v2, as described in :ref:`Running SICOPOLIS-AD v2 `.