starting.tex

\section{Getting started}
\label{Starting}

\subsection{myroms.org}
\label{Myroms}
Starting off with ROMS is not the easiest thing to do, and it just
seems to be getting more complex as time goes by. There are
some resources, however, beginning with the
electronic home for ROMS users at
\href{http://www.myroms.org}{www.myroms.org}. Go to register,
which gives you access to the subversion server for the code and to
the  discussion forum for all things ROMS. There is also a wiki,
a bug tracking system, and even a developer blog.

The wiki contains parts of this manual, but the nature of wikis is that
they can be more fluid, with more authors, than a document such as
this. Dave Robertson (robertson@marine.rutgers.edu) is the one to
talk to if you would like to contribute to the wiki. On the other
hand, this manual is now at
\href{https://github.com/kshedstrom/roms_manual}{github}, with all that
implies (not that I've gotten pull requests there yet).

\subsection{Prerequisites}
As mentioned in Chapter \ref{Intro}, ROMS has some external
requirements. These are:
\begin{itemize}
    \item UNIX or UNIX-like environment, such as
      \href{http://www.redhat.com/services/custom/cygwin/}{Cygwin}.
    \item A Fortran 90 compiler.
    \item The
      \href{http://www.unidata.ucar.edu/software/netcdf/index.html}{NetCDF
      library} and the
      \href{https://www.unidata.ucar.edu/downloads/netcdf/netcdf-fortran/index.jsp}{netcdf-fortran
      library} compiled with the above compiler (and a C compiler).
    \item \href{http://subversion.tigris.org}{svn}, the subversion
      revision control software. See Appendix \ref{Svn} and the
      \href{https://www.myroms.org/wiki/index.php/Subversion}{ROMS wiki}.
    \item \href{https://git-scm.com/}{git} is optional, but can be
      used instead of svn, especially for downloading the ice branch
      of roms from \href{https://github.com/kshedstrom/roms}{github}.
    \item \href{http://www.gnu.org/software/make/}{Gnu make} version
      3.81 or higher. Appendix \ref{Gmake} contains more than you
      ever wanted to know about this software.
    \item A \href{http://en.wikipedia.org/wiki/C_preprocessor}{C
      preprocessor}---the one from gnu with the \code{-traditional}
      flag works well. See Appendix \ref{Cpp}.
    \item The \href{http://www.perl.org}{Perl} scripting language.
    \item \href{http://www.mathworks.com/products/matlab/}{Matlab} is
      optional, but it is a common tool for pre- and post-processing
      of ROMS files.
    \item \href{https://www.python.org/}{Python} is also
      optional, but might be a good thing to learn if you want an
      alternative to Matlab.
\end{itemize}
Make sure you've got the right environment before attempting to
download or compile ROMS. In particular, it's best to download ROMS
using the same environment as will be used when compiling and
running it.

\subsection{Acquiring the ROMS code}
The main ROMS code is available for download via \code{svn} at
\url{https://www.myroms.org/svn/src/}. The version of the model
described in this document is a merger between ROMS 3.7 and the
integrated Budgell sea-ice model and is available at
\href{https://github.com/kshedstrom/roms}{github}. I am no longer
maintaining the svn branch on the myroms.org site.

ROMS comes with several cases all ready to go at the flip
of a switch. Try these out first and learn how they are set up.
    \begin{itemize}
      \item \S\ref{Build} describes how to pick the cases and set up
      the build environment.
      \item \S\ref{Cpp1} lists all the ROMS options that can be
      added to your case.
      \item \S\ref{Functionals} lists the fields which can be
      provided to ROMS via analytic expressions.
      \item \S\ref{ASCII_in} lists the input parameters ROMS reads
      from a text file at run time.
      \item Chapters \ref{Code} and \ref{Wave} are meant to be
      informative for the simple and not-so-simple cases. If that
      isn't the case, please let me know.
    \end{itemize}
In addition to this manual, there are some other ROMS resources:
\begin{itemize}
   \item You may be best served by going to the 
\href{https://www.myroms.org/wiki/}{ROMS wiki} which includes
sections called Getting Started and Tutorials, as well as a Frequently
Asked Questions (FAQ) section.
   \item Don't be afraid to use the forum. It has everything from
employment opportunities to debugging help. Posting there can get
you help from one of several people, improving your odds of success
over private emails. Registered users get an email once a day about
new postings, so you might have to wait a day (or more) for a reply.
   \item There have been ROMS meetings and classes in which a tutorial
session is included as part of the program.
\end{itemize}

\subsection{Compiling ROMS}
\label{Build}

\subsubsection{Environment Variables for \code{make}}
\label{make_var}

ROMS has a growing list of choices the user must make before starting
the compile process. These are user-defined variables, some about
the model, some about your computer. This section is on the
computer/compiler pairing that is your build system. Since we now
use \code{gnu make}, it is possible to set the value of these
variables in the Unix environment in a build script, rather than
necessarily inside the \code{Makefile} (see \S\ref{Gmake}).  The
user-definable variables understood by the ROMS \code{makefile} are:
\begin{klist}

\kitem{ROMS\_APPLICATION} Set the \code{cpp} option defining the particular
application. This is used for setting up options inside the code
specific to this application and also determines the name of the
\code{.h} header file for it. This can be either a predefined
case, such as \code{BENCHMARK}, or one of your own, such as \code{NEP5}
(a Northeast Pacific application).
It should be in all upper case by convention.

\kitem{MY\_HEADER\_DIR} Sets the path to the user's header file, if
any. It can be left empty for the standard cases, since \code{benchmark.h}
and the like are found in \code{ROMS/Include}, which is already in
the search path. In the case of \code{NEP5}, this is set to
\code{Apps/NEP} where \code{nep5.h} resides.

\kitem{MY\_ANALYTICAL\_DIR} Sets the path to the user's analytic files
described in \S\ref{Functionals}, if any. This can be \code{User/Functionals}
or some other location. I tend to place both the header file and the
functionals in the same directory, one directory per application.

\kitem{MY\_CPP\_FLAGS} Set tunable \code{cpp} options. Sometimes it is desirable
to activate one or more \code{cpp} options to run different variants of the
same application without modifying its header file. If this is the
case, specify each option here using the \code{-D} syntax. Notice that
you need to use the shell's quoting syntax (either single or double
quotes) to enclose the definition if you are using one of the build
scripts below.

%\kitem{NestedGrids} Integer number of grids in the setup, usually 1.

\kitem{Compiler-specific Options} These flags are used by the files
inside the \code{Compilers} directory.
\begin{klist}
  \kitem{USE\_DEBUG} Set this to \code{on} to turn off optimization
and turn on the \code{-g} flag for debugging.

  \kitem{USE\_MPI} Set this if running an MPI parallel job.
  \kitem{USE\_OpenMP} Set this if running an OpenMP parallel job.
  \kitem{USE\_MPIF90} This flag changes the name of the
  Fortran compiler to \code{mpif90}, while
  \code{USE\_MPI} changes the \code{cpp} flags
  for changing the source code (you might need both). \code{OpenMPI} and others
  provide a shell script called \code{mpif90} for finding the required MPI
  files.
  \kitem{USE\_LARGE}  Some systems support both 32-bit and 64-bit
  options. Select this to get 64-bit addressing, usually used for
  programs needing more than 2 GB of memory on 32-bit systems.
  I run on machines which are 64-bit by default, making this option
  obsolete.
  \kitem{USE\_NETCDF4} Set this if linking against the NetCDF4
  library, which is highly
  recommended. If you set this, you can skip the next two.
  \kitem{NETCDF\_INCDIR} The location of the \code{netcdf.mod} and
  \code{typesizes.mod} files.
  \kitem{NETCDF\_LIBDIR} The location of the NetCDF and netcdf-fortran
  libraries.
\end{klist}

\kitem{FORT} A shorthand name for the compiler to be used when
selecting which system-compiler file is to be included from the
\code{Compilers} directory. See section \S\ref{Inc_fort} and
\S\ref{make_env}.

\kitem{Local File Options:} \mbox{}

\begin{klist}
  \kitem{BINDIR} Directory in which to place the binary executable.
The default is ``\code{.}'', the current directory.

  \kitem{SCRATCH\_DIR} Put the \code{.f90} and the temporary binary files
in a build directory to avoid clutter. The default is \code{Build}
under the current directory. Set it to something else if you want
to keep these files for multiple projects at the same time, each in
their own directory or if they need to be outside your source tree to
comply with system quotas.
\end{klist}
\end{klist}

\subsubsection{Providing the Environment}
\label{make_env}

Before compiling, you will need to find out some background information:
\begin{itemize}
 \item What is the name of your compiler?
 \item What is returned by \code{uname -s} on your system?
 \item Is there a working NetCDF-fortran library?
 \item Where is it?
 \item Was it built with the above compiler?
 \item Did it come with a working \code{nc-config}? How about
 \code{nf-config}?
 \item Do you have access to MPI or OpenMP?
\end{itemize}
As described more fully in \S\ref{Inc_fort}, the \code{makefile} will be
looking for a file in the \code{Compilers} directory with the
combination of your operating system and your compiler. For
instance, using Linux and the Gnu compiler, the file would be
called \code{Linux-gfortran.mk}. Is the corresponding file for your
system and compiler in the \code{Compilers} directory? If not, you
will have to create it following the existing examples there.

Next, there are several ways to provide the location for the NetCDF
files (and optional HDF5 library). The recommended way is to take
advantage of the \code{nc-config} program that comes with NetCDF 4.0.1
and newer. The files in the \code{Compilers} directory will attempt
to do this if \code{USE\_NETCDF4} is defined. Specifically, it will
invoke ``nc-config --flibs'' and ``nc-config --prefix'' which you can
run on the command line yourself for testing purposes:
\begin{verbatim}
    pacman3 201% nc-config --flibs
    -L/usr/local/pkg/netcdf/netcdf-4.3.0.gnu-4.7.3/lib -lnetcdff
    -L/usr/local/pkg/hdf5/hdf5-1.8.10-p1.gnu-4.7.3/lib
    -L/usr/local/pkg/szip/szip-2.1.gnu/lib
    -L/usr/local/pkg/udunits/udunits-2.1.24.gnu-4.7.3/lib
    -L/usr/local/pkg/netcdf/netcdf-4.3.0.gnu-4.7.3/lib -lnetcdf -lnetcdf
    pacman3 202% nc-config --prefix
    /usr/local/pkg/netcdf/netcdf-4.3.0.gnu-4.7.3
\end{verbatim}
As you can see, if it needs to link to the hdf5 library, it does
this automatically. Another example:
\begin{verbatim}
    kate@ThinkPad-W520$ nc-config --flibs

    kate@ThinkPad-W520$ nf-config --flibs
    -L/usr/local/lib -lnetcdff -lnetcdf -lnetcdf
    kate@ThinkPad-W520$ nf-config --prefix
    /usr/local
\end{verbatim}
This is an example of a system where \code{nc-config} is broken but
\code{nf-config} works. Edit this line of the corresponding file in the
\code{Compilers} directory:
\begin{verbatim}
    NC_CONFIG ?= nf-config
\end{verbatim}
If you insist on using the old method, it requires setting the
location of the NetCDF files in one of three ways:
\begin{enumerate}
  \item Hardcoding lines in the Compilers/xxx-yyy.mk file:
\begin{verbatim}
    NETCDF_INCDIR ?= /usr/local/include
    NETCDF_LIBDIR ?= /usr/local/lib
\end{verbatim}
  \item Setting them in the build.bash (if you can figure out where in
all those if cases).
  \item Edit your \code{.profile} or \code{.login} files to globally
set them. Here is an example for
\code{tcsh}:
\begin{verbatim}
    setenv NETCDF_INCDIR /usr/local/netcdf4/include
    setenv NETCDF_LIBDIR /usr/local/netcdf4/lib
    setenv HDF5_LIBDIR /usr/local/hdf5/lib
\end{verbatim}
The \code{bash} equivalent is:
\begin{verbatim}
    export NETCDF_INCDIR=/usr/local/netcdf4/include
    export NETCDF_LIBDIR=/usr/local/netcdf4/lib
    export HDF5_LIBDIR=/usr/local/hdf5/lib
\end{verbatim}
\end{enumerate}
Perhaps you can see why we recommend using \code{nc-config}!

\subsubsection{Build scripts}

If you have more than one application (or more than one compiler),
you will get tired of editing the \code{makefile}.
One option is to have a \code{makefile} for each configuration, then
type:
\begin{verbatim}
    make -f makefile.circle_pgi
\end{verbatim}
for instance. Another option for keeping track of the user-defined
choices is in a \code{build script}. The advantage is that updates
to the \code{build scripts} are less frequent than updates to the
\code{makefile}. There are now two of these scripts in the \code{ROMS/Bin}
directory: \code{build.sh} (which is surprisingly a \code{csh} script)
and \code{build.bash}.  The \code{build scripts} use environment variables
to provide values for the list above, overwriting those found in the ROMS
\code{makefile}. Just as in the multiple \code{makefile} option, you will
need as many copies of the build script as you have applications. The
scope of these variables is local to the build script, allowing you to
compile different applications at the same time from the same sources
as long as each \code{\$(SCRATCH\_DIR)} is unique.

Both scripts have the same options:
\begin{klist}
  \kitem{-j \code{[N]}} Compile in parallel using \code{N} cpus,
  omit argument for all available CPUs.
  \kitem{-noclean}    Do not clean already compiled objects.
\end{klist}
Note that the default is to compile serially and to issue a
``\code{make clean}'' before compiling. It is left as an exercise
for the user if you prefer a different default behavior.

There are also a few variables which are not recognized by the ROMS
\code{makefile}, but are used locally inside the build script. These
are:
\begin{klist}
\kitem{MY\_PROJECT\_DIR} This is used in setting
\code{\$(SCRATCH\_DIR)} and \code{\$(BINDIR)}.

\kitem{MY\_ROMS\_SRC} Set the path to the user's local current ROMS source
code. This is used so that the script can be run from any directory,
not necessarily only from the top ROMS directory.
\end{klist}

\subsection{Running ROMS}
\label{Running}
ROMS expects to read a number of variables from an ASCII file
(details of the file are in \S\ref{ASCII_in}).
For serial or OpenMP execution, the syntax is:
\begin{verbatim}
       oceanS (or oceanO) < ocean.in > roms.out &
\end{verbatim}
while MPI execution requires:
\begin{verbatim}
       oceanM ocean.in > roms.out &
\end{verbatim}
so that each process can read the ocean.in file.

Realistically, you would only want to run relatively small
applications such as \code{UPWELLING} interactively on the command
line as shown here. Also, for either of the parallel options, you
will have to provide some information to ROMS and to the operating
system about how many threads or processes to use. Parallel
computers may also have some sort of batch queuing system in place
in which you would submit a job script. I have had easy access to a
number of systems over the years, each with its own style of job
script. Some MPI environments require that you run ROMS with:
\begin{verbatim}
     cd $PBS_O_WORKDIR
     mpirun -np 32 ./oceanM ocean_benchmark3.in
\end{verbatim}
while others need:
\begin{verbatim}
     aprun -np 32 ./oceanM ocean_benchmark3.in
\end{verbatim}
You just have to find out from the locals, for this as well as the syntax
of the batch queue scripts.

If all goes according to plan, ROMS will create both a collection
of NetCDF files and a verbose text file on standard out. There is a
large and ever-changing list of ways to view NetCDF files, but I like
\href{http://meteora.ucsd.edu/\~pierce/ncview\_home\_page.html}{\code{ncview}}
for a first view of what's going on.
Other tools include
\href{http://www.mathworks.com/products/matlab/}{Matlab},
\href{http://www.python.org/}{Python},
\href{http://www.ncl.ucar.edu/}{NCL},
\href{https://www.myroms.org/wiki/index.php/Plotting_Package_Installation}{Fortran}
and \href{http://www.ferret.noaa.gov/Ferret/}{ferret}.

If things don't go according to plan, the text output file is your
friend. Examine it carefully. If it fails on the \code{UPWELLING}
problem, you can compare your output to that in \S\ref{Output}. If it
fails on your application, make sure simple domains like \code{UPWELLING}
work on your system.

\subsection{Warnings and bugs}
ROMS is not a large program by some standards, but it is still complex
enough to require some effort to use effectively.
Some specific things to be wary of include:
\begin{itemize}
  \item It is recommended that you use 64 bits of precision rather
than 32 bits.
  \item The code must be run through the C preprocessor before it
is compiled.  This can occasionally be dangerous, especially with
the newer ANSI C versions of \code{cpp}.  Potential problems are listed
in Appendix \ref{Cpp}. The gnu \code{cpp} with the \code{-traditional} flag
is known to work well. On the other hand, the output of \code{cpp} is
saved in the \code{\$(SCRATCH\_DIR)}, where it can be examined.
  \item The vertical $\sigma$-coordinate was chosen as being a sensible
way to handle variations in the water depth as seen in the coastal
oceans. Changes to the code have allowed us to expand the well-behaved
range of depths and the range of values for \code{THETA\_S}, plus there
are some new vertical coordinate options.
  \item $\sigma$-coordinates have long had a bad reputation
because errors in the pressure gradient terms can lead to spurious
currents. These errors are much less troublesome than in the past due
to code improvements and there are more improvements possible
through an alternate equation of state \citep{SS2008c}, though
they are not yet in the myroms.org code. The pressure gradient
errors can also be controlled with some smoothing
of the bathymetry. This in turn changes the shape of the basin and
leads to its own set of problems, such as altered sill depths. Also, the
currents will react to the change in shelf slope---you are now solving a
different problem.
%You may want to explore a matlab tool for minimally
%smoothing the bathymetry found at:
%\href{http://www.liga.ens.fr/~dutour/Bathymetry/index.html}{http://www.liga.ens.fr/$\sim$dutour/Bathymetry/index.html}.
  \item I occasionally get the model to run for multiple
days/months, then suddenly blow up mid-depth. Just restart with a
smaller timestep and keep on going.
  \item There remain bugs in ROMS. If you find any, please report
them on the forum and/or the bug tracking system at myroms.org.
\end{itemize}