% $Id$

%\subsection{Background}

The ESMF I/O provides a unified interface for input and output of
high level ESMF objects such as Fields.  ESMF I/O capability is integrated
with third-party software such as \htmladdnormallink{Parallel I/O (PIO)}{https://github.com/NCAR/ParallelIO}
to read and write Fortran array data in NetCDF format, and 
\htmladdnormallink{JSON for Modern C++}{https://github.com/nlohmann/json}
Library to read Info attribute data in JSON format.  Other file I/O
functionalities, such as writing of error and log messages, input of
configuration parameters from an ASCII file, and lower-level I/O utilities are
covered in different sections of this document.  See the Log Class 
\ref{sec:Log}, the Config Class \ref{sec:Config}, and the Fortran 
I/O Utilities, \ref{sec:IOUtil} respectively.

%\subsection{I/O architecture}
%
%Future ESMF I/O development will include an ESMF\_IO object that combines
%other ESMF objects, such as Components, States, FieldBundles, ArrayBundles,
%Fields, Arrays, Grids, as well as Array and Info to perform robust and
%unified input/output system.
%


%\subsection{Data models}
%
%Earth system models use a variety of discrete grids to maintain information 
%about fields in continuous space, as well as observations. The primary ESMF 
%codes employ finite-difference and finite-volume grids, spectral grids, 
%unstructured land-surface grids, and ungridded observational networks.
%
%Fields within a model component are frequently defined on the same
%physical grid and are decomposed in memory in an identical fashion;
%that is, they share a distributed grid. They form a {\em bundle of
%fields} defined on the same distributed grid. The gridded data are
%supported by three ESMF elements: {\em PhysGrid} element 
%for physical grids, {\em DistGrid} element for distributed grids, and 
%{\em Fields} class for fields (\cite{ESMF-PhysGrid-Req},
%\cite{ESMF-DistGrid-Req}, \cite{ESMF-Field-Req}). 
%
%ESMF I/O will support input/output of data defined on all ESMF
%supported grids and location streams (\cite{ESMF-PhysGrid-Req},
%\cite{ESMF-DistGrid-Req}). For
%the purpose of this document, we will consider data belonging to three
%broad categories:
%
%\begin{description}
%
%\item[\bf Structured Gridded Data.] A {\em structured grid} is one on 
%which the relationship between gridpoints can be derived from their
%indices, without the need for an explicit map.  A simple example is fields
%defined on a rectangular lat/lon grid.
%
%\item[\bf Unstructured Gridded Data.] For the more general 
%{\em unstructured grid} the relationship between gridpoints cannot be
%derived from their indices, and the specification of an explicit map
%is necessary.  An example is a {\em catchment grid} used by some
%land-surface models.
%
%\item[\bf Observational Data on location streams.] As defined in 
%the {\em Physical Grid Requirements}, a location stream contains 
%a list of locations which 
%describe the measurements. Each observation is 
%associated with a spatial point or region. A neighbor relationship is not 
%defined for observations. 
%\end{description}
%
%As we have already mentioned, logically rectangular grids are naturally 
%represented by multi-dimensional arrays. The two latter data models can be 
%represented as one-dimensional arrays of structures with each structure 
%containing information about location, field values associated with this 
%location, and a list of neighbors, if relevant. 
%

\subsection{Data I/O}
\label{io:dataio}

\begin{sloppypar}
ESMF provides interfaces for high performance, parallel I/O using ESMF data
objects such as Arrays and Fields.  Currently ESMF only supports I/O of
NetCDF files.  The current ESMF implementation relies on the 
\htmladdnormallink{Parallel I/O (PIO)}{https://github.com/NCAR/ParallelIO}
library developed as a collaboration between NCAR and DOE laboratories.  PIO
is built as part of the ESMF build when the environment variable ESMF\_PIO is
set to "internal", or is linked against when ESMF\_PIO is set to "external"; by
default ESMF\_PIO is not set (which results in using the internal PIO if other
aspects of the ESMF build configuration allow it).  When PIO is built with ESMF,
the ESMF methods internally call the PIO interfaces.  When ESMF is not built with
PIO, the ESMF methods are non-operable (no-op) stubs that simply return with
a return code of ESMF\_RC\_LIB\_NOT\_PRESENT.  Details about the environment
variables can be found in ESMF User Guide, "Building and Installing the ESMF",
"Third Party Libraries".
\end{sloppypar}

The following methods support parallel data I/O using PIO:

\begin{description}
\item {\tt ESMF\_FieldBundleRead()}, section \ref{api:FieldBundleRead}.
\item {\tt ESMF\_FieldBundleWrite()}, section \ref{api:FieldBundleWrite}.
\item {\tt ESMF\_FieldRead()}, section \ref{api:FieldRead}.
\item {\tt ESMF\_FieldWrite()}, section \ref{api:FieldWrite}.
\item {\tt ESMF\_ArrayBundleRead()}, section \ref{api:ArrayBundleRead}.
\item {\tt ESMF\_ArrayBundleWrite()}, section \ref{api:ArrayBundleWrite}.
\item {\tt ESMF\_ArrayRead()}, section \ref{api:ArrayRead}.
\item {\tt ESMF\_ArrayWrite()}, section \ref{api:ArrayWrite}.
\end{description}


\subsection{Data formats}

The only format currently supported is NetCDF. The environment variables
ESMF\_NETCDF and/or ESMF\_PNETCDF must be set to enable this NetCDF-based I/O.
Details about the environment variables can be found in ESMF User Guide,
"Building and Installing the ESMF", "Third Party Libraries".

\begin{description}
\item[\bf NetCDF] Network Common Data Form (NetCDF) is an interface for 
array-oriented data access. The NetCDF library provides an
implementation of the interface. It also defines a 
machine-independent format for representing scientific data. Together,
the interface, library, and format support the creation, access, and
sharing of scientific data. The NetCDF software was developed at the
Unidata Program Center in Boulder, Colorado. See \cite{NetCDF3_UsersGuide_C}.
In geoscience, NetCDF can be naturally used to represent fields
defined on logically rectangular grids. NetCDF use in geosciences is 
specified by CF conventions mentioned above \cite{NetCDF_CF}.

To the extent that data on unstructured grids (or even observations) can be 
represented as one-dimensional arrays, NetCDF can also be used to store these 
data. However, it does not provide a high-level abstraction for this type of 
data. 

\end{description}

%Modern data management approaches could potentially provide significant 
%advantages in manipulating data and have to be carefully studied.
%For example, ESMWF has created and employed relational-database based 
%Observational Data Base (ODB) software \cite{ODB}.  However, such complex 
%data management systems are beyond the scope of the basic ESMF I/O. 
