update for release of 1.11.0

Author: wkliao

Release/pnetcdf-1.11.0.tar.gz

@@ -0,0 +1,222 @@
+[PnetCDF](https://parallel-netcdf.github.io) Version 1.11.0 Release Notes (December 19, 2018)
+------------------------------------------------------------------------------
+
+* New features
+  + NetCDF-4 driver -- Accessing HDF5-based NetCDF-4 files is now supported.
+    PnetCDF can be built on top of NetCDF-4 library to let users to use PnetCDF
+    APIs to read and write a NetCDF-4 file. Users now can add NC_NETCDF4 flag
+    when calling ncmpi_create() to create NetCDF-4 files. For opening NetCDF-4
+    files, no additional flag is needed, as PnetCDF automatically detects the
+    file format and uses the HDF5 I/O driver underneath. This feature is
+    provided for convenience purpose. The parallel I/O performance to NetCDF-4
+    files is expected no difference from using NetCDF-4 library directly.
+  + Per-file thread-safe capability is added. This feature can be enabled at
+    configure time by adding command-line option `--enable-thread-safe`. In
+    addition, option `--with-pthread` can be used to specify the install path
+    to the pthreads library. This feature currently only supports
+    one-thread-per-file I/O operations and the classic CDF-1, 2, and 5 files.
+
+* New optimization
+  + On some systems, e.g. Cori @NERSC, collective MPI-IO may perform poorly
+    when the I/O buffer is noncontiguous, compared to a contiguous one. To
+    avoid this, `ncmpi_wait()` and `ncmpi_wait_all()` now check whether the
+    buffer is noncontiguous and size is less than 16 MiB. If both are true, a
+    temporary contiguous buffer is allocated to copy the data over and used in
+    the MPI read or write calls. The size of the buffer can be adjusted through
+    a new hint `nc_ibuf_size`. See `New PnetCDF hint` below and
+    [PR #26](https://github.com/Parallel-NetCDF/PnetCDF/pull/26). Programs
+    developed to test this issue is available in
+    https://github.com/Parallel-NetCDF/E3SM-IO/tree/master/mpi_io_test
+  + Burst buffer driver is updated to run varn APIs more efficiently. Previous
+    implementation breaks a single varn request into multiple vara requests,
+    which can be slow and require a large amount of meta data. It has changed
+    to consider each varn request a single entity. See
+    [PR #30](https://github.com/Parallel-NetCDF/PnetCDF/pull/30) and
+    [PR #31](https://github.com/Parallel-NetCDF/PnetCDF/pull/31).
+
+* New Limitations
+  + For creating new files, the NetCDF-4 driver in PnetCDF supports only the
+    classic model I/O operations. Advanced NetCDF-4 features, such as chunking,
+    compression, etc. are not supported in PnetCDF. This is due to the
+    unavailability of PnetCDF APIs for those operations.
+  + The burst buffering driver does not support NetCDF-4 file formats.
+  + Due to a bug in HDF5 1.10.2 that fails zero-length write requests to record
+    variables in the collective mode, PnetCDF is not able to support such
+    requests when NetCDF-4 feature is enabled. See discussion in
+    https://github.com/NCAR/ParallelIO/pull/1304
+    The bug fix has appeared in HDF5 1.10.4 release.
+
+* Update configure options
+  + Enable NetCDF-4 support.
+    - `--enable-netcdf4`: enable NetCDF4 format classic mode support
+    - `--with-netcdf4=/path/to/netcdf-4`: path to NetCDF-4 library installation
+  + Enable multi-threading support.
+    - `--enable-thread-safe`: enable per-file thread-safe support
+    - `--with-pthread`: path to the pthread library installation
+
+* New constants
+  + none
+
+* New APIs
+  + C++ API `NcmpiFile::set_fill()` is added for setting and inquiring the
+    fill mode of an opened NetCDF file.
+
+* API syntax changes
+  + none
+
+* API semantics updates
+  + none
+
+* New error code precedence
+  + none
+
+* Updated error strings
+  + none
+
+* New error code
+  + none
+
+* New PnetCDF hint
+  + `nc_ibuf_size` -- to set the size of a temporal buffer to be allocated by
+    PnetCDF internally to pack noncontiguous user write buffers supplied to the
+    nonblocking requests into a contiguous space. Similarly for read case to
+    unpack the temporal buffer to user read buffers, if they are noncontiguous.
+    This affects both blocking and nonblocking APIs. On some systems, using
+    noncontiguous user buffers in MPI collective read/write functions performs
+    significantly worse than using contiguous buffers. Note if the size of
+    aggregated user buffers is larger than `nc_ibuf_size`, packing/unpacking
+    will be disabled to save memory footprint. The default value is 16 MiB.
+
+* New run-time environment variables
+  + none
+
+* Build recipes
+  + doc/README.NetCDF4.md is added to describe the usage of the new feature of
+    NetCDF-4 support.
+
+* New/updated utility program
+  + none
+
+* Other updates:
+  + The automatic file layout alignment for fixed-size variables is disabled.
+    This is because modern MPI-IO implementations have already aligned the file
+    access with the file lock boundaries and the automatic alignment can create
+    a file view with "holes" in between variables, which can adversely degrade
+    I/O performance. Users can still set hints `nc_header_align_size`,
+    `nc_var_align_size`, and `nc_record_align_size` to use customized alignment
+    sizes.
+  + The internal data buffering mechanism used in the burst buffer driver is
+    removed. This mechanism caches the request data in memory until the
+    accumulated size is more than 8 MiB, so the write requests to burst buffers
+    can be aligned with 8 MiB boundaries. However, experiments on Cray DataWarp
+    show a negligible performance improvement unless the I/O request is small
+    and fragment. On the other hand, it can degrade performance for mid- and
+    large-sized requests. The burst buffer driver now writes directly to the
+    burst buffers for each user write request.
+
+* Bug fixes
+  + Fix bug of checking interleaved requests for scalar variables. See
+    [PR #27](https://github.com/Parallel-NetCDF/PnetCDF/pull/27).
+  + When building PnetCDF using the IBM xlc compiler with -O optimization
+    option on Little Endian platforms, users may encounter errors related to
+    strict ANSI C aliasing rules. Thanks to Jim Edwards for reporting and Rafik
+    Zurob for providing the fix. See
+    [Issue #23](https://github.com/Parallel-NetCDF/PnetCDF/issues/23) and
+    [Pull Request #24](https://github.com/Parallel-NetCDF/PnetCDF/issues/24).
+  + Shell ksh has a different way to redirect stdout and stderr from bash.
+    PnetCDF configure.ac and acinclude.m4 have been developed mainly on bash.
+    This bug can cause configure command to fail when using ksh. Thanks to
+    @poohRui for reporting the bug. See
+    [Issue #21](https://github.com/Parallel-NetCDF/PnetCDF/issues/21) and
+    [PR #22](https://github.com/Parallel-NetCDF/PnetCDF/pull/22).
+    However, running configure under ksh is still buggy. A GNU automake bug
+    report of hanging problem can be found in
+    https://lists.gnu.org/archive/html/bug-automake/2015-04/msg00000.html
+    PnetCDF users are recommended to run configure under other shells.
+  + For put and get APIs when buftype is MPI_DATATYPE_NULL, bufcount is
+    ignored. This is not implemented correctly in blocking put and get APIs.
+    See bug fix committed on Aug. 25, 2018.
+  + ncmpidiff -- when comparing two files that contain record variables but
+    no record has been written. See bug fix committed on Aug. 25, 2018.
+  + ncmpidiff -- when comparing two scalar variables, error NC_EBADDIM may
+    mistakenly reported. See bug fix committed on Aug. 12, 2018.
+  + When the MPI communicator used in ncmpi_create or ncmpi_open is freed by
+    the user after the call and before file is closed, programs would crash at
+    ncmpi_close with MPI error of "Invalid communicator". The fix moves the
+    duplication of MPI communicator to the place before calling driver create
+    and open subroutines. See bug fix committed on Jul 21, 2018.
+
+* New example programs
+  + examples/C/time_var.c and examples/F77/time_var.f - show how to define,
+    write, and read record variables.
+  + examples/C/pthread.c - demonstrates the one-file-per-thread I/O example.
+    When running on some parallel machines, users may need to set certain
+    environment variable to enable MPI multi-threading support, for example on
+    Cori @NERSC with command
+    ```
+    export MPICH_MAX_THREAD_SAFETY=multiple
+    ```
+  + examples/C/transpose2D.c - a 2D version of examples/C/transpose.c
+
+* New programs for I/O benchmarks
+  + none
+
+* New test program
+  + test/F90/test_fill.f90 - another test for bug fix r3730.
+  + test/testcases/error_precedence.m4 - tests the error code reporting
+    precedence
+  + test/nc4/tst_zero_req.c - tests a HDF5 1.10.2 bug that causes test program
+    to hang when writing to and reading back a 2D record variable in collective
+    mode with some of the processes making zero-length requests.
+  + test/nc4/put_get_all_kinds.m4 - tests all supported variable read/write
+    API. Make sure they are properly wired up
+  + test/nc4/interoperability_rd.m4 - tests whether NetCDF-4 file written using
+    NetCDF can be read by PnetCDF
+  + test/nc4/interoperability_wr.m4 - tests whether NetCDF-4 file written using
+    PnetCDF can be read by NetCDF
+  + test/nc4/simple_xy.c - tests reading NetCDF-4 files, borrowed the test
+    program simple_xy.c from NetCDF
+  + test/testcases/tst_pthread.c - tests thread-safe capability for scenario of
+    each thread operating on a unique file.
+  + test/testcases/tst_free_comm.c - free MPI communicator right after calling
+    ncmpi_create to see if PnetCDF duplicates the communicator correctly.
+
+* Conformity with NetCDF library
+  + none
+
+* Discrepancy from NetCDF library
+  + In contract to NetCDF-4 which allows to read/write variables in define mode
+    when the file format is in NetCDF-4 format, PnetCDF still requires reading
+    and writing variables in data mode.
+  + In contrast to the semantics of nc_set_fill() defined in NetCDF-4,
+    ncmpi_set_fill() changes the fill mode of all variables newly defined in
+    the current scope of defined mode. Variables affected include the ones
+    defined before and after the call to ncmpi_set_fill(). Note this API has no
+    effect on the already existing variables created in the previous define
+    mode. This behavior follows the convention adopted by NetCDF-3. To change
+    fill mode for individual variables after the call to ncmpi_set_fill(), API
+    ncmpi_def_var_fill() can be used for this purpose. Refer NetCDF 4.1.3 user
+    guide for semantics of
+    [nc_set_fill()](https://www.unidata.ucar.edu/software/netcdf/documentation/historic/netcdf-c/nc_005fset_005ffill.html).
+    A discussion with NetCDF developers regarding this issue can be found in
+    [1114](https://github.com/Unidata/netcdf-c/pull/1114).
+  + The error code return precedence can be different between NetCDF and
+    PnetCDF in some cases. A test program for error code return precedence is
+    available in test/testcases/error_precedence.m4. This program can be used
+    to test both PnetCDF and NetCDF libraries. Note when testing NetCDF
+    programs, because NetCDF does not follow the same precedence, failures are
+    expected. A discussion with NetCDF developers regarding this issue can be
+    found in [334](https://github.com/Unidata/netcdf-c/issues/334).
+
+* Issues related to MPI library vendors:
+  + none
+
+* Issues related to Darshan library:
+  + none
+
+* Clarifications
+  + PnetCDF currently does not support Fortran default integer type set to 8
+    bytes (for GNU Fortran compiler, this change of default setting is done by
+    using compile option -fdefault-integer-8). Checking this has been added
+    and configure command will fail, once default 8-byte integer is detected.
+

Release_notes/1.11.0.md

@@ -21,7 +21,7 @@ <h1>PnetCDF: A Parallel I/O Library for NetCDF File Access</h1>
 
 <h2 id="News">News</h2>
 <ul>
-<li><strong>December 4, 2018</strong>: Pre-release of 1.11.0 is available.</li>
+<li><strong>December 19, 2018</strong>: Release of 1.11.0 is available.</li>
 <li><strong>November 2, 2018</strong>: The official PnetCDF project web pages
 have migrated to github.com from its previous location at <a
 href=https://trac.mcs.anl.gov/projects/parallel-netcdf>

index.html

@@ -33,7 +33,7 @@
 <center> <h2>PnetCDF Source Code Downloads</h2> </center>
 <br>
 
-The latest stable release of <a href="../index.html">PnetCDF</a> is version 1.10.0.
+The latest stable release of <a href="../index.html">PnetCDF</a> is version 1.11.0.
 <p>
 We encourage PnetCDF users to use the latest released version. If for some
 reason you would like to try out older versions, below is a table of (most of)
@@ -48,15 +48,15 @@
    <th>SHA-1 Checksum</th>
  </tr>
  <tr>
-   <td align=center>December 4, 2018</td>
-   <td align=center><a href=https://github.com/Parallel-NetCDF/PnetCDF/blob/master/sneak_peek.md>1.11.0.pre</a></td>
-   <td><a href="../Release/pnetcdf-1.11.0.pre.tar.gz" onClick="var that=this; ga('send', 'event', 'PnetCDF', 'Source Download (GZ)', 'pnetcdf-1.11.0.pre.tar.gz', 1); ga('pnetcdfTracker.send', 'event', 'PnetCDF', 'Source Download (GZ)', 'pnetcdf-1.11.0.pre.tar.gz', 1); ga('parallelnetcdfTracker.send', 'event', 'PnetCDF', 'Source Download (GZ)', 'pnetcdf-1.11.0.pre.tar.gz', 1); setTimeout(function(){location.href=that.href;},500); return false;">pnetcdf-1.11.0.pre.tar.gz</a></td>
+   <td align=center>December 19, 2018</td>
+   <td align=center><a href=../Release_notes/1.11.0.md>1.11.0</a></td>
+   <td><a href="../Release/pnetcdf-1.11.0.tar.gz" onClick="var that=this; ga('send', 'event', 'PnetCDF', 'Source Download (GZ)', 'pnetcdf-1.11.0.tar.gz', 1); ga('pnetcdfTracker.send', 'event', 'PnetCDF', 'Source Download (GZ)', 'pnetcdf-1.11.0.tar.gz', 1); ga('parallelnetcdfTracker.send', 'event', 'PnetCDF', 'Source Download (GZ)', 'pnetcdf-1.11.0.tar.gz', 1); setTimeout(function(){location.href=that.href;},500); return false;">pnetcdf-1.11.0.tar.gz</a></td>
    <td>2.1 MB</td>
-   <td>bf9dd8a53b5b8caf0ed0cb7075dadbc70b8f908b</td>
+   <td>a13fd24953abce1f3fd7aa94fd66f1b9895182de</td>
  </tr>
  <tr>
    <td align=center>July 2, 2018</td>
-   <td align=center><a href=../Release_notes/1.10.0>1.10.0</a></td>
+   <td align=center><a href=../Release_notes/1.10.0.md>1.10.0</a></td>
    <td><a href="../Release/parallel-netcdf-1.10.0.tar.gz" onClick="var that=this; ga('send', 'event', 'PnetCDF', 'Source Download (GZ)', 'parallel-netcdf-1.10.0.tar.gz', 1); ga('pnetcdfTracker.send', 'event', 'PnetCDF', 'Source Download (GZ)', 'parallel-netcdf-1.10.0.tar.gz', 1); ga('parallelnetcdfTracker.send', 'event', 'PnetCDF', 'Source Download (GZ)', 'parallel-netcdf-1.10.0.tar.gz', 1); setTimeout(function(){location.href=that.href;},500); return false;">parallel-netcdf-1.10.0.tar.gz</a></td>
    <td>2.0 MB</td>
    <td>9ac144f305e14ae6306f5e51af1b4329bfb86480</td>