Skip to content

Commit f1f3bc7

Browse files
committed
rewording sneak_peek.md
1 parent 9e4bf2b commit f1f3bc7

1 file changed

Lines changed: 61 additions & 49 deletions

File tree

sneak_peek.md

Lines changed: 61 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,15 @@ This is essentially a placeholder for the next release note ...
44

55
* New features
66
+ A single read/write request made by an MPI process is now allowed to be of
7-
size larger than 2 GiB. Such large requests are passed to the MP-IO
7+
size larger than 2 GiB. Such large requests will be passed to the MP-IO
88
library. This feature makes use of "the large count feature" introduced in
9-
MPI standard 4.0, which includes `MPI_XXX_c` APIs whose arguments are of type
10-
`MPI_Count`. `MPI_Count` can be an 8-byte integer type, enabling large MPI
11-
operations. As some MPI libraries today have begun implementing MPI 4.0,
12-
PnetCDF now can rely on the MPI libraries to support large single requests.
13-
When the MPI library used to build PnetCDF does not support large requests,
14-
PnetCDF will pass the MPI errors to the users. Because of this change, the
15-
PnetCDF configure option `--enable-large-single-req` is thus deprecated.
9+
MPI standard 4.0, which includes `MPI_XXX_c` APIs whose arguments are of
10+
type `MPI_Count`. `MPI_Count` can be an 8-byte integer type, enabling large
11+
MPI operations. As some MPI libraries today have begun implementing MPI
12+
4.0, PnetCDF now can rely on the MPI libraries to support large single
13+
requests. When the MPI library used to build PnetCDF does not support large
14+
requests, the MPI errors are returned. Because of this change, the PnetCDF
15+
configure option `--enable-large-single-req` is thus deprecated.
1616
See [PR #131](https://github.com/Parallel-NetCDF/PnetCDF/pull/131)
1717
+ Flexible APIs now can operate as high-level APIs, when argument `bufcount`
1818
is set to `NC_COUNT_IGNORE` and `buftype` is set to an MPI predefined data
@@ -30,20 +30,23 @@ This is essentially a placeholder for the next release note ...
3030
+ none
3131
3232
* New Limitations
33-
+ Hint `nc_header_read_chunk_size` is limited to `NC_MAX_INT`. PnetCDF reads
34-
file header in chunks. This hint customizes the chunk size.
33+
+ Hint `nc_header_read_chunk_size`, introduced in version 1.4.0, is now
34+
limited to `NC_MAX_INT`. As PnetCDF reads file header in chunks, this hint
35+
can be used to customize the chunk size. The default is 256 KB.
36+
See [4209056](https://github.com/Parallel-NetCDF/PnetCDF/commit/4209056e9a66465421f7ce9f1b44518923638b04)
3537
3638
* Configure options
37-
+ `--enable-large-single-req` has been deprecated and removed, as PnetCDF now
39+
+ `--enable-large-single-req` is deprecated and removed, as PnetCDF now
3840
allows a single reqd/write request of size larger than 2 GiB.
39-
+ `--disable-file-sync` is now deprecated and removed. This configure option
41+
+ `--disable-file-sync` is deprecated and removed. This configure option
4042
alone was not able to provide a sufficient data consistency. Users are
4143
suggested to call `ncmpi_sync` and `MPI_Barrier` to achieve a desired
4244
consistency, as suggested by MPI standard.
4345
+ A new option `--enable-install-examples` installs the example programs
4446
under folder `${prefix}/pnetcdf_examples` along with run script files. An
4547
example is `${prefix}/pnetcdf_examples/C/run_c_examples.sh`. The default of
4648
this option is `disabled`.
49+
See [PR #91](https://github.com/Parallel-NetCDF/PnetCDF/pull/91)
4750
+ Add three new environment variables `SEQ_CFLAGS`, `SEQ_LDFLAGS` and
4851
`SEQ_LIBS` for setting the compile, link, and library flags, respectively
4952
to be used to build the sequential utility programs, i.e. `cdfdiff`,
@@ -57,7 +60,8 @@ This is essentially a placeholder for the next release note ...
5760
Fortran 10 and later.
5861
See [PR #114](https://github.com/Parallel-NetCDF/PnetCDF/pull/114)
5962
+ Handle the case when MPICC environment variable is not set and `--with-mpi`
60-
is not used. See commit 6142135.
63+
is not used. See commit
64+
[6142135](https://github.com/Parallel-NetCDF/PnetCDF/commit/61421356ecd38878a4ef46771ed6520d4257251f)
6165
+ Upgrade Autotools version requirement to autoconf 2.71, automake 1.16.5, and
6266
libtool 2.4.6. (Note this change affects PnetCDF developers only.)
6367
See [PR #95](https://github.com/Parallel-NetCDF/PnetCDF/pull/95)
@@ -71,6 +75,7 @@ This is essentially a placeholder for the next release note ...
7175
is `NC_COUNT_IGNORE`, `buftype` must be a predefine MPI datatype and the
7276
APIs operate as the high-level APIs. Fortran equivalents are
7377
`NF_COUNT_IGNORE` and `NF90_COUNT_IGNORE`.
78+
See [PR #92](https://github.com/Parallel-NetCDF/PnetCDF/pull/92)
7479
7580
* New APIs
7681
+ none
@@ -81,6 +86,7 @@ This is essentially a placeholder for the next release note ...
8186
* API semantics updates
8287
+ File open flag `NC_SHARE` is now deprecated. It is still defined, but takes
8388
no effect.
89+
See [PR #119](https://github.com/Parallel-NetCDF/PnetCDF/pull/119)
8490
+ `NC_SHARE` alone is not sufficient to provide data consistency for accessing
8591
a shared file in parallel and thus is now deprecated. PnetCDF follows the
8692
file consistency defined in MPI standard, which only addresses the case
@@ -97,32 +103,35 @@ This is essentially a placeholder for the next release note ...
97103
* New error code
98104
+ none
99105
100-
* New PnetCDF hint
101-
+ `nc_hash_size_dim` sets the hash table size for dimension names. Default: 256
102-
+ `nc_hash_size_var` sets the hash table size for variable names. Default: 256
106+
* New PnetCDF hints
107+
+ `nc_hash_size_dim` sets the hash table size for dimension names.
108+
Default: 256
109+
+ `nc_hash_size_var` sets the hash table size for variable names.
110+
Default: 256
103111
+ `nc_hash_size_gattr` sets the hash table size for global attribute names.
104112
Default: 64
105113
+ `nc_hash_size_vattr` sets the hash table size for variable attribute names.
106114
Default: 8
107-
+ The above 4 new hints allow users to set different hash table sizes for
108-
different objects. Hashing tables are used for quick data object name
109-
lookup. It can be useful for files containing a large number of
110-
dimensions, variables, and attributes. For instance, when the number of
111-
variables to be defined is large and the number of attributes per variable
112-
is small, increasing the value of `nc_hash_size_var` can speed up the
113-
variable definition and inquiring time. On the other hand, reducing the
114-
value of `nc_hash_size_vattr` can reduce the memory footprint. See
115-
[PR #132](https://github.com/Parallel-NetCDF/PnetCDF/pull/132).
115+
+ The above 4 new hints can be used to set different hash table sizes for
116+
dimensions, variables, and attributes. Hashing tables are used for quick
117+
data object name lookup. It can be useful for files containing a large
118+
number of dimensions, variables, and attributes. For instance, when the
119+
number of variables to be defined is large and the number of attributes per
120+
variable is small, increasing the value of `nc_hash_size_var` can speed up
121+
the variable definition and inquiring time. On the other hand, setting a
122+
smaller value for hint `nc_hash_size_vattr` can reduce memory footprint.
123+
See [PR #132](https://github.com/Parallel-NetCDF/PnetCDF/pull/132).
116124
117125
* New run-time environment variables
118126
+ none
119127
120128
* Build recipes
121129
+ none
122130
123-
* Updated utility program
124-
+ `ncvalidator` - When the file size is larger or smaller than expected, the
125-
file may still be a valid netCDF file.
131+
* Updated utility programs
132+
+ `ncvalidator` - When the file size is larger or smaller than what is
133+
calculated based on the metadata stored in the file header, the file may
134+
still be a valid netCDF file.
126135
* The larger-than-expected case can happen if opening an existing file that
127136
contains no variable. Deleting a global attribute already defined in the
128137
file will reduce the file header size. In this case, the file is still a
@@ -134,51 +143,53 @@ This is essentially a placeholder for the next release note ...
134143
sizes of all variables. In this case, the file is still a valid netCDF
135144
file, and `ncvalidator` will report a warning, rather than an error.
136145
* Print the dimension size of a variable on stdout when its size is larger
137-
than the limitation allowed by the file format. See commit 5584d44.
146+
than the limitation allowed by the file format. See commit
147+
[5584d44](https://github.com/Parallel-NetCDF/PnetCDF/commit/5584d44a433a68966b0be601e7a73e939c695dbf)
138148
+ Add file src/utils/README.md which gives short descriptions of the utility
139149
programs and collapsible bullets to display their manual pages.
140150
141151
* Other updates:
142-
+ When file header extent size grows, PnetCDF now uses 64 MiB as the data
143-
movement unit size per process.
152+
+ When file header extent size grows, PnetCDF now uses a movement unit per
153+
process of size up to 64 MiB.
144154
See [PR #137](https://github.com/Parallel-NetCDF/PnetCDF/pull/137)
145155
+ Since version 1.1.0, PnetCDF has been using file striping size, if
146156
obtainable from the MPI-IO hint `striping_unit`, to align the starting file
147157
offset of the data section. This offset is also referred to as the file
148158
header extent, which can be larger than the header size to allow header to
149-
grow later on when new data objects are added. Starting from this release,
150-
file stripe size is no longer used for setting the starting offset of the
151-
data section. This is because automatically setting file header extent
152-
using the file striping size may grow the file header unexpectedly when
153-
adding new objects to an existing file.
159+
grow when new data objects are added. Starting from this release, file
160+
stripe size is no longer used for setting the starting offset of the data
161+
section. This is because automatically setting file header extent using the
162+
file striping size may grow the file header unexpectedly when adding new
163+
objects to an existing file.
154164
See [PR #124](https://github.com/Parallel-NetCDF/PnetCDF/pull/124) and
155165
[PR #125](https://github.com/Parallel-NetCDF/PnetCDF/pull/125).
156166
+ Use unsigned int to perform byte swap.
157167
See [PR #113](https://github.com/Parallel-NetCDF/PnetCDF/pull/113).
158168
+ Silence Intel icc compilation warnings: when CFLAGS contains
159169
"-Wimplicit-const-int-float-conversion" and "-Wstringop-overread".
160170
See [PR #110](https://github.com/Parallel-NetCDF/PnetCDF/pull/110).
161-
+ In all previous PnetCDF's implementations, file header is always
162-
written/read by rank 0 using MPI independent APIs. This can nullify ROMIO
163-
hint `romio_no_indep_rw` if set by the user. To warrant no independent
164-
read/write, PnetCDF now first checks hint `romio_no_indep_rw` and if set to
165-
`true`, then all file header I/Os are done using MPI collective I/O calls,
166-
where only rank 0 makes non-zero length requests while all others zero
167-
length (in order to participate the collective calls). See
171+
+ In all previous PnetCDF's implementations, file header is always written/
172+
read by rank 0 using MPI independent APIs. This can nullify ROMIO hint
173+
`romio_no_indep_rw` if set by the user. To warrant no independent read/
174+
write user hint, PnetCDF now checks hint `romio_no_indep_rw` and if set to
175+
`true`, then all file header I/Os are made through MPI collective I/O
176+
calls, where only rank 0 makes non-zero length requests while all others
177+
zero length (in order to participate the collective calls). See
168178
[PR #104](https://github.com/Parallel-NetCDF/PnetCDF/pull/104) and
169179
[PR #138](https://github.com/Parallel-NetCDF/PnetCDF/pull/138).
170180
+ In all prior versions, the file name was checked whether it contains
171181
character ':'. The prefix name ending with ':' is considered by ROMIO as
172182
the file system type name. The prefix name, if found, is then stripped, so
173-
the file name can be used in the successive POSIX function calls. However,
183+
the file name can be used in the POSIX function calls internally. However,
174184
the prefix was not checked against the file system type names recognized
175185
by ROMIO. Starting from this release, the prefix is checked against the
176186
known file system type names to ROMIO. If the prefix is not one of the
177187
recognized types, e.g. "ufs", "nfs", "xfs", "pvfs2", "gpfs", "panfs",
178188
"lustre", "daos", "testfs", "ime", or "quobyte", then the prefix name is
179189
not stripped. This change is for the case when the file name contains ':',
180-
but it is not for specifying the file system type.
181-
See [PR #79](https://github.com/Parallel-NetCDF/PnetCDF/pull/79).
190+
but it is not for specifying the file system type. See
191+
[PR #79](https://github.com/Parallel-NetCDF/PnetCDF/pull/79) and
192+
[MPICH PR 5951](https://github.com/pmodels/mpich/pull/5951).
182193
183194
* Bug fixes
184195
+ Return I/O hints that are actually used. See commit
@@ -227,10 +238,11 @@ This is essentially a placeholder for the next release note ...
227238
+ test/testcases/flexible_var.c - tests flexible var API
228239
+ test/testcases/flexible_api.f - tests flexible API when `bufcount == -1`
229240
+ test/testcases/scalar.c - adds tests for scalar variables using nonblocking
230-
APIs. See commit 07ff7b1
241+
APIs. See commit
242+
[07ff7b1](https://github.com/search?q=repo%3AParallel-NetCDF%2FPnetCDF+07ff7b1&type=commits)
231243
+ test/nonblocking/test_bputf.f90, test/nonblocking/test_bputf77.f -
232-
add tests of APIs `inq_buffer_usage` and `inq_buffer_size`.
233-
See commit 94ce438
244+
add tests of APIs `inq_buffer_usage` and `inq_buffer_size`. See commit
245+
[94ce438](https://github.com/Parallel-NetCDF/PnetCDF/commit/94ce438262fe7fcade031dcae1a677a827549bb3)
234246
235247
* Issues with NetCDF library
236248
+ none

0 commit comments

Comments
 (0)