Skip to content

Commit d72ea39

Browse files
wkliaowkliao
committed
2 parents 6f444fa + ad3b21d commit d72ea39

1 file changed

Lines changed: 171 additions & 0 deletions

File tree

‎doc/netcdf4.html‎

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
2+
<html xmlns="http://www.w3.org/1999/xhtml">
3+
<head>
4+
<title>
5+
Using DataWarp as Burst Buffers in PnetCDF
6+
</title>
7+
<script>
8+
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
9+
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
10+
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
11+
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
12+
ga('create', 'UA-45938012-1', 'auto');
13+
ga('create', 'UA-45937491-1', 'auto', {'name': 'pnetcdfTracker'});
14+
ga('create', 'UA-46002884-1', 'auto', {'name': 'parallelnetcdfTracker'});
15+
ga('send', 'pageview');
16+
ga('pnetcdfTracker.send', 'pageview');
17+
ga('parallelnetcdfTracker.send', 'pageview');
18+
</script>
19+
</head>
20+
<body>
21+
<h2 id="ParallelnetCDF:AParallelIOLibraryforNetCDFFileAccess">Using PnetCDF on
22+
NetCDF-4 files</h2>
23+
PnetCDF has a built in I/O driver to handle NetCDF-4
24+
file. THe driver is a convenience wrapper around the NetCDF API.
25+
Through this driver, existing PnetCDF application can easily access NetCDF-4
26+
formated files without rewriting the program using NetCDF or HDF5.&nbsp;
27+
28+
<h3 id="BuildNC4">Enable the NetCDF4 Driver</h3>
29+
To build PnetCDF with NetCDF4&nbsp; support, add &quot;<tt>--enable-</tt>netcdf4&quot; option at the configure command line.<p>
30+
NetCDF and HDF5 library built with parallel I/O support is required to build
31+
the NetCDF4 driver. Use "--wht-netcdf4=&lt;path&gt;" and "--with-hdf5=&lt;path&gt;" to
32+
provide location of NetCDF and HDF5 library.</p>
33+
<pre>
34+
./configure --prefix=/path/to/install --enable-netcdf4 --with-netcdf=/path/to/netcdf/install --with-hdf5=/path/to/hdf5/install
35+
</pre>
36+
The remaining steps are the same as usual, to build PnetCDF library, run:
37+
<pre>
38+
make
39+
</pre>
40+
To install PnetCDF, run:
41+
<pre>
42+
make install
43+
</pre>
44+
45+
<h3 id="BuildNC">To build NetCDF for NetCDF4 driver </h3>
46+
Source tar ball of NetCDF-4 can be downloaded from URL:
47+
<a href="https://github.com/Unidata/netcdf-c/releases">https://github.com/Unidata/netcdf-c/releases</a>
48+
<p>version 4.6.2 or later is recommended
49+
</p>
50+
<p>Download and run:
51+
</p>
52+
53+
<pre>
54+
gzip -dc v4.6.2.tar.gz | tar -xf -
55+
cd netcdf-c-4.6.2
56+
./configure --prefix=/NetCDF4/install/path \
57+
--enable-netcdf-4 \
58+
CC=mpicc \
59+
CPPFLAGS="-I/HDF5/install/path/include" \
60+
LDFLAGS="-L/HDF5/install/path/lib"
61+
make install</pre>
62+
63+
<h3 id="BuildHDF">To build HDF5 for NetCDF4 driver </h3>
64+
Source tar ball of HDF5 can be downloaded from URL:
65+
<a href="https://www.hdfgroup.org/downloads/hdf5/source-code/">https://www.hdfgroup.org/downloads/hdf5/source-code/</a>
66+
<p>Download and run:
67+
</p>
68+
69+
<pre>
70+
gzip -dc hdf5-1.10.4.tar.gz | tar -xf -
71+
cd hdf5-1.10.4
72+
./configure --prefix=/HDF5/install/path \
73+
--enable-parallel=yes \
74+
CC=mpicc FC=mpifort CXX=mpicxx
75+
make install
76+
</pre>
77+
78+
79+
<h3 id="AcccessNC4">Accessing NetCDF-4 file</h3>
80+
To create a NetCDF-4 file, add the flag NC_NETCDF4 into argument cmode when calling ncmpi_create(). For example,
81+
<pre>
82+
int cmode;
83+
cmode = NC_CLOBBER | NC_NETCDF4;
84+
ncmpi_create(MPI_COMM_WORLD, "testfile.nc", cmode, MPI_INFO_NULL, &ncid);
85+
</pre>
86+
or
87+
<pre>
88+
cmode = NC_CLOBBER | NC_NETCDF4 | NC_CLASSIC_MODEL;
89+
ncmpi_create(MPI_COMM_WORLD, "testfile.nc", cmode, MPI_INFO_NULL, &ncid);
90+
</pre>
91+
NC_NETCDF4 flag is not required when opening an existing file in NetCDF-4 format, as PnetCDF checks the file format and selects the proper I/O driver.
92+
Users can also set the default file format to NetCDF-4 by calling API ncmpi_set_default_format() with argument NC_FORMAT_NETCDF4C or NC_FORMAT_NETCDF4_CLASSIC. For example,
93+
<pre>
94+
ncmpi_set_default_format(NC_FORMAT_NETCDF4, &old_formatp);
95+
</pre>
96+
or
97+
<pre>
98+
ncmpi_set_default_format(NC_FORMAT_NETCDF4_CLASSIC, &old_formatp);
99+
</pre>
100+
When no file format specific flag is set in argument cmode, PnetCDF will use the default setting.
101+
102+
103+
<h3 id="Issue">Known Issues</h3>
104+
Some features are not supported due to the availability of APIs different between PnetCDF and NetCDF libraries. I/O semantics are also slightly different.
105+
<ul>
106+
<li>API families of vard, varn, and nonblocking I/O are not supported. This is because NetCDF does not have corresponding APIs. Error code NC_ENOTSUPPORT will be returned. For vard APIs, NetCDF does not have APIs that allow accessing the file directly by an MPI derived file type. For varn APIs, a potential solution is to split the request into into multiple vara calls. However, such solution must deal with the situation when the numbers of requests are different among processes in the collective data mode. Supporting varn is thus a future work.</li>
107+
<li>Although a new file of format NC_FORMAT_NETCDF4 or NC_FORMAT_NETCDF4_CLASSIC can be created, the I/O operations are limited to the NetCDF classic model I/O, This is because PnetCDF APIs do not include those for operating enhanced data objects, such as groups, compound data types, compression etc. As for reading a NetCDF-4 file created by NetCDF-4, PnetCDF supports the classic way of reading variables, attributes, and dimensions. Inquiry APIs for enhanced metadata is currently not supported.</li>
108+
<li>Due to a bug in HDF5 version 1.10.2 and prior, collective I/O on record variables with a subset of participating processes making zero-length requests may cause an HDF5 error and the program to hang. Readers refer to the HDF bug issue HDFFV-10501. The bug fix will appear in HDF5 1.10.4 release.</li>
109+
<li>PnetCDF allows different kinds of APIs called by different processes in a collective I/O. For example, ncmpi_put_vara_int_all() is called at process 0 and ncmpi_put_vars_float_int() is called at process 1. However, this is not allowed in NetCDF-4. Thus, the same kind of API must be used in a collective call when accessing a NetCDF-4 file.</li>
110+
111+
<!--
112+
</ul>
113+
<h3 id="QA">Questions and Answers</h3>
114+
<p>
115+
Q: Can I run the Burst Buffer Driver on a different machine with a different burst buffer configuration?
116+
<br/>
117+
A: We currently have only tested the Burst Buffer Driver on Cori at NERSC and the Cray
118+
burst buffer system installed there.
119+
It may also work on other burst buffer implementations where the burst buffer is mounted as a file system.
120+
<p>
121+
Q: Can I run the Burst Buffer Driver without the burst buffer?<br/>
122+
A: Yes, as long as the hint "nc_burst_buf_dirname" is a valid directory, PnetCDF will use it as a burst buffer.
123+
However, the performance will be affected depending on the performance of the devices.
124+
125+
<p>
126+
Q: Do I need to enable DataWarp module on Cori to build PnetCDF with Burst
127+
Buffer Driver support?<br/>
128+
A: No, we currently do not use DataWarp APIs in the implementation.<p>
129+
Q: Can I leave the data on burst buffer without flushing it to the parallel
130+
file system?<br/>
131+
A: No, in this version, the data will always be flushed when the file is
132+
closed. We do not recommand terminating the program without closing the file.
133+
<p>
134+
Q: What is the file name of intermediate files Burst Buffer Driver used to buffer
135+
I/O data? Will it overwrite my existing file if there is a name conflict? <br/>
136+
A:
137+
The intermediate files are named as &lt;NetCDF file name&gt;_&lt;ncid&gt;_&lt;rank&gt;.meta and
138+
&lt;NetCDF file name&gt;_&lt;ncid&gt;_&lt;rank&gt;.data. To prevent overwriting other files by
139+
accident, Burst Buffer Driver will not proceed when there is a file name conflict.
140+
In such case,
141+
<a href="http://cucis.ece.northwestern.edu/projects/PnetCDF/doc/pnetcdf-c/ncmpi_005fcreate.html">ncmpi_create</a>
142+
and
143+
<a href="http://cucis.ece.northwestern.edu/projects/PnetCDF/doc/pnetcdf-c/ncmpi_005fopen.html">ncmpi_open</a> will fail with error code NC_EEXISTS.
144+
<p>
145+
Q: Does the directory used by burst buffer to store I/O data need to be an empty folder?<br/>
146+
A: No, there's no such restriction. However, we strongly recommend the use of
147+
an empty folder to prevent errors caused by file name conflict. <p>
148+
Q: Is there a way to trigger a flush without closing the NetCDF file?<br/>
149+
A: Yes, calling
150+
<a href="http://cucis.ece.northwestern.edu/projects/PnetCDF/doc/pnetcdf-c/ncmpi_005fsync.html">
151+
ncmpi_sync</a> triggers a flsuh. A flush can also be triggered if the user
152+
waits on a non-blocking write request that is still buffered on burst buffer. <h3 id="Src">Source Code Checkout</h3>
153+
Use the following command to download the source codes.
154+
<pre>
155+
git clone https://github.com/Parallel-NetCDF/PnetCDF.git
156+
157+
</pre>
158+
Initialize autoconf utility settings using command:
159+
<pre>
160+
autoreconf -i
161+
</pre>
162+
Next, run configure command to build the PnetCDF library.
163+
For more information about running configure commands, readers are referred to the README files under folder
164+
<a href="https://github.com/Parallel-NetCDF/PnetCDF/tree/master/doc">doc</a>.
165+
166+
<hr>
167+
Please email your questions to &lt;<a href="mailto:parallel-netcdf@lists.mcs.anl.gov">parallel-netcdf@lists.mcs.anl.gov</a>&gt;
168+
<p>
169+
-->
170+
</body>
171+
</html>

0 commit comments

Comments
 (0)