Deisa

Author: jbigot

CMakeLists.txt

@@ -60,3 +60,5 @@ target_link_libraries(ex9 m MPI::MPI_C paraconf::paraconf PDI::pdi)
 
 add_executable(ex10 ex10.c)
 target_link_libraries(ex10 m MPI::MPI_C paraconf::paraconf PDI::pdi)
+
+add_subdirectory(ex11/)

deisa_example/CMakeLists.txt

@@ -0,0 +1,5 @@
+set(CMAKE_C_STANDARD 99)
+
+add_executable(simulation simulation.c)
+target_link_libraries(simulation m MPI::MPI_C  paraconf::paraconf PDI::pdi)
+

deisa_example/Launcher.sh

@@ -0,0 +1,33 @@
+# Dask-Enabled In Situ Analytics
+
+This directory contains an example showing how Deisa[1] can be used. 
+
+To try it on Ruche, just run:
+```
+sbatch launch.sh
+```
+
+## How it works ?
+
+This example uses PDI **_DEISA_** plugin that retrieves simulation data it, creates corresponding keys in Dask, and sends the data to the workers.
+
+Internally, a *DEISA Bridge* is created per MPI process.
+Once a piece of data is shared with PDI, the Bridge sends it to a worker that has been chosen in a round-robin fashion. 
+
+*DEISA* python library implements a *DEISA Adaptor*.
+This component is used from the Dask client-side to create Dask arrays describing the data generated by the simulation.
+The adaptor waits for an array descriptor to be sent from the bridge in MPI rank 0.
+This descriptor is a dictionary with data names as keys and a dictionary containing the sizes, dimensions, and chunk sizes, as values. 
+The adaptor uses this information to create Dask arrays, that can be retrieved by calling `get_data()` method.
+
+## Files :
+
+- `simulation.c`: a toy simulation code;
+- `simulation.yml`: the code configuration and PDI specification tree;
+- `client.py`: a python script for analytics;
+- `launch.sh`: the script that runs both the simulation and Dask cluster
+- `dask-interface.py`: *DEISA* python library;
+
+## References
+
+[1] [A. Gueroudji, J. Bigot and B. Raffin, "DEISA: Dask-Enabled In Situ Analytics," 2021 IEEE 28th International Conference on High Performance Computing, Data, and Analytics (HiPC), 2021, pp. 11-20, doi: 10.1109/HiPC53243.2021.00015.](https://ieeexplore.ieee.org/document/9680456)