diff --git a/docs/quickcli.md b/docs/quickcli.md index e9003715..765ef2d9 100644 --- a/docs/quickcli.md +++ b/docs/quickcli.md @@ -10,6 +10,19 @@ dpgen2 submit input.json where `input.json` is the input script. A guide of writing the script is found [here](inputscript). When a workflow is submitted, a ID (WFID) of the workflow will be printed for later reference. +### Run the orchestrator locally and tasks on Slurm + +DPGEN2 can run dflow locally while dispatching computational steps to a Slurm cluster over SSH. Start the local orchestrator with: + +```bash +export DFLOW_DEBUG=1 +dpgen2 submit examples/water/input_dpgen_slurm.json +``` + +For this arrangement, set `bohrium_config`, `dflow_config`, and `dflow_s3_config` to `null` or omit them. Configure a `dispatcher` executor under each computational `step_configs` entry instead. Important fields include the SSH `host`, `username`, `private_key_file`, and `remote_root`, plus `machine_dict.batch_type: Slurm` and the queue/resources in `resources_dict`. + +The maintained `examples/water/input_dpgen_slurm.json` shows separate GPU training/exploration and CPU labeling resources. Commands, modules, container images, remote paths, and queue names remain site-specific. The submission host must be able to reach the Slurm login node and retain local dflow artifacts for the lifetime of the workflow. + ## Check the convergence of a workflow The convergence of stages of the workflow can be checked by the `status` command. It prints the indexes of the finished stages, iterations, and the accurate, candidate and failed ratio of explored configurations of each iteration. ```bash diff --git a/dpgen2/entrypoint/args.py b/dpgen2/entrypoint/args.py index df11ff7f..ad40e434 100644 --- a/dpgen2/entrypoint/args.py +++ b/dpgen2/entrypoint/args.py @@ -646,8 +646,14 @@ def input_args(): def dflow_conf_args(): - doc_dflow_config = "The configuration passed to dflow" - doc_dflow_s3_config = "The S3 configuration passed to dflow" + doc_dflow_config = ( + "The configuration passed to dflow. It may be null when the workflow " + "orchestrator runs locally in debug mode." + ) + doc_dflow_s3_config = ( + "The S3 configuration passed to dflow. It may be null for local debug " + "mode when artifacts remain on the submission host." + ) return [ Argument( diff --git a/examples/water/input_dpgen_slurm.json b/examples/water/input_dpgen_slurm.json index fface13e..3750130b 100644 --- a/examples/water/input_dpgen_slurm.json +++ b/examples/water/input_dpgen_slurm.json @@ -1,11 +1,6 @@ { "name": "water-dpgen", - "bohrium_config": { - "username": "", - "password": "", - "project_id": 1, - "_comment": "all" - }, + "bohrium_config": null, "default_step_config": { "template_config": { "image": "", diff --git a/tests/test_check_examples.py b/tests/test_check_examples.py index 1a4e40d9..87848352 100644 --- a/tests/test_check_examples.py +++ b/tests/test_check_examples.py @@ -23,6 +23,7 @@ p_examples / "chno" / "input.json", p_examples / "water" / "input_dpgen_abacus.json", p_examples / "water" / "input_dpgen_cp2k.json", + p_examples / "water" / "input_dpgen_slurm.json", p_examples / "diffcsp" / "dpgen.json", )