feat(skills): add dpgen-run skill for concurrent learning workflow

skills/dpgen-run/SKILL.md

@@ -0,0 +1,45 @@
+---
+name: dpgen-run
+description: Prepare, explain, validate, and run DP-GEN concurrent-learning workflows. Use for dpgen run param.json or machine.json, training and exploration settings, labeling backends, trust levels, model_devi_jobs, execution, or output inspection.
+license: LGPL-3.0-or-later
+metadata:
+  author: MatMaster
+  version: 0.2.0
+  repository: https://github.com/deepmodeling/dpgen
+---
+
+# DP-GEN Run
+
+Use this skill for the iterative DP-GEN loop: train an ensemble, explore with MD, select uncertain structures, label them, and add the labels to training data.
+
+## Core contract
+
+- Keep workflow settings in `param.json` and execution settings in `machine.json`.
+- The exact launcher is `dpgen run param.json machine.json`.
+- Treat the launcher shell and dispatched stage environments as separate layers.
+- Prepare and validate by default. Execute only when the user explicitly requests it and confirms the exact command after validation.
+
+## Workflow
+
+1. Confirm this is a `dpgen run` task and inspect existing files first.
+1. Collect only missing inputs; patch working configurations instead of rebuilding them.
+1. Load only the references needed for the current task.
+1. Preserve scientific and site-specific choices unless the user approves changes.
+1. Validate files, data, commands, and environments before proposing execution.
+1. Report exact paths, the command, unresolved fields, and post-run inspection targets.
+
+## Progressive references
+
+Read only the relevant files:
+
+- [Workflow policy](references/workflow.md): scope, requirements, missing-input policy, layout, and reporting.
+- [param.json guide](references/param-json.md): systems, training, exploration, FP fields, descriptors, thresholds, and examples.
+- [machine.json guide](references/machine-json.md): stage commands, contexts, resources, scheduler environments, and examples.
+- [Validation and execution](references/validation-and-run.md): schema checks, data checks, authorization, launch, and output inspection.
+
+## Non-negotiable guardrails
+
+- Never merge the two JSON files or invent cluster paths, queues, modules, or activation commands.
+- Preserve descriptor, fitting network, FP backend, thresholds, schedules, ensemble settings, and `type_map` ordering.
+- Spell `se_atten_v2` exactly; do not use `fp_style: "none"` for current `dpgen run`.
+- Stop and ask when required scientific or execution inputs cannot be discovered safely.

skills/dpgen-run/references/machine-json.md

@@ -0,0 +1,58 @@
+# DP-GEN Run machine.json Guide
+
+Load this reference when creating, patching, explaining, or reviewing `machine.json`.
+
+## Runtime boundary
+
+Treat execution as two environments:
+
+1. the outer shell that launches `dpgen run param.json machine.json`
+1. the dispatched `train`, `model_devi`, and `fp` tasks
+
+The outer shell must resolve `dpgen`. Scheduler jobs must activate their own software in `resources.source_list`; outer activation is not inherited reliably.
+
+## Stage blocks
+
+Keep separate `train`, `model_devi`, and `fp` blocks. For each stage, collect or preserve:
+
+- `command`
+- `machine.batch_type`
+- `machine.context_type`
+- `machine.local_root`
+- `machine.remote_root` when the selected context requires it
+- `resources.number_node`
+- `resources.cpu_per_node`
+- `resources.gpu_per_node`
+- `resources.group_size`
+- `resources.source_list` for scheduler runtime activation
+- queue, partition, account, and custom scheduler flags when applicable
+
+`train_backend` does not rewrite the machine command. `dp` selects the DeePMD installation's default backend; a DeePMD-kit 3.x PyTorch installation may require `dp --pt`. Verify the installed entry point with `dp train -h` or `dp --pt train -h`, then make the `train` command agree with `train_backend`.
+
+Typical later-stage commands are `lmp` for LAMMPS exploration and a backend executable such as `vasp_std`, `cp2k.popt`, `abacus`, `pw.x`, or `g16` for labeling. Preserve known working commands.
+
+## Context and batch compatibility
+
+`remote_root` is context-dependent. Require it for remote or scheduler contexts whose installed dpdispatcher schema needs a remote working root. Preserve a working omission for lazy or local execution and validate against the installed dpdispatcher version.
+
+`batch_type` must be registered by the installed dpdispatcher. Canonical names and lowercase aliases can both be valid. Preserve a working value; validate newly introduced values rather than normalizing them by assumption.
+
+## Starting profiles
+
+Choose a profile that matches where the outer command runs:
+
+- server-local scheduler: [scheduler example](../../../examples/machine/DeePMD-kit-1.x/machine-lsf-slurm-cp2k.json)
+- pure local shell: [local example](../../../examples/machine/DeePMD-kit-1.x/machine-local.json)
+
+Patch the closest existing working configuration. Do not transplant site-specific paths or scheduler settings blindly.
+
+## Guardrails
+
+- Never combine all stages into one vague block.
+- Never invent executables, modules, activation commands, queues, accounts, or paths.
+- Keep CPU, GPU, node, and grouping requests explicit.
+- Require explicit `source_list` activation for scheduler stages.
+- Preserve working local omissions and installed-version aliases.
+- Ensure the training command selects the same backend as `train_backend` and all commands match the scientific stack selected in `param.json`.
+
+External machine reference: https://docs.deepmodeling.com/projects/dpgen/en/latest/run/mdata.html

skills/dpgen-run/references/param-json.md

@@ -0,0 +1,107 @@
+# DP-GEN Run param.json Guide
+
+Load this reference when creating, patching, explaining, or reviewing `param.json`.
+
+The current authoritative schema is [dpgen/generator/arginfo.py](../../../dpgen/generator/arginfo.py).
+
+## System and data fields
+
+Collect or preserve:
+
+- `type_map`: ordered element symbols
+- `mass_map`: optional; defaults to `"auto"`
+- `init_data_prefix`: optional prefix for `init_data_sys`
+- `init_data_sys`: DeepMD NumPy-system directories
+- `sys_configs_prefix`: optional prefix for `sys_configs`
+- `sys_configs`: list of systems, each represented by a list of structure paths
+
+`init_data_sys` entries must resolve to directories containing `type_map.raw`, `type.raw`, and `set.000/`. The ordering in every `type_map.raw` must match `type_map`.
+
+`sys_configs` is a list of lists. The outer list separates systems; each inner list contains structure files for that system.
+
+## Training fields
+
+Required core inputs include:
+
+- `numb_models`: ensemble size; four is a common recommendation, not a schema default
+- `default_training_param.model`
+- `default_training_param.learning_rate`
+- `default_training_param.loss`
+- `default_training_param.training`
+
+DeePMD-kit 1.x uses `training.systems`. DeePMD-kit 2.x and 3.x use `training.training_data.systems`. Leave the version-appropriate systems value unset or empty because DP-GEN fills it from `init_data_sys`. Verify `deepmd_version` and installed DeePMD-kit before using version-specific features.
+
+### Descriptor selection
+
+Do not change a user-selected descriptor silently.
+
+| Descriptor    | Typical use                                   | Notes                                 |
+| ------------- | --------------------------------------------- | ------------------------------------- |
+| `se_e2_a`     | simple systems and conservative compatibility | explicit `sel` list                   |
+| `se_atten`    | multicomponent systems                        | supports `sel: "auto"`                |
+| `se_atten_v2` | modern attention workflow                     | spell exactly; supports `sel: "auto"` |
+
+DP-GEN exposes `train_backend` values `"tensorflow"` and `"pytorch"`, but this alone does not guarantee every descriptor or DeePMD-kit feature works in the installed stack. Verify backend, `deepmd_version`, and DeePMD-kit compatibility. Prefer dpgen2 when a requested DPA-2/DPA-3 or PyTorch-only workflow is not verified with this DP-GEN installation.
+
+## Exploration fields
+
+Collect or preserve:
+
+- `model_devi_dt`: MD timestep in ps
+- `model_devi_skip`: initial frames to skip
+- `model_devi_f_trust_lo`
+- `model_devi_f_trust_hi`
+- `model_devi_clean_traj`
+- `model_devi_jobs`
+
+Each `model_devi_jobs` item normally defines `sys_idx`, `temps`, `press`, `trj_freq`, `nsteps`, and `ensemble`.
+
+Do not set `model_devi_engine` for ordinary LAMMPS exploration; LAMMPS is the default. Set it only for a verified alternative engine.
+
+### Trust-window interpretation
+
+- below `model_devi_f_trust_lo`: accurate, normally not selected
+- between low and high: candidate for labeling
+- above `model_devi_f_trust_hi`: failed or unreliable, normally discarded
+
+Values such as 0.05-0.10 eV/Angstrom for the low threshold and 0.15-0.30 eV/Angstrom for the high threshold are starting ranges, not universal defaults. Tighter thresholds increase labeling cost and require system-specific justification.
+
+## First-principles fields
+
+`fp_style` must be a backend accepted by the current schema, for example `"vasp"`, `"cp2k"`, `"abacus"`, `"gaussian"`, or `"pwscf"`. The value `"none"` is not valid for current `dpgen run`.
+
+Always collect:
+
+- `fp_style`
+- `fp_task_max`
+- `fp_task_min`
+
+Backend-specific inputs include:
+
+- VASP: `fp_pp_path`, `fp_pp_files`, `fp_incar`
+- CP2K: `user_fp_params`
+- ABACUS: `user_fp_params`, `fp_pp_path`, `fp_pp_files`, `fp_orb_files`
+- Gaussian: `fp_params`
+- PWSCF: `user_fp_params`
+
+CP2K is supported natively. For multiple KIND sections in `user_fp_params`, use `"_": ["H", "O"]` with parallel `POTENTIAL` and `BASIS_SET` arrays.
+
+## Logical construction order
+
+1. elements and masses
+1. initial data and exploration structures
+1. model ensemble and DeePMD training input
+1. model-deviation controls and job schedule
+1. first-principles backend and task limits
+
+Do not assume a field is optional from an old example. Validate against the current schema.
+
+## Checked repository examples
+
+- [CP2K methane parameters](../../../examples/run/dp2.x-lammps-cp2k/param_CH4_deepmd-kit-2.0.1.json)
+- [VASP CH4 parameters](../../../examples/run/dp2.x-lammps-vasp/CH4/param_CH4_deepmd-kit-2.x.json)
+
+External references:
+
+- https://docs.deepmodeling.com/projects/dpgen/en/latest/run/param.html
+- https://docs.deepmodeling.com/projects/deepmd/en/latest/model/index.html

skills/dpgen-run/references/validation-and-run.md

@@ -0,0 +1,86 @@
+# DP-GEN Run Validation and Execution
+
+Load this reference before claiming a configuration is valid or before executing `dpgen run`.
+
+## 1. Verify the outer environment
+
+```bash
+dpgen -h
+```
+
+Stop if the launcher is unavailable. Actual execution also requires DeePMD-kit, the exploration engine, the selected first-principles software, and scheduler access where applicable.
+
+## 2. Check JSON syntax
+
+```bash
+python -m json.tool param.json
+python -m json.tool machine.json
+```
+
+## 3. Validate against installed schemas
+
+Use the installed DP-GEN and dpdispatcher code, not documentation alone.
+
+```python
+import json
+
+from dpgen.generator.arginfo import run_jdata_arginfo
+from dpgen.remote.decide_machine import convert_mdata
+from dpgen.util import normalize
+
+with open("param.json") as stream:
+    normalize(run_jdata_arginfo(), json.load(stream), strict_check=False)
+
+with open("machine.json") as stream:
+    convert_mdata(json.load(stream))
+```
+
+Schema normalization is necessary but not sufficient; it does not prove paths, executables, scheduler permissions, or scientific choices are correct.
+
+## 4. Validate data and structures
+
+Resolve every `init_data_sys` entry against `init_data_prefix` when set. Verify each NumPy system contains:
+
+- `type_map.raw`
+- `type.raw`
+- `set.000/` with the expected arrays
+
+Compare every `type_map.raw` line-for-line with `param.json.type_map`. Verify every structure referenced through `sys_configs_prefix` and `sys_configs` exists.
+
+## 5. Validate scientific stages
+
+Confirm:
+
+- DeePMD-kit version, backend, descriptor, and training-input layout agree
+- `train_backend` and the machine `train.command` select the same backend; for DeePMD-kit 3.x PyTorch, verify `dp --pt train -h` and use `dp --pt` when required
+- each exploration job references valid systems and MD settings
+- force trust thresholds are ordered and scientifically intentional
+- FP inputs and support files match `fp_style`
+- CP2K basis and potential files are accessible when CP2K is selected
+- `fp_task_min` and `fp_task_max` reflect intended cost limits
+
+## 6. Validate execution stages
+
+For `train`, `model_devi`, and `fp`, verify:
+
+- commands exist in the dispatched environment
+- context and batch types are accepted by installed dpdispatcher
+- required local and remote roots exist
+- scheduler resources and flags are valid
+- `resources.source_list` activates the correct inner runtime
+
+A successful outer `dpgen -h` does not validate dispatched environments.
+
+## 7. Confirm before launch
+
+Show the exact files, validation summary, unresolved risks, and command:
+
+```bash
+dpgen run param.json machine.json
+```
+
+Execute only after the user explicitly requests the run and confirms this exact validated command. If the user requested only preparation or validation, stop here.
+
+## 8. Inspect outputs
+
+After launch, report the current `iter.*` directory and the status of training, model-deviation, and FP tasks. Summarize failures from stage logs and report selected and labeled structure counts when available.

skills/dpgen-run/references/workflow.md

@@ -0,0 +1,97 @@
+# DP-GEN Run Workflow Policy
+
+Load this reference for end-to-end preparation, task scoping, project layout, or result reporting.
+
+## Scope and requirements
+
+`dpgen run` implements the concurrent-learning loop:
+
+1. train an ensemble of Deep Potential models
+1. explore configuration space, normally with LAMMPS MD
+1. select configurations in the model-deviation candidate window
+1. label selected structures with the configured first-principles backend
+1. add labeled data to the next training iteration
+
+Preparation and validation require Python and DP-GEN. Execution additionally requires DeePMD-kit, a compatible exploration engine, the software selected by `fp_style`, and any scheduler runtime.
+
+The workflow always uses:
+
+- `param.json` for scientific and workflow parameters
+- `machine.json` for commands, contexts, and resources
+- `dpgen run param.json machine.json` as the launcher
+
+## Working policy
+
+### Inspect before asking
+
+Look for existing configuration files, training inputs, dataset metadata, and machine templates. Ask only for values that cannot be discovered. Patch existing working files rather than rebuilding them without need.
+
+### Preserve scientific choices
+
+Do not silently change:
+
+- descriptor family or settings
+- fitting network
+- training backend
+- first-principles backend
+- trust thresholds
+- `type_map` ordering
+- `model_devi_jobs` schedule
+- ensemble size, temperatures, pressures, or MD ensembles
+
+Explain a concern and request direction before changing a scientific choice.
+
+### Preserve site-specific choices
+
+Reuse known activation commands, executable names, queues, partitions, accounts, paths, and scheduler flags exactly. Never guess conda environments, modules, personal paths, or cluster policy.
+
+### Keep execution separately authorized
+
+Preparing files, validating them, or displaying a command does not authorize an HPC, MD, training, or first-principles workload. Run only when the user explicitly requests execution and confirms the exact validated command.
+
+## Recommended layout
+
+```text
+project/
+|-- param.json
+|-- machine.json
+|-- init_data/
+|   `-- system_000/
+|       |-- type_map.raw
+|       |-- type.raw
+|       `-- set.000/
+|-- assets/
+|   `-- structures/
+|-- cp2k_basis_pp_file/   # only when CP2K needs it
+`-- iter.*/              # created by DP-GEN
+```
+
+Keep repeated experiments in separate, clearly named directories derived from one reviewed base configuration.
+
+## Result contract
+
+Before execution, report:
+
+1. absolute paths to `param.json` and `machine.json`
+1. exact launcher command
+1. validation results
+1. unresolved required inputs
+1. expected cost-bearing stages
+
+After execution, report:
+
+1. run and current iteration status
+1. failed or pending stages
+1. main output and log locations
+1. candidate and labeled counts when available
+1. the next files that need inspection
+
+## General guardrails
+
+- Never run before both JSON files exist and pass validation.
+- Keep `type_map` consistent from data through training and labeling.
+- Do not overwrite working user templates blindly.
+- Do not assume outer-shell activation reaches dispatched jobs.
+- Stop rather than guess a missing scientific or site-specific value.
+
+Official overview: https://docs.deepmodeling.com/projects/dpgen/en/latest/run/index.html