The prep-caly-model-devi step in _prep_run_caly receives no step config at all, and mixes the two phases' settings: it builds its PythonOPTemplate from run_template_config but runs on prep_executor.
At e45c147ed261d7c43b4b5bc9ada6e7d271e32c51:
|
# prep_caly_model_devi |
|
prep_caly_model_devi = Step( |
|
"prep-caly-model-devi", |
|
template=PythonOPTemplate( |
|
prep_caly_model_devi_op, |
|
python_packages=upload_python_packages, |
|
**run_template_config, |
|
), |
|
parameters={ |
|
"task_name": "prep-calypso-model-deviation", |
|
"config": prep_run_caly_steps.inputs.parameters["explore_config"], |
|
}, |
|
artifacts={ |
|
"traj_results": caly_evo_step.outputs.artifacts["traj_results"], |
|
}, |
|
key="%s--prep-caly-model-devi" |
|
% (prep_run_caly_steps.inputs.parameters["block_id"],), |
|
executor=prep_executor, |
|
) |
|
prep_run_caly_steps.add(prep_caly_model_devi) |
The Step(...) ends at executor=prep_executor, — there is no **prep_config and no **run_config. So parallelism, continue_on_failed, continue_on_num_success and continue_on_success_ratio cannot be set for this step by any means, whether the user configures prep_explore_config or run_explore_config.
Separately, line 254 takes **run_template_config (image, timeout, retry settings, envs) while line 265 takes executor=prep_executor. Every other step in the file keeps those two consistent: prep-caly-input is prep_template_config + prep_executor + **prep_config, and run-caly-model-devi is run_template_config + run_executor (+ **run_config once #380 lands). This step is the only one that crosses them, and nothing in the file explains why.
Origin
Introduced in this exact shape by 07df321 ("Calypso speedup by refactorizing model deviation", #217, 2024-04-30) — the same commit that introduced the **prep_config miswiring on run-caly-model-devi that #380 fixes. A config spread was never present here, so nothing was lost later; it was simply never added.
Suggested fix
Decide which phase this step belongs to and make the template, executor and step config agree. If it is a prep step, use prep_template_config + prep_executor + **prep_config. If it is a run step, use run_template_config + run_executor + a run config.
Either way, note that this step is not sliced, so spreading a config that still contains continue_on_num_success / continue_on_success_ratio will raise UnboundLocalError inside dflow — see the companion issue on the prep_config pop/spread mismatch. The no_slice_run_config idiom at caly_evo_step.py L150-L152 is the existing pattern for this.
Found while reviewing #380.
The
prep-caly-model-devistep in_prep_run_calyreceives no step config at all, and mixes the two phases' settings: it builds itsPythonOPTemplatefromrun_template_configbut runs onprep_executor.At
e45c147ed261d7c43b4b5bc9ada6e7d271e32c51:dpgen2/dpgen2/superop/prep_run_calypso.py
Lines 248 to 267 in e45c147
The
Step(...)ends atexecutor=prep_executor,— there is no**prep_configand no**run_config. Soparallelism,continue_on_failed,continue_on_num_successandcontinue_on_success_ratiocannot be set for this step by any means, whether the user configuresprep_explore_configorrun_explore_config.Separately, line 254 takes
**run_template_config(image, timeout, retry settings, envs) while line 265 takesexecutor=prep_executor. Every other step in the file keeps those two consistent:prep-caly-inputisprep_template_config+prep_executor+**prep_config, andrun-caly-model-deviisrun_template_config+run_executor(+**run_configonce #380 lands). This step is the only one that crosses them, and nothing in the file explains why.Origin
Introduced in this exact shape by
07df321("Calypso speedup by refactorizing model deviation", #217, 2024-04-30) — the same commit that introduced the**prep_configmiswiring onrun-caly-model-devithat #380 fixes. A config spread was never present here, so nothing was lost later; it was simply never added.Suggested fix
Decide which phase this step belongs to and make the template, executor and step config agree. If it is a prep step, use
prep_template_config+prep_executor+**prep_config. If it is a run step, userun_template_config+run_executor+ a run config.Either way, note that this step is not sliced, so spreading a config that still contains
continue_on_num_success/continue_on_success_ratiowill raiseUnboundLocalErrorinside dflow — see the companion issue on theprep_configpop/spread mismatch. Theno_slice_run_configidiom atcaly_evo_step.pyL150-L152 is the existing pattern for this.Found while reviewing #380.