Summary
File: src/flowx/bundler/dab_writer.py (new _rewrite_cross_bundle_run_job_refs, wired into write_bundle)
ExecutePipelineActivity.prepare emits run_job_task.job_id = ${resources.jobs.<callee>.id}. That reference only resolves when the callee job is a resource in this bundle. In a multi-pipeline migration dab_writer writes one bundle per pipeline, so an ExecutePipeline calling a sibling pipeline references a node that doesn't exist in the current bundle.
Observed failure (deploy-time)
Error: invalid dependency "${resources.jobs.<sibling>.id}",
no such node "resources.jobs.<sibling>"
Root cause
The feature was half-wired: SETUP.md's "Cross-bundle job references" text and _build_databricks_yml's variable-declaration loop (driven by a _cross_bundle_variables map) both already existed, but nothing populated the map — the detection/rewrite step between them was missing.
Impact
bundle deploy hard-fails for any migration where pipelines call each other across bundles (most real multi-pipeline ADF factories). Blocks deployment entirely.
Proposed fix
In write_bundle, before databricks.yml and resource YAML are written, walk all tasks (incl. inner workflows and for_each_task bodies); for each run_job_task.job_id = ${resources.jobs.X.id} where X is not a job in this bundle, rewrite to ${var.X} and register X in _cross_bundle_variables (which the existing YAML builder then declares). The operator supplies the numeric job id at deploy via --var "X=<job_id>", exactly as SETUP.md documents.
Relationship to the lineage epic
This is the stopgap ("FLOWX-7") that #24 (ordered cross-pipeline deploy/run from control lineage) is designed to build on. #24 keeps this ${var.X} rewrite as the fallback for unresolved callees (partial exports) and layers an ordered orchestration bundle on top once #23's lineage.control_edges are available. This fix is complementary — it makes multi-pipeline deploy work today; #24 later automates the ordering. See #24.
Test gap
No multi-bundle test asserts a sibling-pipeline run_job_task becomes a ${var.X} ref with a matching declared variable. Fix PR adds coverage.
Summary
File:
src/flowx/bundler/dab_writer.py(new_rewrite_cross_bundle_run_job_refs, wired intowrite_bundle)ExecutePipelineActivity.prepareemitsrun_job_task.job_id = ${resources.jobs.<callee>.id}. That reference only resolves when the callee job is a resource in this bundle. In a multi-pipeline migrationdab_writerwrites one bundle per pipeline, so anExecutePipelinecalling a sibling pipeline references a node that doesn't exist in the current bundle.Observed failure (deploy-time)
Root cause
The feature was half-wired: SETUP.md's "Cross-bundle job references" text and
_build_databricks_yml's variable-declaration loop (driven by a_cross_bundle_variablesmap) both already existed, but nothing populated the map — the detection/rewrite step between them was missing.Impact
bundle deployhard-fails for any migration where pipelines call each other across bundles (most real multi-pipeline ADF factories). Blocks deployment entirely.Proposed fix
In
write_bundle, beforedatabricks.ymland resource YAML are written, walk all tasks (incl. inner workflows andfor_each_taskbodies); for eachrun_job_task.job_id = ${resources.jobs.X.id}whereXis not a job in this bundle, rewrite to${var.X}and registerXin_cross_bundle_variables(which the existing YAML builder then declares). The operator supplies the numeric job id at deploy via--var "X=<job_id>", exactly as SETUP.md documents.Relationship to the lineage epic
This is the stopgap ("FLOWX-7") that #24 (ordered cross-pipeline deploy/run from control lineage) is designed to build on. #24 keeps this
${var.X}rewrite as the fallback for unresolved callees (partial exports) and layers an ordered orchestration bundle on top once #23'slineage.control_edgesare available. This fix is complementary — it makes multi-pipeline deploy work today; #24 later automates the ordering. See #24.Test gap
No multi-bundle test asserts a sibling-pipeline
run_job_taskbecomes a${var.X}ref with a matching declared variable. Fix PR adds coverage.