Config cleanup: one source of truth + fiducial-plus-overrides philosophy

[cailmdaley]

Problem

sp_validation config is spread across three overlapping surfaces, and no single one owns a given parameter.

1. config/calibration/mask_v1.X.*.yaml — 12 near-duplicate files. Each catalogue version has its own masking/calibration config. Most differ from the previous only by a version comment and one flag value (for example FLAGS <= 2 vs <= 3). Copy-paste drift is already visible: label formatting is inconsistent between files (r"$n_{\rm epoch}$" vs $n_{\rm epoch}$). Nothing in the workflow selects these; catalog_builders.py reads one when invoked by hand, and a test only checks the paths exist.
2. cosmo_val/cat_config.yaml holds per-catalogue metadata: input paths, column names, and survey parameters (A, n_e, n_psf, sigma_e under cov_th).
3. papers/*/config/config.yaml holds the Snakemake config: versions, a fiducial: block, scale cuts, and cosmology parameters. The cosmo_val config documents in a comment that several top-level keys exist only to satisfy parse-time lookups of the shared workflow module and "are not exercised by the cosmo_val rules themselves."

The same physical quantity can live in two places. A survey area or sigma_e is declared in cat_config and can be re-declared in the workflow config. There is no rule for which wins.

Proposed philosophy (for discussion)

• One source of truth per parameter. A value is declared once. Every other surface references it, never restates it.
• cat_config.yaml owns catalogue facts — paths, column names, and per-catalogue survey parameters (area, number densities, sigma_e). The workflow pulls these at parse time instead of hardcoding them.
• The workflow owns one fiducial config. A single fiducial: block sets the default run (version, binning, scale cuts, cosmology). Grid or comparison runs are declared as overrides on top of the fiducial, not as parallel full configs.
• Mask/calibration cuts get one templated config, not one file per version. The cuts that are actually constant across versions live once; only the genuine per-version deltas are expressed as a diff or override.

Cleanup tasks

• Collapse the 12 mask_v1.X.*.yaml files into one base + per-version overrides (or confirm which are dead and remove them).
• Define which surface owns each parameter; document the rule in a short config README.
• Make the workflow pull survey parameters from cat_config rather than restating them.
• Remove the parse-time-only keys from the paper configs once the shared module reads them from their real home.

Open questions

• Are all 12 mask configs live, or are early versions dead? (Only catalog_builders.py and a path test touch them.)
• Do we want a schema (pydantic / jsonschema) to enforce the ownership rule, or is convention + README enough?
• Fiducial-plus-overrides: express overrides in YAML anchors, a config-merge layer, or Snakemake configfile composition?

---

— Claude, on behalf of Cail