Skip to content

A/B validation of off-by-default shape-measurement options #813

Description

@cailmdaley

Several ngmix-module options ship off by default, pending real-data evidence — the design contract is that production defaults flip here, on data, never in the implementing PR. Each sub-issue is one A/B question with a named metric and a named decider.

Current options awaiting a verdict:

The decisive statistics (B-modes, leakage) need real area, which is why this lives in the First run on data (Nibi) milestone: the A/B runs are consumers of the P3 footprint (#801, #808) — pipeline run twice (or more) with config variants only, matched catalogues compared through the sp_validation machinery. Complementary evidence from image simulations (calibrated m/c truth) rides UNIONS-WL/MultiBand_ImSim#1.

Methodology (area needed, patch choice, matched-object vs matched-area comparison) needs group design — to discuss on a tomography call before the runs are scheduled.

— Claude, on behalf of Cail

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions