Ais v4 optim - #1369
Closed
constantinpape wants to merge 17 commits into
Closed
Ais v4 optim#1369constantinpape wants to merge 17 commits into
constantinpape wants to merge 17 commits into
Conversation
Snapshot of the APG 2D/3D optimization experiments on this branch: library changes to the v2 automatic prompt generator and UniSAM2 decoder width, the campaign screening/training/benchmark scripts and their JSON configs, campaign notes and reviews, development check scripts, and the accompanying tests. Committed as-is so the full experimental state stays on this branch; the clean-up happens on a separate branch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every optimization tested in the 2026-09 APG campaigns was refuted under the generalization rule and is removed from the library, together with the learned prompt-selection machinery that PR #1340 had merged earlier: - micro_sam/v2/automatic_prompt_generation.py drops the learned multimask selector, filter and refinement gate (multimask_scorer, multimask_selection, score_filter, set_multimask_models, gate kwargs), the structural hooks (prompt_type, arbitration, fusion, recover_residual and their helpers), the label-free refinement gating rules (protect_neighbours, negative_scope, gate="isolated", isolated_fallback, touch_radius), the 3D learned candidate filter (return_metadata, VOLUME_CANDIDATE_FEATURE_NAMES, candidate_* options) and every experiment-only hook (prompts=, keep_trace, return_multimask_features). The plain second-round refinement (points / boxes / masks, 2D and the 3D anchor-slice variant) stays, as does the tiled generator. - micro_sam/v2/multimask_selection.py is deleted; the annotator no longer passes score_filter. - finetuning/v2/evaluation keeps the generic harness core (2D/3D manifests and runners, comparator, SLURM submitter, task registry, 3D case tools, pre-existing library benchmarks) stripped of the learned/structural plumbing; the 17 feature-specific screens/trainers/readers and 39 of 44 configs are removed. The 3D runner loses its trace-based recall attribution and anchor export. The stray test_apg_3d_tiling.py becomes check_apg_3d_tiling.py so bare pytest no longer collects it. - Tests for the removed features are deleted or trimmed; the two comparator test files are merged into test_apg_optimization.py. The full campaign state stays preserved on branch apg-optim-fable (356b76d). This commit starts a new implementation checksum epoch (f76ee7170ca77da882c0078dfaa5b301, seven files). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every campaign note gets a banner stating that the mechanisms it describes were tested, refuted and removed on this branch, that the full state is preserved on apg-optim-fable (356b76d), and which sections describe kept functionality (the second-round refinement). A "Status on this branch" section at the end of each note maps the removed scripts, configs and library hooks. EXPERIMENTAL_SETUP.md records the reproducible set-up of the campaigns for future instance-segmentation (AIS) optimization: environment and SLURM presets, datasets and splits, checkpoints and the v4 staging recipe, the 2D subset manifests and the leak-free 3D crop manifests with their checksums, metrics and aggregation, parameter defaults and controls, acceptance and generalization gates, the timing-trial protocol, implementation checksum epochs (now f76ee7170ca77da882c0078dfaa5b301), the output-root layout and the steps to run an AIS campaign on the same set-up. The two tiled-3D-APG design notes move from the repository root into the notes directory unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…al_features torch_em 0.10.4 takes 'initial_features' as an argument of UNETR3D and sizes the decoder from it, so the joint/v4 checkpoints (32-wide decoders) load strictly through the plain kwargs forwarding UniSAM2 already had on dev. Verified against joint/v4 hvit_t geodesic best.pt: all 325 decoder keys match with no rebuild. micro_sam/v2/models/util.py returns to its dev version; the UniSAM2 width test stays. torch_em 0.10.1 built a 64-wide decoder regardless, so v4 checkpoints now require the newer torch_em (dependency floor to follow). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
submit_all_evaluations.py defaulted to the 'super' environment, which does not exist on grete, and to 3g.40gb slices for 3D jobs, which are scarce; it now activates 'new-stack' and puts 3D jobs on 1g.20gb (the (8, 512, 512) production crops peak around 6 GiB). The parameter_search.py array template activated 'super' as well and now uses the same environment; as a checksum file this starts implementation epoch e1903b1b3c1e4e3610c71e1d0bd81f1d (harness-only, results unaffected). EXPERIMENTAL_SETUP.md gains section 14 with the 2026-09-06 baseline reruns of the cleaned harness on joint/v4 geodesic defaults: the 2D subset benchmark (bit-identical to the recorded v4 controls), the 3D deep crops (holdout identical, primary identical on 56/57 crops) and the AIS/APG production defaults on nine test splits, each with its run directories, result files and reference numbers. The campaign notes point to it from their status sections. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Predict every manifest sample once, cache the (4, *spatial) decoder prediction with its labels, and run configurations, parameter sweeps, seed diagnostics and ground-truth oracles on the cache. Run directories follow the APG layout so the comparator reads them; the report applies the generalization gate. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…cessing The geodesic decoder predicts a distance magnitude that falls to zero along the boundary of every object it recognised, while a false foreground region carries no such dip. drop_instances_without_boundary_dip removes instances whose median boundary magnitude exceeds a threshold; flow_instance_segmentation exposes it as boundary_magnitude_max (off by default). The AIS benchmark mirrors it, the cached sweep scorer applies it, and the campaign notes record the diagnostics and prototypes that led to it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…the A1 screens report_ais_sweep.py joins the per-dataset sweep tables of a grid over several manifests and scores every combination as a shared default: balanced mSA, change per dataset against the library defaults, the gate verdict and the ratio to each dataset's optimum. The notes record the Phase 1 diagnostics, the three prototypes and the epoch A1 filter screens. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
evaluate_automatic_segmentation.py --ais_params reads a benchmark-style configuration, resolves it against the library defaults (sparse and dense) and scores the test split with it; submit_all_evaluations.py passes --ais_params and --result_tag through. The sweep ranking accepts the deep 3d manifests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…-aware The flow post-processing defaults of hvit_t become sigma 1.0, min_size 50 and the boundary-magnitude filter at 0.4 for images; volumes take min_size 200 and foreground 0.6 through a new 'sparse_volume' override table that default_postprocessing resolves by dimensionality. Against the registry values this gains +2.4 % balanced mSA on eleven 2d development datasets (9 up), +4.3 % on the 2d holdout and +22 % on the 3d LM crops with the joint/v4 geodesic checkpoint. The filter now derives every instance's boundary median from one sort over the boundary pixels, 18x faster with identical output. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…the filter test Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…boundary filter The volume overrides (foreground 0.6, size floor 200 voxels) that won on the 3d tuning crops lost on the seven test-only 3d datasets (two up, two below the loss limit); the registry values plus the filter gain +3.4 % there with every dataset up, and +4.7 % / +9.7 % on the tuning crops. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…results Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
lower_height_under_seeds lowers the height map under the seeds ('zero', or the minimum of a ring around the
seed) so that the monotone flooding of the watershed does not hold a seed's front at the seed's own height.
Off by default; the harness and the sweep scorer mirror it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ampaign Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
Author
|
This is a draft, but shows some decent improvements: 
|
Contributor
Author
|
Closed in favor of #1371 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.