Freeze CalVer synthetic data specification

.gitignore

@@ -4,6 +4,7 @@
 .idea/
 .vscode/
 .venv/
+.venv*/
 __pycache__/
 *.py[cod]
 *.egg-info/

CONTRIBUTING.md

@@ -34,12 +34,18 @@ evaluation artifacts use calendar versioning without a `v` prefix.
 | Dataset release | `data-2026.08.12` | `data-2026.08.12.1` |
 | Schema release | `canonical-table-record-2026.08.12` | `canonical-table-record-2026.08.12.1` |
 | Experiment | `experiment-2026.08.12-<topic>` | append `.1` when needed |
+| Data smoke | `data-smoke-2026.08.12` | `data-smoke-2026.08.12.1` |
 
 Use the most precise meaningful date. The identifier does not claim that every
 artifact was created at exactly the same time; its purpose is to distinguish
 research states reproducibly. Upstream packages, datasets, and benchmarks keep
 their official external version names.
 
+Project-authored research documents other than conventional repository files
+such as `README.md`, `CONTRIBUTING.md`, and `LICENSE` include their CalVer in
+the filename. Do not rename upstream action versions, benchmark releases,
+pretrained-model names, or historical source-dataset identifiers.
+
 ## Branches and pull requests
 
 Use a focused branch from the current default branch:

README.md

@@ -26,13 +26,14 @@ OmniDocBench is used as one document-parsing evaluation protocol. The methods
 and infrastructure in this repository are designed around the broader research
 problem of table structure recognition.
 
-## Research snapshot 2026.08.12
+## Research snapshot 2026.08.12.1
 
-The `2026.08.12` snapshot establishes the shared representation and evaluation
-foundation for two independent modeling tracks. It includes Canonical Table
-normalization, order-invariant topology targets, candidate-integrity checks,
-table-only model interfaces, and synthetic data-free tests. Model checkpoints
-and dataset payloads are maintained outside this repository.
+The `2026.08.12.1` snapshot establishes the shared representation, evaluation,
+and synthetic-corpus specification for two independent modeling tracks. It
+includes Canonical Table normalization, order-invariant topology targets,
+candidate-integrity checks, table-only model interfaces, a frozen 256-record
+generator-smoke design, and data-free regression tests. Model checkpoints and
+dataset payloads are maintained outside this repository.
 
 Project-authored releases follow calendar versioning:
 
@@ -54,7 +55,7 @@ separate signals so that a structural hypothesis can be inspected and replayed.
 The current repository includes the candidate representation, reversible
 interface, and validation primitives needed by the model.
 
-See [Explicit Layout Transformer](docs/methods/EXPLICIT_LAYOUT_TRANSFORMER.md).
+See [Explicit Layout Transformer](docs/methods/EXPLICIT_LAYOUT_TRANSFORMER_2026.08.12.1.md).
 
 ### LoRA Table Model
 
@@ -96,11 +97,11 @@ The common foundation provides:
 borderless-table-structuring-lab/
 ├── dataset/                   # External dataset registry; no payloads
 ├── docs/
-│   ├── methods/               # Research-track formulations
-│   ├── DATA_GOVERNANCE.md
-│   ├── DATASET_STORAGE_AND_SHARING.md
-│   ├── REPRODUCIBILITY.md
-│   └── SYNTHETIC_CORPUS_SPECIFICATION.md
+│   ├── corpus/                # CalVer data specifications and policies
+│   ├── experiment-records/    # Sealed public research evidence
+│   ├── methods/               # CalVer research-track formulations
+│   └── REPRODUCIBILITY_2026.08.12.1.md
+├── configs/                   # Calendar-versioned generation parameters
 ├── schemas/                   # Calendar-versioned record schemas
 ├── src/borderless_table_structuring/
 │   ├── canonical.py           # Canonical table normalization
@@ -142,8 +143,8 @@ compression, blur, background, and scanning artifacts.
 Dataset roles are assigned by document, template, content, renderer, and seed
 families before rendering. Exact and near-duplicate audits operate on images,
 text, normalized structure, geometry, and provenance. See the
-[synthetic corpus specification](docs/SYNTHETIC_CORPUS_SPECIFICATION.md) and
-[data governance guide](docs/DATA_GOVERNANCE.md).
+[synthetic data specification](docs/corpus/SYNTHETIC_DATA_SPECIFICATION_2026.08.12.1.md)
+and [data governance guide](docs/corpus/DATA_GOVERNANCE_2026.08.12.1.md).
 
 ## Installation
 
@@ -172,8 +173,8 @@ and complete failure accounting. Dataset payloads and model weights are stored
 outside Git; this repository contains the code, schemas, manifests, and
 documentation needed to reproduce them.
 
-For details, see [Reproducibility](docs/REPRODUCIBILITY.md) and
-[Dataset Storage and Sharing](docs/DATASET_STORAGE_AND_SHARING.md).
+For details, see [Reproducibility](docs/REPRODUCIBILITY_2026.08.12.1.md) and
+[Dataset Storage and Sharing](docs/corpus/DATASET_STORAGE_AND_SHARING_2026.08.12.1.md).
 
 ## Collaboration
 

configs/generation_parameters_2026.08.12.1.json

@@ -0,0 +1,75 @@
+{
+  "configuration_release": "2026.08.12.1",
+  "dataset_release": "data-smoke-2026.08.12.1",
+  "schema_release": "synthetic-table-record-2026.08.12.1",
+  "seed_namespace": "borderless-table-structuring-data-smoke-2026.08.12.1",
+  "seed_start": 202608120100000,
+  "requested_records": 256,
+  "allow_failed_seed_replacement": false,
+  "gate_costs": {
+    "false_edit": 4.0,
+    "missed_edit": 1.0,
+    "uncertain_action": "KEEP"
+  },
+  "categories": {
+    "exact_keep": {"count": 64, "gate_label": "KEEP"},
+    "hard_keep": {"count": 64, "gate_label": "KEEP"},
+    "single_minimal_edit": {"count": 64, "gate_label": "EDIT"},
+    "complex_correction": {"count": 64, "gate_label": "EDIT"}
+  },
+  "roles_per_category": {
+    "train": 44,
+    "development": 12,
+    "holdout": 8
+  },
+  "counterfactual_pairs": {
+    "minimum_pairs": 64,
+    "same_role_required": true,
+    "shared_fields": [
+      "content_family_id",
+      "template_family_id",
+      "font_family_id",
+      "renderer_family_id",
+      "base_degradation"
+    ]
+  },
+  "structure": {
+    "rows": {"minimum": 2, "maximum": 28},
+    "columns": {"minimum": 2, "maximum": 14},
+    "header_depth": {"minimum": 0, "maximum": 4},
+    "span_fraction": {"minimum": 0.0, "maximum": 0.45},
+    "empty_cell_fraction": {"minimum": 0.0, "maximum": 0.35},
+    "maximum_cells": 240
+  },
+  "rendering": {
+    "canvas_width_px": [960, 1280, 1600, 2048],
+    "dpi": [96, 150, 200, 300],
+    "font_size_px": {"minimum": 12, "maximum": 32},
+    "cell_padding_px": {"minimum": 4, "maximum": 18},
+    "rotation_degrees": {"minimum": -4.0, "maximum": 4.0},
+    "border_alpha": {"minimum": 0.0, "maximum": 1.0},
+    "jpeg_quality": {"minimum": 55, "maximum": 100},
+    "gaussian_blur_radius": {"minimum": 0.0, "maximum": 1.8},
+    "noise_sigma": {"minimum": 0.0, "maximum": 12.0}
+  },
+  "corruptions": {
+    "single_minimal_edit_exactly_one": true,
+    "complex_correction_minimum_operations": 2,
+    "complex_correction_maximum_operations": 4,
+    "allowed": [
+      "extra_split",
+      "missing_split",
+      "incorrect_merge",
+      "missing_merge",
+      "span_extent_error",
+      "token_ownership_error",
+      "geometry_inconsistency",
+      "localized_grid_coverage_error"
+    ]
+  },
+  "determinism": {
+    "semantic_replay_runs": 2,
+    "normalized_pixel_replay_runs": 2,
+    "sha256_algorithm": "sha256"
+  }
+}

dataset/README.md

@@ -6,13 +6,13 @@ dataset references used by reproducible experiments.
 ## Active dataset entry template
 
 ```yaml
-dataset_name: borderless-table-structuring-data
+dataset_name: data-2026.08.12.1
 service: private-hugging-face-or-approved-object-store
 repository: DearKarl/borderless-table-structuring-data
 revision: null
 root_manifest_sha256: null
-schema_release: null
-research_snapshot: null
+schema_release: synthetic-table-record-2026.08.12.1
+research_snapshot: 2026.08.12.1
 roles_available: []
 license_manifest_sha256: null
 overlap_audit_sha256: null

docs/REPRODUCIBILITY.md → docs/REPRODUCIBILITY_2026.08.12.1.md

@@ -1,4 +1,4 @@
-# Reproducibility
+# Reproducibility 2026.08.12.1
 
 ## Immutable inputs
 
@@ -17,6 +17,7 @@ Never reuse an output directory. A run writes to a new directory containing:
 - `quarantine.jsonl`;
 - `SHA256SUMS`;
 - `EXPERIMENT_RECORD.md`;
+- `EVIDENCE_CARD.md`;
 - an immutable checksum manifest.
 
 ## Determinism

docs/corpus/ACCEPTANCE_CRITERIA_2026.08.12.1.md

@@ -0,0 +1,63 @@
+# Acceptance Criteria 2026.08.12.1
+
+Status: `FROZEN_FOR_GENERATOR_SMOKE`
+
+## Preregistered smoke gate
+
+`data-smoke-2026.08.12.1` passes only when all criteria below pass. No failed
+seed may be silently replaced, no category quota may be rebalanced after
+generation, and no model performance metric may be used to select records.
+
+| Criterion | Required result | Failure action |
+|---|---:|---|
+| Requested records | 256 | Stop |
+| Generated records | 256 | Stop |
+| Accepted records | 256 | Stop |
+| Exact KEEP / Hard KEEP / Single / Complex | 64 / 64 / 64 / 64 | Stop |
+| Counterfactual pairs | at least 64 complete same-role pairs | Stop |
+| Schema validation | 256/256 | Stop |
+| Canonical legality | 256/256 | Stop |
+| Complete grid coverage | 256/256 | Stop |
+| Gold recompilation | 256/256 semantic identity | Stop |
+| Unique OCR-token ownership | 100% of owned tokens | Stop |
+| Finite complete geometry | 256/256 | Stop |
+| KEEP semantic identity | 128/128 | Stop |
+| EDIT replay to Gold | 128/128 | Stop |
+| Explicit KEEP empty diff | 128/128 | Stop |
+| LoRA target completeness | 256/256 | Stop |
+| Semantic replay | identical across 2 runs | Stop |
+| Normalized-pixel replay | identical across 2 runs | Stop |
+| Source and license inventory | complete and approved | Stop |
+| Cross-role exact overlap | 0 | Stop |
+| Cross-role unresolved near overlap | 0 | Stop |
+| Terminal-input usage | false for every record | Stop |
+| Complete count accounting | requested/generated/accepted/quarantined/failed | Stop |
+
+## Validation reason codes
+
+Every failure uses a stable reason code, including:
+
+- `SCHEMA_INVALID`;
+- `CANONICAL_ILLEGAL`;
+- `GRID_INCOMPLETE`;
+- `GOLD_RECOMPILE_MISMATCH`;
+- `TOKEN_OWNERSHIP_INVALID`;
+- `GEOMETRY_INVALID`;
+- `KEEP_NOT_IDENTITY`;
+- `EDIT_NOT_REPLAYABLE`;
+- `NONDETERMINISTIC_SEMANTICS`;
+- `NONDETERMINISTIC_PIXELS`;
+- `SOURCE_OR_LICENSE_INCOMPLETE`;
+- `CROSS_ROLE_EXACT_OVERLAP`;
+- `CROSS_ROLE_NEAR_OVERLAP`;
+- `TERMINAL_DERIVATION_RISK`.
+
+Quarantine manifests preserve every failed record identifier, frozen seed,
+reason code, and diagnostic hash. Visual examples are QA evidence only.
+
+## Post-pass boundary
+
+A sealed pass authorizes a new non-overwriting shared-corpus build with frozen
+distributions. It does not authorize Explicit or LoRA training. Corpus scale is
+chosen from measured generation cost, coverage, and quarantine rate, not from
+current official benchmark scores.

docs/COLLABORATOR_HANDOFF.md → docs/corpus/COLLABORATOR_HANDOFF_2026.08.12.1.md

@@ -1,4 +1,4 @@
-# Collaborator Handoff
+# Collaborator Handoff 2026.08.12.1
 
 ## Goal
 
@@ -23,6 +23,13 @@ Deliverables:
 - exact and perceptual duplicate reports;
 - an English experiment record.
 
+The frozen starting point is the
+[`2026.08.12.1` synthetic-data specification](SYNTHETIC_DATA_SPECIFICATION_2026.08.12.1.md),
+its [coverage matrix](COVERAGE_MATRIX_2026.08.12.1.csv), and the
+[`2026.08.12.1` generation parameters](../../configs/generation_parameters_2026.08.12.1.json).
+Changes require a new CalVer revision; collaborators must not edit the frozen
+files in place.
+
 ## Other work packages
 
 ### Coverage specification

docs/corpus/COVERAGE_MATRIX_2026.08.12.1.csv

@@ -0,0 +1,30 @@
+category,gate_label,phenomenon,requested_count,train_count,development_count,holdout_count,counterfactual_pair_required,primary_risk
+exact_keep,KEEP,simple_regular,12,8,2,2,false,false_edit
+exact_keep,KEEP,hierarchical_header,12,8,2,2,false,false_edit
+exact_keep,KEEP,row_or_column_span,12,8,2,2,false,false_edit
+exact_keep,KEEP,empty_or_sparse_cells,8,6,2,0,false,false_edit
+exact_keep,KEEP,dense_or_multiline_text,8,6,2,0,false,text_loss
+exact_keep,KEEP,mixed_script_or_formula,6,4,2,0,false,text_loss
+exact_keep,KEEP,clean_high_resolution_control,6,4,0,2,false,false_edit
+hard_keep,KEEP,multilevel_irregular_header,12,8,2,2,true,false_edit
+hard_keep,KEEP,mixed_two_dimensional_spans,12,8,2,2,true,false_edit
+hard_keep,KEEP,weak_missing_or_partial_borders,12,8,2,2,true,false_edit
+hard_keep,KEEP,empty_near_empty_or_sparse,8,6,2,0,true,false_edit
+hard_keep,KEEP,dense_small_long_or_multiline_text,8,6,2,0,true,text_loss
+hard_keep,KEEP,rotation_blur_compression_or_noise,8,4,2,2,true,false_edit
+hard_keep,KEEP,color_background_or_alignment_trap,4,4,0,0,true,false_edit
+single_minimal_edit,EDIT,extra_split,12,8,2,2,true,cell_inflation
+single_minimal_edit,EDIT,missing_split,10,6,2,2,true,under_segmentation
+single_minimal_edit,EDIT,incorrect_merge,10,6,2,2,true,content_reassignment
+single_minimal_edit,EDIT,missing_merge,10,6,2,2,true,over_segmentation
+single_minimal_edit,EDIT,span_extent_error,8,6,2,0,true,topology_error
+single_minimal_edit,EDIT,token_ownership_error,6,4,2,0,true,text_loss
+single_minimal_edit,EDIT,geometry_inconsistency,4,4,0,0,true,location_error
+single_minimal_edit,EDIT,localized_grid_coverage_error,4,4,0,0,true,illegal_grid
+complex_correction,EDIT,multilevel_header_with_spans,12,8,2,2,false,topology_error
+complex_correction,EDIT,joint_split_and_merge,12,8,2,2,false,cell_inflation
+complex_correction,EDIT,weak_border_with_topology_error,10,6,2,2,false,missed_edit
+complex_correction,EDIT,empty_cells_with_spans,8,6,2,0,false,ownership_error
+complex_correction,EDIT,dense_multiline_with_topology_error,8,6,2,0,false,text_loss
+complex_correction,EDIT,mixed_script_formula_with_topology_error,6,4,2,0,false,text_loss
+complex_correction,EDIT,rotation_noise_with_topology_error,8,6,0,2,false,geometry_error