feat(curator): add governed maintenance and accuracy evaluation - #92
Closed
humandebri wants to merge 2 commits into
Closed
feat(curator): add governed maintenance and accuracy evaluation#92humandebri wants to merge 2 commits into
humandebri wants to merge 2 commits into
Conversation
Add strict AI annotation, proposal recall, cohort scoring, and reusable controlled seed tooling. Validate retained seed manifests and exact write batches before any staging mutation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Changes
curator scan, strictkinic.curator.plan.v1validation, dry-run-first proposal selection, etag revalidation, and one atomic batch only when--confirmis supplied./Sourcesread-only and reject move, delete, retry, overwrite, cross-store mixing, low-confidence proposals, duplicate targets, and batches over 100 operations.curator_accuracy_evalfor answer-masked annotation input, two-run annotation comparison, explicit third-run adjudication, deterministic and semantic scoring, and proposal recall enforcement./Sources, with strict manifest validation and exact batch validation immediately before every write.controlled_seedandstaging_existingcohorts so synthetic fixtures cannot satisfy the real-data sample threshold.kinic-wiki-curatorskill and update CLI, staging, and accuracy documentation. Curator artifacts remain private and ignored by Git.Verification
./.local/check.shcargo test -p kinic-vfs-cli curator_accuracy --locked(22 accuracy and seed tests passed)cargo clippy -p kinic-vfs-cli --all-targets --all-features -- -D warningscargo fmt --all -- --checknode scripts/check-docs-links.mjs(24 files passed)bash -n scripts/staging/seed_curator_accuracy.shgit diff --checkkinic-vfs-cli curatorandcurator_accuracy_evalStaging Evaluation
insufficient_sample; the result is not production accuracy evidence.0.5; a fresh same-snapshot semantic annotation and plan are required before claiming a semantic pass.Impact
query_contextbehavior.0600under the ignored.benchmarks/tree.