chore(scripts): fuzzy duplicate-search workflow for merge candidates

[mbarlow12]

Throwaway developer tooling under scripts/, opened as a Draft because others may find it useful. Not wired into CI, not imported by any package, no lockfile change.

Why

entity-linkage blocks on name.strip().casefold() and confirms on country.strip().upper() — both exact. On the demo corpus (9,187 resources) that finds zero groups: GEM writes Darquain Oil Field (Iran) while CCR/BC/WM write Darquain, so the names never collide. The blocking key is the constraint, not the comparison.

Normalizing the decoration away finds 273 high-confidence pairs on the same data.

bare source	GEM
Darquain (IRN, ccr)	Darquain Oil Field (Iran)
DEEP BASIN (CAN, bc)	Deep Basin Gas Field (British Columbia, Canada)
Dębowiec Śląski (POL)	Dębowiec Śląski Gas Field (Poland)

Workflow

uv run scripts/fetch_resources.py                                   # corpus -> data/resources.jsonl
uv run --with rapidfuzz --with numpy scripts/find_candidates.py     # -> data/candidates.json
# review: set "decision" to merge/skip per group
uv run scripts/create_merge_candidate.py --from-file --dry-run
uv run scripts/review_merge_candidates.py list --mine

The search script proposes and explains, never decides — every group comes out with "decision": null plus the signals that justified it. Output is gated to a high confidence tier by default; merge that tier, re-fetch, re-run wider. Approving mints a new resource, so snapshots go stale by design.

Matching

• exact normalized name; a - split also blocks on each half, but only when that half is the rarest segment on both sides — otherwise Belmont County - Ascent matches Harrison County - Ascent on the operator.
• fuzzy token_sort_ratio >= 90 within a country bucket. Deliberately not token_set_ratio: it scores MARTIN against MARTIN CREEK at 100 and chains the corpus into one 1,021-member blob.
• geo within 10 km with name similarity >= 70.
• Vetoes on conflicting rare parentheticals (Daqing (Lamadian) vs (Saertu)) and directional qualifiers (Blueberry East vs West). Parenthetical rarity is learned from the corpus, so (Iran) is treated as boilerplate.

Verified

Run end-to-end against the PR-0242 demo deployment:

• 8/8 hand-picked ground-truth duplicates group correctly; Daqing (3 reservoirs) and Dalmatian North/South correctly stay apart.
• Created 6 candidates, approved one; merged resource 9226 correctly combines the ccr and gem records. Re-running skipped already-created groups.
• ruff check and ruff format --check pass on scripts/.

Notes for reviewers

• scripts/data/ is gitignored — fetched snapshots and review state are per-run and per-deployment.
• _common.py defaults to the PR-0242 demo URL (inherited from the original helper); that deployment is ephemeral, so set STITCH_API_BASE_URL in .env. Credentials come from STITCH_CLIENT_PRIVILEGED_BEARER_TOKEN; no token is ever printed.
• The normalizer is not folded into entity-linkage here. If it proves out, that is a separate pre-approved architectural conversation per CONTRIBUTING.md, and it would want the DB-side normalized-name match already tracked as STIT-573.

AI assistance

Written with Claude Code. Verification was the ground-truth pass/fail check above plus the live end-to-end run against the demo deployment; the 273-row high-confidence tier was read in full during review, which is where the two held-back groups (OTHER AREAS, a catch-all bucket, and Qasr (Pre-2021), a vintage qualifier) came from.

[github-actions]

CD summary 594868a

Frontend: https://witty-mushroom-017a3dc1e-243.westus2.1.azurestaticapps.net

Deployments (4)

service	url	fqdn
api	open	pr-0243-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io
entity-linkage	open	pr-0243-el.purplegrass-c07d0a94.westus2.azurecontainerapps.io
frontend	https://witty-mushroom-017a3dc1e-243.westus2.1.azurestaticapps.net
stitch-llm	open	pr-0243-llm.purplegrass-c07d0a94.westus2.azurecontainerapps.io

Database (1)

db_name	postgres_host	postgres_port	postgres_db
pr_0243	stitch-dev.postgres.database.azure.com	5432	pr_0243

Jobs (2)

job	image	postgres_db	api_url	auth_mode
db-migrations	ghcr.io/rmi/stitch-api:pr-0243@sha256:591538fbaa81adc5618c7815742ce19d8591921291cf1d1cb945ac44e6530c37	pr_0243
seed	ghcr.io/rmi/stitch-seed:pr-0243@sha256:5e5d4772db6aac2c37d189f4d1df0601725ba6ad922a00b1bf59176bff807fc1		https://pr-0243-api.purplegrass-c07d0a94.westus2.azurecontainerapps.io/api/v1	stitch-client-bearer-token

Images (4)

build_time	commit_time	git_sha	image	image_digest
2026-08-18T23:36:19Z	2026-08-18T23:36:03Z	abaf4ef	ghcr.io/rmi/stitch-api:pr-0243	ghcr.io/rmi/stitch-api:pr-0243@sha256:591538fbaa81adc5618c7815742ce19d8591921291cf1d1cb945ac44e6530c37
2026-08-18T23:36:21Z	2026-08-18T23:36:03Z	abaf4ef	ghcr.io/rmi/stitch-entity-linkage:pr-0243	ghcr.io/rmi/stitch-entity-linkage:pr-0243@sha256:8ceef22052337354202b9c7cb0efa1c8c36a76f70adebe3e75e43611312f340d
2026-08-18T23:36:20Z	2026-08-18T23:36:03Z	abaf4ef	ghcr.io/rmi/stitch-seed:pr-0243	ghcr.io/rmi/stitch-seed:pr-0243@sha256:5e5d4772db6aac2c37d189f4d1df0601725ba6ad922a00b1bf59176bff807fc1
2026-08-18T23:36:22Z	2026-08-18T23:36:03Z	abaf4ef	ghcr.io/rmi/stitch-stitch-llm:pr-0243	ghcr.io/rmi/stitch-stitch-llm:pr-0243@sha256:ac7b26a10ddddf115eed3e198207ca59a4a38ea1186b6bc0bd4aef28b10a9d05