Skip to content

Stop NZB job reuse from handing back an already not-wanted job hash - #483

Merged
mash2k3 merged 21 commits into
godver3:devfrom
zab1996:fix-nzb-reuse-not-wanted-hash
Aug 29, 2026
Merged

Stop NZB job reuse from handing back an already not-wanted job hash#483
mash2k3 merged 21 commits into
godver3:devfrom
zab1996:fix-nzb-reuse-not-wanted-hash

Conversation

@zab1996

@zab1996 zab1996 commented Aug 28, 2026

Copy link
Copy Markdown

Summary

Fixes cli_mount NZB reuse loops, junk/trailer collection, rescrape re-bind regressions, queue/mount drift, and adds repair UX plus a Junk Files audit for symlink-mode libraries.

Job reuse & blacklist

  • Reuse paths check is_magnet_not_wanted() on the provider job hash (not just guid/segment ID or “job alive”).
  • Terminal failures (ghost job, folder never appeared, health broken, ffprobe reject, supersede cancel) blacklist hash/guid/segment and reset item fields via shared blacklist_and_cleanup_nzb_failure().
  • Supersede cancel deletes the prior in-flight job when Adding binds a new one; clears health-cache so orphans don’t accumulate.
  • Movie reuse also blocks not-wanted hashes and jobs with known-unplayable files.
  • Bulk rescrape fully clears blacklist state so items can try previously blocked releases again.
  • Drop stale in-memory NZB queue entries when the mount folder is gone.

Size validation (individual movies/episodes only)

  • Reject when downloaded video is <10% of advertised size — in Adding and again at symlink time.
  • Pick largest episode-matching file when a folder has multiple videos (RAR splits); fix climount_sync assigning tiny splits to duplicate rows.
  • Fall back to item.size when scrape_results are cleared (e.g. after rescrape). Fails open when size data is missing; season packs skipped.

Rescrape

  • Clear stale bind fields (debrid_folder_name, scrape_results, nzb_segment_id, etc.) and sync in-memory queues immediately.
  • Block all sibling season-pack reuse while rescrape_original_torrent_title is set; preserve that marker across intermediate NZB failures.
  • Prevents rescrapped episodes from immediately re-binding the dead season pack that triggered the rescrape.

Repair — junk guard

  • Detect junk sources (sample/trailer release tags, size mismatch, relative junk in pack) before NZB/debrid repair runs.
  • Segment-based sample/trailer detection: flags standalone release tags (S01E01.trailer.1080p) but not title words (Punch Drunk Trailer Trashed, Trailer Park Boys).
  • Shared logic in is_unwanted_file() for scrape filtering, repair guards, and Junk Files audit.
  • Skip replacement scrape/submit; blacklist bad source; log skipped_junk_source in activity (Debrid Manager shows Junk outcome).
  • Filter junk replacement candidates from scrape results so real broken files still repair normally.

Repair — in-flight panel (Debrid Manager)

  • New In-Flight Repairs table on the Usenet tab for active nzb_playback_repairs rows (candidate search, pending verification, cleanup).
  • Cancel stops the worker immediately (clears lease/backoff; activity → Cancelled).
  • Send to Wanted cancels and resets the item for fresh scrape; tears down partial repair mounts (broken/candidate cli_mount jobs, symlinks, Plex entries) before re-scraping.

Junk Files audit (Debrid Manager)

  • New Audit → Junk Files tab for symlink-mode libraries.
  • Scans for undersized episodes/movies, sample/trailer tags, split-pack orphans, and NZB size mismatches.
  • Splits likely junk (high confidence) from review-tier older/smaller releases.
  • Fix resets Collected/Checking rows to Wanted, or deletes duplicate rows when a good Collected sibling already exists.
  • Removes orphan junk files from the mount when no symlink depends on them.

Queue / mount alignment

  • remove_nzb / remove_nzb_exact always call the queue delete endpoint after a successful browse delete, so queue.db stays aligned with entries.db.
  • Applies during ffprobe reject, repair teardown, upgrades/supersedes, Junk Files Fix, and manual browse delete.
  • INFO logging when companion queue rows are removed alongside storage delete.

Other

  • Clear _nzb_confirmed_complete on broken-job rejection so Adding retries the next scrape result instead of advancing to Checking.
  • Skip duplicate playback-repair candidate search when the item already collected elsewhere.
  • Raise default Usenet retention days to 5000 for new installs (~13 years unless changed).

Test plan

  • Stuck NZB (folder never appears) → blacklisted, fresh submission on retry (no reuse loop)
  • Healthy in-flight job still reused when hash not blacklisted
  • ~70–90 MiB download on ~4 GiB advertised release → rejected in Adding, blacklisted, next scrape tried
  • Multi-file NZB folder → largest matching file symlinked; tiny splits rejected
  • Rapid scrape cycling → superseded jobs deleted/blacklisted, no orphans
  • ffprobe reject → hash in not-wanted, reuse skipped
  • Junk source flagged broken → repair skipped, logged as Junk (no Wanted reset)
  • In-flight playback repair → visible in Debrid Manager; Cancel / Send to Wanted stops retry loop
  • Junk Files audit → flags undersized/split/mismatch junk; Fix resets to Wanted and removes mount/queue rows
  • Bar Rescue S04E03 “Trailer Trashed” episode title → not flagged as junk after segment-based detection
  • Rescrape → fresh scrape, no sibling pack re-bind, no “not found in checked location” loop
  • pytest tests/test_nzb_size_validation.py tests/test_pick_best_video_file.py tests/test_nzb_supersede_cleanup.py tests/test_nzb_failure_cleanup.py tests/test_rescrape_pack_reuse.py tests/test_nzb_playback_repair.py tests/test_is_unwanted_file.py tests/test_climount_client_remove.py

Both cli_mount job-reuse paths in torrent_processor.py (_process_nzb_result's
DB-dedup check and its /api/torrents title/hash listing match) only verified
a matched job was still alive on the provider (is_nzb_job_alive(), added in
15bca85/5da002a5). A job can be alive yet permanently stuck - e.g. the
"folder never appeared after 10 ticks" health-check failure - and
is_nzb_job_alive() only proves existence, not health, so the reuse kept
handing the same stuck job back out every retry.

The not-wanted list also never tracked the provider job hash itself, only
NZB guid and segment ID, so nothing could catch this even if the reuse path
checked it. Real log evidence: two Love After Lockup jobs ping-ponged every
~90s for 4+ hours, "3 results remain" on every cycle without ever actually
trying a new download, because each retry's title-match reuse just found the
same broken job again.

Now both reuse paths check is_magnet_not_wanted() on the candidate hash
before returning it, falling through to a fresh submission on a match, and
both terminal failure branches in task_nzb_health_check (progress -1 folder-
never-appeared, progress -2 ghost job) add the job hash to the not-wanted set
alongside the existing guid/segment additions.
…lected elsewhere

The candidate-search retry loop (status='awaiting_candidate') never checked
whether the item had already been fixed by something else before searching
for and submitting yet another replacement. The main repair sweep
(_run_repair_inner) and this completion worker both write the same item's
filled_by_torrent_id independently; when they raced, the completion worker
would reject a candidate that had actually already succeeded (source-UUID
mismatch against a stale record) and then blindly submit a second NZB for
the same top-scoring release — producing two identical, fully-downloaded
copies in decypharr with no coordination to cancel either one.

Real example: The Matrix (1999) got two separate 14.41GB jobs of
The.Matrix.1999.BDRip.1080p.TrueHD.5.1.multisub-HighCode submitted ~90s
apart, both completing to 100%, because the first replacement had already
collected successfully by the time this loop reprocessed the row.

Mirrors the existing 'superseded externally' handling already present a few
lines down for the awaiting_collection stage.
@zab1996
zab1996 force-pushed the fix-nzb-reuse-not-wanted-hash branch from 5e21a1b to c03538e Compare August 28, 2026 13:24
zab1996 and others added 16 commits August 28, 2026 11:34
… size

Post-download health check compares cli_mount file size against the selected
scrape result and reuses the existing broken-job retry path when actual size
is below 10% of advertised (movies and single-episode releases only). Clear
the confirmed-complete cache on rejection so rejected jobs retry the next
scrape result in Adding instead of advancing to Checking.

Co-authored-by: Cursor <cursoragent@cursor.com>
Pick the largest episode-matching video when a folder has multiple files,
apply the same advertised-vs-actual size gate before symlinks are created,
and stop climount_sync from assigning tiny RAR splits to duplicate rows.

Co-authored-by: Cursor <cursoragent@cursor.com>
When Adding binds a new cli_mount job for an item, cancel and blacklist the
previous in-flight job so rapid scrape-result cycling cannot leave orphans.
On ffprobe playability failure, also add the cli_mount job hash (and segment
ID) to not-wanted so title-match reuse cannot resurrect a rejected download.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add a shared blacklist/cleanup helper for ffprobe reject, supersede cancel,
health-check broken paths, and failed cli_mount jobs; guard NZB reuse against
not-wanted hashes and unplayable files. Rescrape now clears debrid_folder_name,
scrape_results, and nzb_segment_id so items do not re-bind dead releases.

Co-authored-by: Cursor <cursoragent@cursor.com>
…tion.

Rescrape and reuse paths often clear scrape_results while the item row still
stores the advertised GiB; without this fallback size checks fail open and
junk downloads like 126 MiB 1080p episodes can reach Collected.

Co-authored-by: Cursor <cursoragent@cursor.com>
After rescrape, sync in-memory Checking/Adding queues right away, clear
real_debrid_original_title/location_basename, and skip sibling pack reuse
when the pack title matches rescrape_original_torrent_title.

Co-authored-by: Cursor <cursoragent@cursor.com>
Prevents rescrapped episodes from re-binding dead season packs after NZB failure cleanup overwrote the marker, which caused recurring "not found in checked location" loops.

Co-authored-by: Cursor <cursoragent@cursor.com>
Detect trailers, samples, and size-mismatch junk before repair runs, blacklist the bad source, log a skipped_junk_source activity row, and filter junk replacement candidates so real broken files still repair normally.

Co-authored-by: Cursor <cursoragent@cursor.com>
…tions.

Debrid Manager now lists active nzb_playback_repairs rows (verification retries, candidate search, cleanup) and lets you stop them immediately instead of waiting on backoff.

Co-authored-by: Cursor <cursoragent@cursor.com>
Updates the settings schema default and all install/setup fallbacks so fresh configs filter NZBs up to ~13 years old unless the user changes it.

Co-authored-by: Cursor <cursoragent@cursor.com>
When aborting a playback repair via Send to Wanted, remove broken and
candidate cli_mount jobs plus their symlinks/Plex entries before
re-scraping so stale files don't duplicate in the library.

Co-authored-by: Cursor <cursoragent@cursor.com>
Scans for undersized, sample, split-pack, and NZB-mismatch symlinks;
splits likely junk from review-tier older releases; resets items to
Wanted or deletes duplicate rows when a good Collected sibling exists.

Co-authored-by: Cursor <cursoragent@cursor.com>
remove_nzb and remove_nzb_exact now call the queue delete endpoint
even when browse delete succeeds, so queue.db stays aligned with
entries.db during ffprobe reject, repair teardown, and upgrades.

Co-authored-by: Cursor <cursoragent@cursor.com>
When remove_nzb removes the browse entry first, log at INFO when the
follow-up queue delete succeeds so automatic cleanup is easy to trace.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ives

Replace bare substring checks with release-tag heuristics so episode titles
like Punch Drunk Trailer Trashed are not flagged while real extras such as
S01E01.trailer.1080p still are. Share the logic between scrape filtering,
repair junk guards, and the Junk Files audit.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@zab1996
zab1996 marked this pull request as ready for review August 29, 2026 01:32
When the general repair sweep submits a replacement while the completion worker still sits in awaiting_candidate, finalize or defer instead of launching a parallel scrape that submits a second NZB.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mash2k3
mash2k3 merged commit 6cdf66c into godver3:dev Aug 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants