Skip to content

Catch short-phrase loops, counter/emoji floods, and zero-width floods - #539

Merged
TheGreatAxios merged 5 commits into
mainfrom
cl-6902-repetition-detector-misses-short-phrase-loops-and-zero
Aug 23, 2026
Merged

Catch short-phrase loops, counter/emoji floods, and zero-width floods#539
TheGreatAxios merged 5 commits into
mainfrom
cl-6902-repetition-detector-misses-short-phrase-loops-and-zero

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Collaborator

Two holes in degenerate-output detection let model loops burn to the token limit, plus additional flood shapes surfaced by the trace audit (670K wasted streamed chars, gpt-5.5 the worst offender).

Short-phrase loops

A live subagent emitted "Groaning. " (10-char unit) ~1,363 times; neither detector could see it.

  • src/subagent/repetition.ts: windowMinChars 16 → 8, repeatThreshold 8 → 16. Minimum exactly-periodic span stays at 8 × 16 = 128 chars. "- item\n" normalizes to 7 chars and stays under the floor; 6-row table separators, 3 identical code lines, and repeat(4) prose stay far below 16 consecutive repeats.
  • src/tui/stall-watchdog.ts: REPETITION_MIN_PERIOD 24 → 8, REPETITION_MIN_REPEATS 8 → 24 (3x, in step with the 3x-lower floor; minimum span stays at 192 chars). REPETITION_MIN_DISTINCT_CHARS untouched — "Groaning. " has 9 distinct chars and triggers; the burst-of-"x" fixture still does not.

Counter / timestamp / fence / emoji floods

Incrementing counters (14279 14280 …, 5620/5620. 5621/5621. …) are never byte-periodic; "0% ", "}\n", "```\n", "🤔 " units fall under the plain window floor. Added `DEFAULT_TEXT_FOLDED_REPETITION_CONFIG` — a second, digit-folded pass over text streams (`windowMinChars` 2, `repeatThreshold` 64, `maxFoldedPeriodChars` 16). The 16-char folded-period cap refuses prose-shaped folds (numbered lists and tables fold to ~30–50 char periods), and the 64-repeat bar keeps a legitimate short enumeration ("print 1..40") untripped. Fixtures cover all observed shapes, including the surrogate-pair emoji unit.

Zero-width floods

normalize() strips invisibles before the periodicity check, so streams of thousands of U+200C/U+200D (observed 500–53,000 per stream in 35+ subagent partials) normalized to a short healthy string and were never flagged. Added trackContentlessGrowth (pure reducer in repetition.ts): flags any 2048-raw-char window whose visible (invisible-stripped, whitespace-removed) content stays under 32 chars. Wired into the subagent run-loop abort path (src/subagent/run.ts) for both text and thinking deltas, with a distinct abort reason naming invisible/contentless output; it shares the repetition salvage path so the parent still gets the evidence tail.

Checks

typecheck, build, and test all pass (5,132 tests). The lint step of bun run check fails identically on clean origin/main (main's CI lint job is currently red; ~600 pre-existing prettier violations) — not addressed here to avoid a repo-wide reformat.

Deferred

Keeping degenerate text out of persisted turn history is follow-up scope and deliberately not attempted here.

Fixes CL-6902

Holes in degenerate-output detection let model loops burn to the token
limit (verified from live traces; 670K wasted streamed chars observed):

- Short-phrase loops: a live subagent emitted a 10-char unit
  ("Groaning. ") ~1,363 times; the 16-char window floor and the
  watchdog's 24-char period floor both missed it. Lower the floors to 8
  and raise the repeat thresholds 2x/3x so the minimum exactly-periodic
  span stays at 128 / 192 chars.

- Counter/timestamp/fence/emoji floods: incrementing counters are never
  byte-periodic, and "0% ", "}\n", "```\n", "🤔 " units fall under the
  plain window floor. Add a second, folded pass over text streams
  (windowMinChars 2, repeatThreshold 64, maxFoldedPeriodChars 16) —
  the period cap refuses prose-shaped folds (numbered lists, tables).

- Zero-width floods: detectRepetition strips invisibles before the
  periodicity check, so a wall of U+200C/U+200D normalizes to a short
  healthy string. Add a contentless-growth guard that flags a raw-growth
  window (2048 chars) whose visible content stays under 32 chars, wired
  into the subagent run-loop abort path with its own abort reason.
@linear-code

linear-code Bot commented Aug 23, 2026

Copy link
Copy Markdown

CL-6902

@TheGreatAxios
TheGreatAxios merged commit 1a903fc into main Aug 23, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant