Skip to content

STOPWORDS_ES has e and o but not y or u, so the commonest Spanish conjunction counts as a content word #76

Description

@Railly

STOPWORDS_ES (semantic.ts:496) contains e and o but not y or u. The most common conjunction in Spanish scores as a content word:

contentWordCount('y')       -> 1
contentWordCount('y de la') -> 1

e and u are the variants of y and o used before a matching vowel sound. Including one of each pair and not the other has no linguistic basis; it reads as an oversight in the list rather than a decision.

Blast radius

Every consumer of the shared floor inherits the inflation, because MIN_CONTENT_WORDS = 2 is compared against a count that treats a conjunction as content:

  • repeatedPhrases (semantic.ts)
  • findRepeatedPhrases (verify.ts:336)
  • repetitionCuts (auto-cut.ts:668)
  • measureRepeatWindows (auto-cut.ts:773)

A two-word phrase like y bueno is exactly the connective tissue the floor exists to reject, and it currently scores 1 rather than 0. Any phrase that pairs y with one real word reaches the floor on the strength of the conjunction alone.

Found while building a fixture for #72. Both detectors there return zero on the issue's own text, so the mechanism is confirmed independently of this.

Suggested check

Whatever the fix, worth re-running the floor's own tests: verify.test.ts pins y ese es el as discounted and le damos clic a as kept, and both sit near the boundary.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions