Skip to content

check-glossary: a short term stems to three characters and matches unrelated words #12

Description

@mairas

Found by CodeRabbit on PR #1, then measured against hatlabs/halpi2.

inflectable() builds its pattern from w[: max(3, len(w) - 3)] + r"\w*", so a term keeps all but its last three characters, floored at three. SHORTEST_TERM is 5, so the shortest accepted term keeps only three: Finnish laite compiles to \blai\w*, which matches laiva. The checker then reports a prescribed term as used when a different word is doing the job — a false green in the checker whose own docstring says that one which passes when it should not is no checker at all.

The obvious fix is wrong. Raising the floor to 4 was measured across all nine of halpi2's glossaries: eight are byte-identical, and Italian flips from exit 0 to exit 1 on a term that is genuinely in use.

term: 'solder nut' -> 'dado da saldare'
  floor 3: \bdad\w*(?:\W+\w+){0,2}\W+da\w*(?:\W+\w+){0,2}\W+sald\w*
           matches 'aggiunti dadi da saldare per facilitare il montaggio'
  floor 4: \bdado\w*(?:\W+\w+){0,2}\W+da\w*(?:\W+\w+){0,2}\W+sald\w*
           matches nothing

Italian pluralises dado to dadi, changing the fourth character, so a four-character stem cannot match the correct translation. The three-character stem is doing real work for exactly the languages the glossaries cover; it is also what lets lai reach laiva.

A fix therefore needs something other than a floor: a stem proportional to the word rather than a fixed truncation, a bounded set of per-language endings, or requiring the matched word to be within an edit distance of the term. Whatever it is, verify it against all nine glossaries — the current behaviour is the baseline, and eight of the nine must not move.

Related: the same function's {0,2} intervening-word allowance and its non-word-initial branch have no tests (#10).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions