Skip to content

Search by name only when asked, and only when nothing else answered - #60

Merged
oscarvalenzuelab merged 1 commit into
mainfrom
fix/fuzzy-only-when-asked-for-and-only-when-exact-failed
Sep 1, 2026
Merged

Search by name only when asked, and only when nothing else answered#60
oscarvalenzuelab merged 1 commit into
mainfrom
fix/fuzzy-only-when-asked-for-and-only-when-exact-failed

Conversation

@oscarvalenzuelab

Copy link
Copy Markdown
Collaborator

Closes #59.

The bug

$ src2purl /src/package --output-format json     # no --enable-fuzzy
Strategies: Hash Search, Web Search, SCANOSS

exact  0.85  pkg:npm/express@4.18.2  express
fuzzy  0.83  None                    senam5jari-975
fuzzy  0.83  None                    voiti-1xbet-rech-idet-urxfhkrm
fuzzy  0.83  None                    john-wick-4-izle-full-filmis-4khd-izle-...
... seven more

Two hundredths of confidence separate express from npm spam, which is close
enough that --confidence-threshold cannot tell them apart. The spam carries
purl: null, so nothing downstream can act on it either.

Cause

Keyword search matches on the project name, so it answers with whatever else
shares that name. There were two call sites:

  • the fuzzy fallback in _find_matches — checked enable_fuzzy_matching
  • phase 1 of identify_packages — checked nothing

The second also ran before phase 2's manifest parsing, which is what produces
the exact express match. With --use-swh off (the default) phase 1 finds
nothing, so the search fired on every run, and phase 2 then found express beside
its results.

The flag's help says fuzzy runs "when exact matches fail". Whether they had
failed was not known at that point.

The fix

Both call sites are gone. identify_packages runs the search once, after
everything else has had its turn, when nothing was identified and the flag was
passed — which is what the help already described.

The _find_matches fallback is removed rather than gated, so --use-swh goes
through the same single path. It was calling the same _find_keyword_matches
into the same _process_matches; nothing is lost but the position.

Measured

input flag before after
express none 1 exact + 10 fuzzy 1 exact
express --enable-fuzzy 1 exact + 10 fuzzy 1 exact
unidentifiable dir none 10 fuzzy 0
unidentifiable dir --enable-fuzzy 10 fuzzy 10 fuzzy

The flag now does what it says in all four.

Tests

122 pass, 7 new, covering both the default and --use-swh routes. No new ruff
findings. orchestrator.py was already unformatted before this change and black
is non-blocking here.

Not in scope

The issue also notes express coming back with "official": false.
is_official_organization checks a hardcoded ~30-entry allowlist in
extractor.py that does not include expressjs, so the field means "in our
known-official list" rather than "is the canonical repository". That is a narrow
definition rather than a defect in this path; widening or replacing that list is
a separate decision.

One behaviour note: the removed inner fallback seeded the search from
dir_candidates[0].path while the single one uses the invocation path. For a
normal project-root run these are the same; they could differ if someone ran
--use-swh --enable-fuzzy on a wrapper directory and relied on the scanner's
top candidate to pick the search term.

Run against the extracted express@4.18.2 tarball with no --enable-fuzzy,
src2purl returned the right answer and ten pieces of npm spam:

    exact  0.85  pkg:npm/express@4.18.2  express
    fuzzy  0.83  None                    senam5jari-975
    fuzzy  0.83  None                    voiti-1xbet-rech-idet-urxfhkrm
    ... eight more

Two hundredths of confidence separate express from the noise, which is close
enough that --confidence-threshold cannot tell them apart, and the noise carries
no PURL, so nothing downstream can act on it either. It is expensive to
distinguish from signal and nobody asked for it.

Keyword search matches on the project name, so it answers with whatever else
shares that name. There were two call sites. The one in the fuzzy fallback
checked the flag. The one in phase 1 did not check anything, and ran before the
manifest parsing in phase 2 that produces the exact match. With Software
Heritage off, which is the default, phase 1 finds nothing, so the search fired
every time and phase 2 then found express beside its results. The flag's help
says fuzzy runs when exact matches fail; whether they failed was not known yet.

Both are gone. identify_packages runs the search once, after everything else has
had its turn, when nothing was identified and the flag was passed. That is what
the help already described.

    express, no flag              1 exact
    express, --enable-fuzzy       1 exact
    unidentifiable dir, no flag   nothing
    unidentifiable dir, fuzzy     10 keyword matches

The issue also notes express coming back with "official": false.
is_official_organization checks a hardcoded list of about thirty organisations
that does not include expressjs, so the field means "in our known-official list"
rather than "is the canonical repository". Widening that list, or replacing it,
is a separate decision and is left alone here.

Closes #59
@cla-bot cla-bot Bot added the cla-signed label Sep 1, 2026
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

License Check Report

ℹ️ No licenses detected in changed files.

@oscarvalenzuelab
oscarvalenzuelab merged commit a28fb39 into main Sep 1, 2026
9 checks passed
@oscarvalenzuelab
oscarvalenzuelab deleted the fix/fuzzy-only-when-asked-for-and-only-when-exact-failed branch September 1, 2026 19:42
@oscarvalenzuelab oscarvalenzuelab mentioned this pull request Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fuzzy matches are returned when an exact match succeeded and --enable-fuzzy was not passed

1 participant