Search by name only when asked, and only when nothing else answered - #60
Merged
oscarvalenzuelab merged 1 commit intoSep 1, 2026
Conversation
Run against the extracted express@4.18.2 tarball with no --enable-fuzzy,
src2purl returned the right answer and ten pieces of npm spam:
exact 0.85 pkg:npm/express@4.18.2 express
fuzzy 0.83 None senam5jari-975
fuzzy 0.83 None voiti-1xbet-rech-idet-urxfhkrm
... eight more
Two hundredths of confidence separate express from the noise, which is close
enough that --confidence-threshold cannot tell them apart, and the noise carries
no PURL, so nothing downstream can act on it either. It is expensive to
distinguish from signal and nobody asked for it.
Keyword search matches on the project name, so it answers with whatever else
shares that name. There were two call sites. The one in the fuzzy fallback
checked the flag. The one in phase 1 did not check anything, and ran before the
manifest parsing in phase 2 that produces the exact match. With Software
Heritage off, which is the default, phase 1 finds nothing, so the search fired
every time and phase 2 then found express beside its results. The flag's help
says fuzzy runs when exact matches fail; whether they failed was not known yet.
Both are gone. identify_packages runs the search once, after everything else has
had its turn, when nothing was identified and the flag was passed. That is what
the help already described.
express, no flag 1 exact
express, --enable-fuzzy 1 exact
unidentifiable dir, no flag nothing
unidentifiable dir, fuzzy 10 keyword matches
The issue also notes express coming back with "official": false.
is_official_organization checks a hardcoded list of about thirty organisations
that does not include expressjs, so the field means "in our known-official list"
rather than "is the canonical repository". Widening that list, or replacing it,
is a separate decision and is left alone here.
Closes #59
License Check Reportℹ️ No licenses detected in changed files. |
oscarvalenzuelab
deleted the
fix/fuzzy-only-when-asked-for-and-only-when-exact-failed
branch
September 1, 2026 19:42
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #59.
The bug
Two hundredths of confidence separate express from npm spam, which is close
enough that
--confidence-thresholdcannot tell them apart. The spam carriespurl: null, so nothing downstream can act on it either.Cause
Keyword search matches on the project name, so it answers with whatever else
shares that name. There were two call sites:
_find_matches— checkedenable_fuzzy_matchingidentify_packages— checked nothingThe second also ran before phase 2's manifest parsing, which is what produces
the exact express match. With
--use-swhoff (the default) phase 1 findsnothing, so the search fired on every run, and phase 2 then found express beside
its results.
The flag's help says fuzzy runs "when exact matches fail". Whether they had
failed was not known at that point.
The fix
Both call sites are gone.
identify_packagesruns the search once, aftereverything else has had its turn, when nothing was identified and the flag was
passed — which is what the help already described.
The
_find_matchesfallback is removed rather than gated, so--use-swhgoesthrough the same single path. It was calling the same
_find_keyword_matchesinto the same
_process_matches; nothing is lost but the position.Measured
--enable-fuzzy--enable-fuzzyThe flag now does what it says in all four.
Tests
122 pass, 7 new, covering both the default and
--use-swhroutes. No new rufffindings.
orchestrator.pywas already unformatted before this change and blackis non-blocking here.
Not in scope
The issue also notes express coming back with
"official": false.is_official_organizationchecks a hardcoded ~30-entry allowlist inextractor.pythat does not includeexpressjs, so the field means "in ourknown-official list" rather than "is the canonical repository". That is a narrow
definition rather than a defect in this path; widening or replacing that list is
a separate decision.
One behaviour note: the removed inner fallback seeded the search from
dir_candidates[0].pathwhile the single one uses the invocationpath. For anormal project-root run these are the same; they could differ if someone ran
--use-swh --enable-fuzzyon a wrapper directory and relied on the scanner'stop candidate to pick the search term.