Expose kind and entity filters on knowledge search - #16
Conversation
hybridSearch already filtered by document kind and entity id; the plane's search() and the HTTP search route never surfaced those params, so no in-process or HTTP caller could reach them. Thread kinds/entityIds through KnowledgeSearchParams and accept kinds/ entity_ids on POST /api/knowledge/search. No filter = unchanged behavior.
…ers (CL-5021) Review found the kinds/entityIds wording promised an exact post-fusion guarantee the implementation doesn't provide: hybridSearch's dense/ semantic channel has no kind or entity predicate, so a fused hit can surface a document that doesn't match the requested filter. Making the dense channel filter-aware is a materially larger, untestable- without-a-live-Postgres change (raw SQL query shape, hnsw/index implications) than this ticket's scope, so instead every place the filter is documented now says plainly: these narrow the lexical leg only, and callers needing an exact guarantee must filter hits themselves. Also documents that an empty array on either filter is equivalent to omitting it (no filter, not "match nothing"), including for the empty-query-requires-a-structured-filter check in hybridSearch, and adds a route test pinning that [] passes through unchanged rather than being silently dropped.
kinds/entityIds only reached fetchLexicalCandidates; fetchDenseCandidates had no equivalent predicate, so a document that didn't match the caller's filter could still surface through the dense leg once RRF fusion merged both channels. Push the same predicate into the dense query (kd.kind = ANY(...) and a knowledge_edge subquery for entityIds, bound the same way fetchChunkVectors already binds a text[] param) so every fused hit matches the requested filter regardless of which channel found it. Updates the docs/comments that previously called this out as a known limitation.
|
Pushed a fix for the fusion-bypass gap called out in the "Known limitation" section: Updated Added |
…me (CL-5021) - Extract ChannelFilterFields, shared by LexicalCandidateParams, FetchDenseCandidatesArgs, and HybridSearchArgs instead of redeclaring kinds/entityIds three times. - Document on fuseRrf why filtering each channel independently before fusion is safe: RRF only reads each survivor's rank within its own channel, never the survivor count or pool fraction, so removing non-matching candidates upstream can't distort it. A score-blending fusion would not share this property. - Rename the routes.test.ts empty-array test to match the file's short-name-plus-comment convention instead of embedding the rationale in the test name itself.
… layers (CL-5021) KnowledgeSearchParams used the bare ?: T form for k/kinds/entityIds while services/search.ts's ChannelFilterFields already used ?: T | undefined. Widen KnowledgeSearchParams to match, since callers spread optional values that may be explicitly undefined. Also annotate the hybridSearch test double's return type so mockImplementation call sites can return a narrower/empty result without conflicting with the type inferred from the first mock body.
Guards the invariant just documented on fuseRrf: score depends only on a candidate's own rank within its channel, never how many candidates that channel started with. Constructs inputs where a percentile/pool-size-aware blend would rank oppositely, so a future switch away from rank-based fusion fails this test instead of silently breaking filter safety.
1 participant
Summary
hybridSearchalready implementskinds/entityIdsfiltering; nothing surfaced them. Threads both through:KnowledgeSearchParams(in-process plane callers) →hybridSearchPOST /api/knowledge/searchrequest body (kinds?,entity_ids?) → the planefetchLexicalCandidates) and dense (fetchDenseCandidates) — before RRF fusion runs, so every fused hit matches the requested kind/entity. There is no dense-only escape hatch: a document that doesn't match the filter is never a candidate on either leg.hybridSearch.kind(CL-5011), the authorization asymmetry (CL-5022).Closes CL-5021
Test plan
src/routes/routes.test.ts): kinds/entity_ids thread through, absent filters stay unset, empty arrays pass through unchanged (not silently dropped at the route layer), non-string-array kinds rejected with 400src/knowledge.test.ts):createKnowledgePlane().search()passes kinds/entityIds straight tohybridSearchsrc/services/search.test.ts,fetchDenseCandidates kind/entity filtering): a fake postgres handle evaluates the actual generated SQL/params against a canned two-row dense result set (one row matching the filter, one that would otherwise leak through via pure semantic similarity) — fails if either predicate is dropped from the dense querybun install/bun testlocally (sandboxed, OOM-prone dev machine) — CI is the verification gate for this PR