Make chat search fast and reliable using only its dedicated index - #993
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Chat search can time out because it repeatedly searches both the dedicated index and legacy entry history for each visible conversation. It also loads legacy history again to resolve result metadata. This change searches only
session_entry_search, batches the visible-session filter, and returns participant-specific metadata directly with each hit.Two ordered migrations first install transactional indexing for inserts, edits, and deletes, then fill missing historical rows, including holes below the highest indexed sequence. The trigger also covers older writers during rollout or rollback. The backfill and coverage audit exclude indexed rows before parsing payloads, confirmed with EXPLAIN; text extraction runs once per missing row. Backfill runs as one transaction and may take minutes on large histories; it must finish in the separate migration task before promotion. The repair script now measures actual gaps; the delayed, best-effort turn-end indexer is removed. Participant visibility windows and project membership checks remain enforced.
The search dialog invalidates stale requests immediately when the query changes or closes, keeps the 150 ms debounce, and ignores ordinary Enter while loading. Server failures no longer incorrectly tell the user their connection is broken.
Validation: 97 affected unit/API/UI regression tests and all 53 Postgres store tests passed, plus root and web typechecks, lint, formatting, and the web build. Independent migration/concurrency and authorization/UI reviews completed. Live browser QA used the real local core, web stack, and Postgres: new user/assistant messages, prefix searches, empty results, and rapid query changes. On that small synthetic dataset, API requests took 2–16 ms, excluding debounce. Slack pool connection checks failed, so this web-specific QA used the supported no-Slack dev mode.
Demo uses only synthetic data. Before: original UI with a deliberately failed search API (to reproduce the error presentation, not to benchmark the old backend). After: successful search against real local Postgres. A persistent hosted dev demo is impractical because it requires the local authenticated core/database.