perf(wiki): replace heavy UI deps with lightweight self-built implementations - #95
Merged
Merged
Conversation
Address review findings for the recall feature: - Add stale-state revalidation around the Add-context fetch so a slow recall-fetch cannot insert a previous conversation's memory after navigation, database change, or recall disable. - Detect ChatGPT SPA navigation via DOM mutation + location.href instead of the isolated-world history pushState/replaceState wrappers that the page router never calls. - Exclude the current conversation from recall candidates by the real Evidence Source path (title slug + provider:conversationId hash) and accept /c/, /chat/, and /app/ conversation URL forms. - Unwrap Candid optional preview/excerpt/snippet fields in recall cards. - Ignore IME composition Enter keys (isComposing / keyCode 229).
…ntations - Replace react-markdown/remark-gfm/@flowershow with a self-built GFM markdown renderer (lib/markdown-renderer.tsx) incl. reference links, table alignment, nested lists, and bare-URL autolinking. - Replace sonner with a small self-built toast (components/ui/toast.tsx). - Make pdfjs client-only via import.meta.env.SSR so it is excluded from the server bundle (server pdf chunk removed). - Add Cloudflare Cache API caching for public pages in src/server.ts, guarded against undefined caches. - Drop react-markdown, remark-gfm, @flowershow/remark-wiki-link, sonner from package.json (-101 packages). Verified with typecheck, oxlint, 89 vitest tests, build, and wrangler deploy --dry-run. Playwright comparison against react-markdown shows equivalent structure.
…build Replace next/og (and react/react-dom) in the wiki-generator worker with satori + @resvg/resvg-wasm. next/og and @vercel/og both load a default font via new URL(..., import.meta.url) at module scope, but import.meta.url is undefined in the Workers runtime, so og generation crashed on startup. The worker now builds with @cloudflare/vite-plugin, which precompiles the resvg wasm as a compiled-wasm module. Noto Sans JP is fetched at runtime and passed explicitly, so CJK titles render; on font failure it falls back to Latin. link-preview node tests now cover helpers only; the wasm render path is verified via wrangler dev.
Replace the unicode61 tokenizer with FTS5 trigram so Japanese/CJK text can be matched by substring without manual bigram expansion. Add migration 003 that rebuilds the index, cap FTS term length, and defer sub-3-char terms to the path/content_substring stages.
Add the useSyncExternalStore server snapshot so SSR does not throw, and pass the parse range end into inline emphasis/strikethrough so closing markers outside a link destination cannot corrupt link labels.
Add POST /recall-search to the generator for LLM semantic reranking of Recall hits, wired through the clipper when recallUrl/recallToken are configured with lexical fallback. Apply review findings: keep content_substring/path_substring hits, insert Add context around the matched char offset (Light preview, centered 3000-char window), scope send detection to the composer form, merge duplicate hits keeping the best score and union reasons, capture URL/generation before awaiting config and cancel pending schedules on navigation, and encode path segments in source URLs.
Extend the content-substring search stage to run for ASCII-only queries that contain terms shorter than 3 characters, which the FTS trigram tokenizer cannot match. ASCII queries fold both sides to lowercase so short terms like "AI" match content case-insensitively; CJK queries keep the fast plain substring scan. Add a scale test that rebuilds the FTS index through migration 003 at 500 nodes and records the index growth ratio, plus a test documenting that every search score is finite and negative so consumers can rely on the sign without a fixed magnitude floor.
Remove the -1000 recall score floor, which silently dropped content_fts matches for terms that are common in the store (bm25 magnitude shrinks as term frequency rises, so common-memory terms scored near zero were filtered out). Ranking now relies on the match-reason allowlist and score ordering only; hits without a finite score are still excluded.
…ilure Bundle a small Latin subset of Noto Sans JP as a base64 constant so previews still render when the jsdelivr font CDN is unreachable, and wrap the render path so any layout or wasm failure returns a static placeholder PNG instead of a 500. Restore automated render-path coverage with the font fetch and rasterizer stubbed so the pipeline is exercised under plain node --test.
…loudflare-optimize # Conflicts: # extensions/wiki-clipper/popup/popup.css # extensions/wiki-clipper/popup/popup.html # extensions/wiki-clipper/popup/popup.js # extensions/wiki-clipper/src/chatgpt-recall.js # extensions/wiki-clipper/src/content-ui.tsx # extensions/wiki-clipper/src/offscreen.js # extensions/wiki-clipper/src/recall-context.js # extensions/wiki-clipper/src/recall.js # extensions/wiki-clipper/src/service-worker.js # extensions/wiki-clipper/src/vfs-actor.js # extensions/wiki-clipper/tests/chatgpt-recall.test.mjs # extensions/wiki-clipper/tests/offscreen.test.mjs # extensions/wiki-clipper/tests/recall-context.test.mjs # extensions/wiki-clipper/tests/recall.test.mjs # extensions/wiki-clipper/tests/settings.test.mjs
Drop the LLM semantic rerank integration (Recall search URL/Token settings, offscreen generator branch, worker /recall-search endpoint) ahead of Chrome Web Store submission. Keep the lexical hardening from the same change set (charOffset context window, substring reasons, duplicate merge, send scoping).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Replace heavy UI dependencies in the wikibrowser (Cloudflare Worker) with lightweight self-built implementations to reduce bundle size and SSR CPU usage.
Changes
lib/markdown-renderer.tsx)react-markdown,remark-gfm,@flowershow/remark-wiki-linkcomponents/ui/toast.tsx) replacingsonnerimport.meta.env.SSRguard, removing the server pdf chunk (744K)src/server.ts)/,/p/,/support,/privacy-policy,/docs)cachesto avoid 1101 errorsreact-markdown,remark-gfm,@flowershow/remark-wiki-link,sonner(-101 packages afterpnpm install)Impact
Verification
npm run typecheck/lint(0 warnings) /npm run test(89 tests) /build/wrangler deploy --dry-runall passNotes
nextremoval) are deferred as separate tasks