Skip to content

Commit 530fcc0

Browse files
committed
feat: Connector AI pipeline — live data integration for AI assistant
- New 'My Connectors' system with Hacker News connector (top stories, URLs, authors, comments) - Connector toggle in AI panel header with green active indicator - Unified parallel fetch pipeline (connector + web search via Promise.all) - Grounding instruction header forces AI to use fetched live data - Fixed: Gemma 4 E4B worker discarded context param — only userPrompt was used - Fixed: Context now prepended to user question in Gemma 4 messages array - Fixed: Gemma 4 system prompt enhanced with 'data is real' grounding instruction - Context trimmed to 6000 chars for WebGPU memory safety on E4B - Fixed: Label click toggling connector off via event bubbling (e.preventDefault) - Fixed: hasActiveConnectors() decoupled from DOM — reads localStorage directly - Auto-repair re-enables connected-but-paused connectors on init - Default HN stories reduced 10→5 for compact context - Connector modal with glassmorphic grid/detail views and per-connector config
1 parent 0171c21 commit 530fcc0

10 files changed

Lines changed: 1882 additions & 38 deletions

File tree

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@
4141
| **🔌 API Calls** | `{{API:}}` REST API integration — GET/POST/PUT/DELETE methods, custom headers, JSON body, response stored in `$(api_varName)` variables; inline review panel; toolbar GET/POST buttons |
4242
| **🔗 Agent Flow** | `{{Agent:}}` multi-step pipeline — define Step 1/2/3, chain outputs, per-card model + search provider selector, live step status indicators (⏳/✅/❌), review combined output; `@cloud: yes/no` with ☁️ Cloud / 🖥️ Local badge; `@agenttype:` dropdown selector (openclaw/openfang) for external agents; Docker-based local execution via `agent-runner/server.js`; Agent Execution Settings UI (Codespaces/Local Docker/Custom endpoint); GitHub Codespaces cloud execution via ☁️ toggle; **📦 Agent Containers panel** — floating toolbar panel showing running Docker containers with live status, uptime, instant stop (`docker rm -f`), badge count, daemon readiness check, startup container recovery, and floating toggle accessible when header is fully hidden |
4343
| **🔍 Web Search** | Toggle web search for AI — 7 providers: DuckDuckGo (free), Brave Search, Serper.dev, Tavily (AI-optimized), Google CSE, Wikipedia, Wikidata; search results injected into LLM context; source citations in responses; per-agent-card search provider selector |
44+
| **🔌 Connectors** | Third-party data sources for AI context — Hacker News connector (top stories with URLs, authors, body text, top 3 community comments, configurable story count 1–20); connector toggle in AI panel header with green active indicator; parallel fetch with web search via `Promise.all`; grounding instruction header forces AI to use live data; connector modal with grid view, detail view, connect/disconnect; state persisted in `localStorage`; extensible registry for future Slack, Notion, GitHub, Confluence connectors |
4445
| **🎮 Game Builder** | `{{@Game:}}` tag — AI-generated games (Canvas 2D / Three.js / P5.js) or instant pre-built games via `@prebuilt:` field (chess, snake, shooter, pong, breakout, maths quiz, hiragana, kana master); engine selector pills; per-card model picker; CDN URL normalizer for CSP compliance; auto model-ready check before generation; 📋 Import button for pasting/uploading external HTML game code with source viewer; 📥 Export as standalone HTML; ⛶ fullscreen; single-line field parsing; "Games for Kids" template with 8 playable games |
4546
| **🐧 Linux Terminal** | `{{Linux:}}` tag — two modes: (1) Terminal mode opens full Debian Linux ([WebVM](https://webvm.io)) in new window with `Packages:` field; (2) Compile & Run mode (`Language:` + `Script:`) compiles/executes 25+ languages (C++, Rust, Go, Java, Python, TypeScript, Kotlin, Scala…) via [Judge0 CE](https://ce.judge0.com) with inline output, execution time & memory stats |
4647
| **❓ Help Mode** | Interactive learning mode — click ❓ Help to highlight all buttons, click any button for description + keyboard shortcut + animated demo video; 50% screen demo panel with fullscreen expand; 16 dedicated demo videos mapped to every toolbar button |
@@ -546,6 +547,7 @@ TextAgent has undergone significant evolution since its inception. What started
546547

547548
| Date | Commits | Feature / Update |
548549
|------|---------|-----------------:|
550+
| **2026-04-03** | — | 🔌 **Connector AI Pipeline** — new "My Connectors" system for plugging third-party data sources into the AI assistant; Hacker News connector fetches top stories with full URLs, author metadata, self-post body text, and top community comments; connector toggle in AI panel header with green active indicator; unified parallel fetch pipeline (`Promise.all`) merges connector + web search context; grounding instruction header ("LIVE DATA...Answer using this data") forces models to use fetched data; **Fixed:** Gemma 4 E4B worker completely discarded `context` parameter — only `userPrompt` was used in the messages array; context now injected as `context + "\n---\nUser question: " + userText`; Gemma 4 system prompt enhanced with "data is real and live" grounding instruction; context trimmed to 6000 chars for WebGPU memory safety; connector label click bug fixed (`e.preventDefault()` stops checkbox toggle via event bubbling); `hasActiveConnectors()` decoupled from DOM — reads `localStorage` directly; auto-repair re-enables connected-but-paused connectors on init; default HN stories 10→5; connector registry extensible for Slack, Notion, GitHub, Confluence |
549551
| **2026-04-03** | — | 👁️ **Gemma 4 Vision Tag** — new `{{@Vision:}}` DocGen tag backed by Gemma 4 E2B/E4B running locally via WebGPU/WASM; `ai-worker-gemma4.js` Web Worker with `Gemma4Processor` instantiation (bypasses `AutoProcessor` which lacks `image_processor_type`) and system-prompt persona fix; primary `onnx-community/gemma-4-E2B-it-ONNX` with `textagent/gemma-4-E4B-it-ONNX` fallback; cyan-themed Vision card with 📷 camera capture + 📎 omni-modal upload (image/audio/video); **video frame extraction** — `extractVideoFrames()` seeks 4 evenly-spaced timestamps in a hidden `<video>` element, draws each to Canvas at max 1280px, stores as JPEG 0.85; audio stored as direct base64; upload handler detects Vision card type, sets `accept="image/*,audio/*,video/*"`; generation handler maps attachments to typed inputs and calls `switchToModel('gemma4-e2b')` before execution, restores prior model after; 👁️ Vision toolbar button in AI Tags dropdown; Fixed: Vision card double-rendering raw `@upload:` / `@prompt:` lines caused by broken `\\s*` regex (quadruple-escaped) — now correct `\s*`; removed duplicate static text row; `gemma4-e2b` / `gemma4-e4b` entries in `ai-models.js` with `isDocModel: true` + `supportsVision: true` |
550552
| **2026-04-02** | `55538f3` | 🔧 **DocGen Reject Block Fix** — fixed: rejecting a generated Translate/OCR/TTS/STT/Image/AI block restored a generic hardcoded "AI Generate" card, losing all type-specific UI (language dropdown, mode pills, camera button, step inputs, etc.); reject handler now calls `M.transformDocgenMarkdown(block.fullMatch)` to re-render the exact original typed card with all controls intact; `data-ai-index` patched on restored card and all children; review panel header label+icon now shows correct type for all blocks ("🌐 Translate — Review", "🔍 OCR Scan — Review", etc.) instead of always "✨ AI Generate — Review" |
551553
| **2026-04-02** | `f012a30`, `971de55` |**Share Link Loading Overlay** — eliminates the flash of bare UI when opening shared links (`#space=`, `#s=`, `#id=`, `#d=`); full-screen branded loading splash (TextAgent logo + spinner) activates before any JS loads via inline hash detection in `<head>`; theme-aware (dark `#0d1117` / light `#f6f8fa`); fades out smoothly (0.35s) once content is ready; `hideShareLoader()` called at every terminal path (success, error, form-gate); 15-second safety timeout auto-dismisses on network failure |
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Connector AI Pipeline — Live Data Integration
2+
3+
- New `🔌 My Connectors` system for plugging third-party data sources into the AI assistant
4+
- Hacker News connector fetches top stories with full URLs, author metadata, self-post body text, and top community comments
5+
- Connector toggle in AI panel header for quick enable/disable with green active indicator
6+
- Unified fetch pipeline: connector data and web search run in parallel via `Promise.all`
7+
- Grounding instruction header ("LIVE DATA...Answer using this data as your primary source") prepended to context
8+
- Fixed: Gemma 4 E4B worker completely discarded `context` parameter — only `userPrompt` was used in the messages array
9+
- Fixed: Context now injected as `context + "\n\n---\nUser question: " + userText` so model sees data before the question
10+
- Fixed: Gemma 4 system prompt enhanced with "data is real and live" instruction when context is provided
11+
- Context trimmed to 6000 chars in Gemma 4 worker to prevent WebGPU OOM on E4B quant
12+
- Updated Gemma 4 context window comment to match actual 128K spec (was incorrectly listed as 8192)
13+
- Fixed: Clicking the connector label in the AI header accidentally toggled the checkbox off via event bubbling (`e.preventDefault()` added)
14+
- Fixed: `hasActiveConnectors()` decoupled from DOM checkbox — now reads directly from `localStorage`
15+
- Auto-repair: connected-but-paused connectors automatically re-enabled on page load (fixes state corruption from label-click bug)
16+
- Default HN story count reduced from 10 to 5 for compact context suitable for local models
17+
- Connector modal with grid view, detail view, connect/disconnect flow, and per-connector config fields
18+
- Connectors CSS with glassmorphic modal design, dark mode support, and responsive layout
19+
- Connector state persisted in `localStorage` with `ta-connector-` prefix per connector
20+
- Connector registry (`REGISTRY`) supports extensible connector types (HN, Slack, Notion, GitHub, Confluence stubs)
21+
22+
---
23+
24+
## Summary
25+
26+
Adds a complete "My Connectors" data source system to the AI assistant and fixes a critical bug where the Gemma 4 E4B worker silently discarded all injected context, causing the model to respond with "I don't have real-time access" even when 20KB of live Hacker News data was successfully fetched and passed to the worker. The fix ensures context is prepended to the user's question in the Gemma 4 message array, the system prompt instructs the model to use the provided data, and context is trimmed to 6000 chars to stay within WebGPU memory limits.
27+
28+
---
29+
30+
## 1. Connectors System (`js/connectors.js`, `css/connectors.css`)
31+
**Files:** `js/connectors.js`, `css/connectors.css`, `index.html`, `js/modal-templates.js`, `src/main.js`, `js/ai-assistant.js`
32+
**What:** New modular connector framework with a registry of data sources, modal UI for connecting/configuring, and `getActiveContext()` API that aggregates data from all enabled connectors. Hacker News connector implemented with full story fetching (titles, URLs, authors, body text, top comments).
33+
**Impact:** The AI assistant can now answer questions from live third-party data instead of only training knowledge. Users connect data sources via a visual modal and the data is automatically injected into AI prompts.
34+
35+
## 2. AI Pipeline Unification (`js/ai-chat.js`)
36+
**Files:** `js/ai-chat.js`
37+
**What:** Refactored `sendChatMessage` to use `Promise.all([connectorPromise, searchPromise])` for parallel context fetching. `buildFinalContext()` merges connector and search data with a grounding instruction header. Context is always injected regardless of whether web search is enabled.
38+
**Impact:** Eliminates the prior bug where enabling web search caused connector data to be dropped. Both sources now run in parallel for minimum latency.
39+
40+
## 3. Gemma 4 Context Injection Fix (`public/ai-worker-gemma4.js`)
41+
**Files:** `public/ai-worker-gemma4.js`
42+
**What:** Fixed the root cause — the `context` parameter was completely ignored in the `generate()` function. Only `userPrompt` was used to build the messages array. Now, when context is provided, it's prepended to create `fullUserText = context + "\n\n---\nUser question: " + userText`. System prompt enhanced with grounding instruction. Context trimmed to 6000 chars for WebGPU safety. Updated context window comment from 8192 to actual 128K.
43+
**Impact:** Gemma 4 E4B now reads and uses injected context (connectors, web search, document QA) instead of claiming "I don't have real-time access."
44+
45+
## 4. State Management Bug Fixes (`js/connectors.js`)
46+
**Files:** `js/connectors.js`
47+
**What:** Fixed label click toggling checkbox via `e.preventDefault()`; decoupled `hasActiveConnectors()` from DOM state; added auto-repair logic in `refreshAiStrip()` to re-enable connectors that were corrupted to `enabled: false`.
48+
**Impact:** Connectors stay reliably enabled after being connected. No more "Paused" state appearing unexpectedly.
49+
50+
---
51+
52+
## Files Changed (8 total)
53+
54+
| File | Lines Changed | Type |
55+
|------|:---:|------|
56+
| `js/connectors.js` | +977 | New: Connector framework, registry, HN fetcher, modal UI |
57+
| `css/connectors.css` | +380 | New: Connector modal glassmorphic styles |
58+
| `js/ai-chat.js` | +47 −38 | Unified parallel fetch pipeline with grounding |
59+
| `public/ai-worker-gemma4.js` | +20 −7 | Context injection fix, trimming, system prompt |
60+
| `index.html` | +11 | Connector modal template, CSS/JS imports |
61+
| `js/modal-templates.js` | +43 | Connector modal HTML template |
62+
| `js/ai-assistant.js` | +26 | Connector toggle in AI panel header |
63+
| `src/main.js` | +9 | Module registration for connectors |

0 commit comments

Comments
 (0)