Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions examples/openui-whisper/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# Required: your OpenAI API key (used for both Whisper transcription and the chat model)
OPENAI_API_KEY=sk-your-key-here

# Optional overrides
# CHAT_MODEL=gpt-5.2
# TRANSCRIBE_MODEL=whisper-1
8 changes: 8 additions & 0 deletions examples/openui-whisper/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@

# local env
.env
graphify-out
node_modules/
.next/
next-env.d.ts
.claude/
47 changes: 47 additions & 0 deletions examples/openui-whisper/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# openui-whispr

**Speak to the screen.** A glowing orb sits in the center and listens by default. When you
pause, your utterance is transcribed by OpenAI Whisper and sent to a chat model running the
**OpenUI** system prompt, which streams back [OpenUI Lang](https://www.openui.com) — rendered
full-screen as live UI. No chat bubbles, no text box: the entire screen *is* the interface.

## Flow

```
🎙 always-on mic → Web Audio VAD detects a pause (end of utterance)
→ MediaRecorder segment → /api/transcribe (Whisper) → text
→ processMessage → /api/chat (OpenUI system prompt)
→ streamed OpenUI Lang → <Renderer/> fills the screen
```

- `src/app/api/transcribe/route.ts` — audio (multipart) → OpenAI Whisper → `{ text }`.
- `src/app/api/chat/route.ts` — messages + OpenUI system prompt → streamed completion.
- `src/components/useVoiceListener.ts` — always-on listener. Meters mic level via an
`AnalyserNode` and segments speech into utterances using silence detection (voice activity
detection), transcribing each completed utterance.
- `src/components/Orb.tsx` — the glowing orb. Breathes while idle, scales with your voice
while it hears you, and pulses while the model is thinking.
- `src/components/VoiceCanvas.tsx` — full-screen surface. Renders the latest assistant
message via `<Renderer/>`, shows the orb, and provides **Stop listening** / **Reset**.
- `src/app/page.tsx` — wires `ChatProvider` (headless) to `/api/chat` and mounts the canvas.

## Setup

```bash
cp .env.example .env # then add your OPENAI_API_KEY
pnpm install
pnpm dev
```

Open http://localhost:3000, allow microphone access, and just start speaking — pause when
you're done and the UI renders. Use **Stop listening** to mute and **Reset** to clear.

> Microphone capture requires a secure context — `localhost` works; deploy over HTTPS.

## Configuration

| Env var | Default | Purpose |
| ------------------ | ------------ | ----------------------------- |
| `OPENAI_API_KEY` | _(required)_ | Auth for Whisper + chat model |
| `CHAT_MODEL` | `gpt-5.2` | Model that generates the UI |
| `TRANSCRIBE_MODEL` | `whisper-1` | Speech-to-text model |
18 changes: 18 additions & 0 deletions examples/openui-whisper/eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { defineConfig, globalIgnores } from "eslint/config";
import nextVitals from "eslint-config-next/core-web-vitals";
import nextTs from "eslint-config-next/typescript";

const eslintConfig = defineConfig([
...nextVitals,
...nextTs,
// Override default ignores of eslint-config-next.
globalIgnores([
// Default ignores of eslint-config-next:
".next/**",
"out/**",
"build/**",
"next-env.d.ts",
]),
]);

export default eslintConfig;
5 changes: 5 additions & 0 deletions examples/openui-whisper/next.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import type { NextConfig } from "next";

const nextConfig: NextConfig = {};

export default nextConfig;
30 changes: 30 additions & 0 deletions examples/openui-whisper/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"name": "openui-whispr",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "eslint"
},
"dependencies": {
"@openuidev/react-ui": "latest",
"@openuidev/react-headless": "latest",
"@openuidev/react-lang": "latest",
"next": "16.1.6",
"openai": "^6.22.0",
"react": "19.2.3",
"react-dom": "19.2.3"
},
"devDependencies": {
"@tailwindcss/postcss": "^4",
"@types/node": "^20",
"@types/react": "^19",
"@types/react-dom": "^19",
"eslint": "^9",
"eslint-config-next": "16.1.6",
"tailwindcss": "^4",
"typescript": "^5"
}
}
Loading
Loading