diff --git a/CHANGELOG.md b/CHANGELOG.md index 1cfb95d..04c7ad6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,12 +12,16 @@ All notable changes to DevMap are documented in this file. ### Added +- OpenRouter provider support with interactive provider selection, API-key + validation, streaming responses, and user-selected free or paid models - `ts-morph` analysis for JavaScript and TypeScript behind a normalized analyzer registry with heuristic and fallback analyzers - Lightweight `.devmap/index.json` and per-feature navigation maps for agents ### Changed +- `devmap init` defaults OpenRouter to `openrouter/free` on Enter and explains + how to change the stored model with `devmap config model ` - Feature detection now separates documentation, landing UI, CLI commands, analysis, snapshot, and AI roles before assigning technical features - Generated agent guidance now uses index-first navigation and treats the full diff --git a/PRD.md b/PRD.md index 50f7a5f..78be629 100644 --- a/PRD.md +++ b/PRD.md @@ -265,6 +265,7 @@ Setup wizard. Runs once per machine/project. - Ask for AI provider - Ask for API key +- Ask for an OpenRouter model, defaulting to `openrouter/free` on Enter - Validate API key - Save global config to `~/.devmap/config.json` - Create `.devmap/` project folder if needed @@ -459,7 +460,8 @@ JSON mode rules: - stdout contains exactly one valid JSON document - no ANSI colors, Markdown rendering, box drawing, or progress text - runtime errors use a stable `{ "status": "error", "error": "...", "hint": "..." }` shape -- `init --json` is non-interactive and requires `GROQ_API_KEY` or existing config +- `init --json` is non-interactive and requires `GROQ_API_KEY`, + `OPENROUTER_API_KEY`, or existing config - AI responses are buffered instead of streamed - human-readable output remains the default - package-manager wrappers may still write their own warnings to stderr @@ -579,9 +581,9 @@ For current project structure, use `.devmap/snapshot.json` if available. ## 11. AI Strategy -### MVP Provider +### MVP Providers -**Groq only.** +**Groq and OpenRouter.** Reasons: @@ -589,6 +591,7 @@ Reasons: - Accessible globally - User provides their own API key - No DevMap backend required +- OpenRouter users can choose any model their account can access ### Provider Abstraction @@ -631,6 +634,13 @@ Model availability changes over time. Before changing the default routing, verify the current Groq model list and lifecycle status. Preview models must not be used as a primary default for a public DevMap release. +OpenRouter setup asks for a model ID after validating the API key. Pressing +Enter selects `openrouter/free`; entering another free or paid model stores +that exact model as the user's preferred model. Explicit user selections take +priority and are not replaced by DevMap's Groq routing chain. Users can change +the selection later with `devmap config model `; setting `auto` on +OpenRouter restores the safe `openrouter/free` default. + ### User API Key Principle - Users provide their own API keys diff --git a/README.md b/README.md index 3a3856f..39a4157 100644 --- a/README.md +++ b/README.md @@ -257,6 +257,11 @@ AI features require a provider API key. DevMap uses Groq by default — analysis | OpenAI | Planned | | Gemini | Planned | +`devmap init` lets you choose Groq or OpenRouter with the arrow keys. For +OpenRouter, pressing Enter at `OpenRouter model [openrouter/free]:` keeps the +free router; typing another model ID uses that free or paid model instead. +Change it later with `devmap config model `. + API keys are stored locally: ```txt diff --git a/docs/architecture.md b/docs/architecture.md index 12ed998..e4e288d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -608,9 +608,10 @@ All AI interactions go through a provider abstraction. Commands should not call provider APIs directly. -### MVP Provider - -* Groq +### MVP Providers + +* Groq +* OpenRouter ### Future Providers @@ -649,6 +650,12 @@ on 2026-06-20. Recheck provider lifecycle status before publishing a release. Users can override automatic routing with `devmap config model `. Running `devmap config model auto` restores the defaults above. + +OpenRouter does not use the Groq command-routing table. During `devmap init`, +Enter accepts `openrouter/free`, while a typed model ID is stored and used +exactly as the user selected it. Explicit OpenRouter model choices receive no +hidden DevMap fallback. The OpenRouter adapter supports the provider's native +ordered `models` request when a caller explicitly supplies fallbacks. Raw provider errors should not be shown directly to users. @@ -656,8 +663,8 @@ Raw provider errors should not be shown directly to users. ## Streaming AI Output -Groq chat completions use server-sent events for human-readable `analyze` and -`ask` output. The provider adapter reconstructs the complete response while +Groq and OpenRouter chat completions use server-sent events for human-readable +`analyze` and `ask` output. Each provider adapter reconstructs the response while emitting incremental deltas to the output layer. Terminal Markdown is buffered to paragraph boundaries before rendering. This diff --git a/docs/commands.md b/docs/commands.md index 41617be..303b799 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -42,10 +42,11 @@ devmap init ### Responsibilities -* Confirm AI provider -* Input API key or read environment variable -* Validate API key -* Save global configuration to `~/.devmap/config.json` +* Confirm AI provider +* Input API key or read environment variable +* Validate API key +* Ask for an OpenRouter model; Enter defaults to `openrouter/free` +* Save global configuration to `~/.devmap/config.json` * Detect current project framework * Create `.devmap/` * Add `.devmap/` to `.gitignore` @@ -582,6 +583,16 @@ devmap config model auto * `analyze` uses `openai/gpt-oss-20b` * `analyze --deep` uses `openai/gpt-oss-120b` +For OpenRouter, `devmap init` prompts with: + +```txt +OpenRouter model [openrouter/free]: +``` + +Press Enter for the free router, or type any free or paid OpenRouter model ID. +The typed model is stored as the primary choice and is not silently replaced. +`devmap config model auto` restores `openrouter/free` for OpenRouter. + Automatic routing also uses ordered fallback chains: * `ask`: `qwen/qwen3.6-27b`, `llama-3.3-70b-versatile`, then `openai/gpt-oss-20b` @@ -646,8 +657,8 @@ Contract: * progress sections and Markdown rendering are omitted * AI responses are buffered instead of streamed * runtime failures return a JSON object with `status`, `error`, and optional `hint` -* `init --json` never prompts and therefore requires `GROQ_API_KEY` or an - existing API key +* `init --json` never prompts and therefore requires `GROQ_API_KEY`, + `OPENROUTER_API_KEY`, or an existing API key * package-manager wrapper warnings may appear on stderr and are not part of the DevMap JSON document diff --git a/docs/for-me-personal/PROGRESS.md b/docs/for-me-personal/PROGRESS.md index fc16ae0..c59a0d2 100644 --- a/docs/for-me-personal/PROGRESS.md +++ b/docs/for-me-personal/PROGRESS.md @@ -4,6 +4,21 @@ Terakhir diperbarui: 2026-06-20 ## Update 2026-06-20 +### OpenRouter MVP Provider + +- `devmap init` sekarang menampilkan selector panah untuk Groq dan OpenRouter. +- Setup OpenRouter memvalidasi API key lalu meminta model dengan default + `openrouter/free` ketika user langsung menekan Enter. +- Model OpenRouter yang diketik user, baik gratis maupun berbayar, disimpan dan + selalu diprioritaskan tanpa hidden fallback dari DevMap. +- `devmap config model ` dapat mengganti pilihan; `auto` pada + OpenRouter kembali ke `openrouter/free`. +- `ask`, `analyze`, dan `doctor` sekarang memakai provider factory berdasarkan + config, bukan membuat Groq client secara langsung. +- OpenRouter completion, streaming SSE, usage normalization, validasi key, dan + native ordered `models` request sudah memiliki regression tests. +- Focused tests dan full CLI unit suite lulus dengan 116 test. + ### Standalone React Detection - Framework detector sekarang mengenali standalone React dari dependency diff --git a/docs/for-me-personal/TEST.md b/docs/for-me-personal/TEST.md index 3de12bb..26b5aab 100644 --- a/docs/for-me-personal/TEST.md +++ b/docs/for-me-personal/TEST.md @@ -15,6 +15,43 @@ Ada beberapa versi DevMap yang dapat diuji: | npm link | CLI global sementara | Menguji command `devmap` dari folder mana pun | | CI/runtime | OS dan versi Node berbeda | Verifikasi lintas platform sebelum release | +## OpenRouter Provider + +Focused automated tests: + +```powershell +pnpm --filter devmap exec tsx --test test/init-and-errors.test.ts test/openrouter-client.test.ts test/doctor.test.ts +``` + +Manual source test sebaiknya dijalankan dari project fixture atau project luar, +karena `init` menulis `.devmap/`, `DEVMAP.md`, dan kemungkinan `AGENTS.md`: + +```powershell +pnpm dev:cli -- init +``` + +Expected interactive flow: + +1. Pilih `OpenRouter` dengan tombol panah lalu tekan Enter. +2. Masukkan OpenRouter API key; key tidak boleh dicetak ulang. +3. Pada `OpenRouter model [openrouter/free]:`, tekan Enter untuk free router + atau ketik model ID gratis/berbayar yang ingin diuji. +4. Pastikan output menjelaskan command + `devmap config model ` untuk mengganti model nanti. +5. Jalankan `devmap doctor`, `devmap analyze`, dan `devmap ask` lalu pastikan + provider serta model yang tampil sesuai config. + +Non-interactive setup dapat memakai: + +```powershell +$env:OPENROUTER_API_KEY="your-key" +pnpm dev:cli -- init --json +Remove-Item Env:OPENROUTER_API_KEY +``` + +Jangan simpan atau menyalin API key nyata ke repository, snapshot, output test, +atau dokumentasi debugging. + ## Ts-Morph Dan Agent Navigation Focused tests: diff --git a/docs/roadmap.md b/docs/roadmap.md index ebb1c65..088ea6a 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -33,7 +33,7 @@ adding AI on top. If the foundation is wrong, AI output will be wrong too. **Goal:** Users can understand projects faster with AI interpretation. **Tasks:** -- Groq integration with provider abstraction layer +- Groq and OpenRouter integration through the provider abstraction layer - Prompt templates for analyze and ask - Context Builder — keyword search + file ranking - [x] Streaming output for human `analyze` and `ask` responses @@ -87,12 +87,12 @@ accuracy must be near 100% or developers won't trust it. --- ## Phase 5 — Multi-Provider -**Goal:** Users can choose their preferred AI provider. +**Goal:** Expand beyond the Groq and OpenRouter MVP providers. **Tasks:** - OpenAI adapter (GPT-4o mini as default) - Gemini adapter (1.5 Flash as default) -- Provider selection in `devmap init` +- [x] Provider selection in `devmap init` for Groq and OpenRouter - Per-project provider override in `.devmap/config.json` - Provider-specific model recommendations diff --git a/packages/cli/README.md b/packages/cli/README.md index 41d253d..ddb0854 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -2,7 +2,7 @@ Understand any codebase in minutes, not days. -DevMap is a CLI that combines static analysis with optional Groq-powered +DevMap is a CLI that combines static analysis with optional AI-powered interpretation. It maps project structure, generates reusable context, and answers focused questions without sending an entire repository to an AI model. @@ -23,7 +23,7 @@ npx devmap --help ## Requirements - Node.js 18 or newer -- A Groq API key for AI-powered analysis and answers +- A Groq or OpenRouter API key for AI-powered analysis and answers Static analysis still works when AI is not configured. @@ -39,18 +39,30 @@ devmap onboarding devmap doctor ``` -`devmap init` validates the Groq key, stores configuration locally, prepares +`devmap init` selects a provider, validates its key, stores configuration locally, prepares `.devmap/`, generates `DEVMAP.md`, and integrates with `AGENTS.md` safely. -## Groq Setup +## AI Provider Setup -Create a key at https://console.groq.com/keys, then either enter it during: +Choose Groq or OpenRouter with the arrow keys during: ```bash devmap init ``` -Or provide it to the current shell before non-interactive setup: +Groq keys are available at https://console.groq.com/keys. OpenRouter keys are +available at https://openrouter.ai/keys. + +For OpenRouter, DevMap prompts: + +```txt +OpenRouter model [openrouter/free]: +``` + +Press Enter to use the free router, or type any free or paid OpenRouter model +ID. The selected model is saved and used as the primary choice. + +For non-interactive Groq setup: ```bash GROQ_API_KEY="your-key" devmap init @@ -64,8 +76,15 @@ devmap init Remove-Item Env:GROQ_API_KEY ``` +For non-interactive OpenRouter setup: + +```bash +OPENROUTER_API_KEY="your-key" devmap init +``` + The key is stored locally in `~/.devmap/config.json`. Requests go directly from -your machine to Groq. DevMap does not send the key to a DevMap-owned server. +your machine to the selected provider. DevMap does not send the key to a +DevMap-owned server. ## Commands @@ -81,11 +100,12 @@ devmap doctor devmap config model auto ``` -Automatic model routing uses a fast model for focused questions and larger -models for architecture analysis. Override it with: +Groq automatic routing uses a fast model for focused questions and larger +models for architecture analysis. OpenRouter uses the model selected during +init. Change either provider's model with: ```bash -devmap config model +devmap config model devmap config model auto ``` @@ -131,7 +151,7 @@ support promise. ## Privacy - Project analysis runs locally before AI interpretation. -- Full repository source is not sent to Groq. +- Full repository source is not sent to the selected provider. - `ask` selects a small set of relevant files. - `.env` files and common generated directories are ignored. - API keys are stored locally and should never be committed. diff --git a/packages/cli/package.json b/packages/cli/package.json index 4ad934d..0df273d 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -30,7 +30,8 @@ "developer-tools", "static-analysis", "architecture", - "groq" + "groq", + "openrouter" ], "author": "Muhammad Fadil", "license": "MIT", diff --git a/packages/cli/src/ai/openrouter.ts b/packages/cli/src/ai/openrouter.ts new file mode 100644 index 0000000..f8ce020 --- /dev/null +++ b/packages/cli/src/ai/openrouter.ts @@ -0,0 +1,278 @@ +import { DevmapError } from "../utils/errors.js"; +import type { + AiClient, + AiCompletionRequest, + AiCompletionResult, + AiDeltaHandler, + AiTokenUsage +} from "./types.js"; + +const OPENROUTER_CHAT_URL = "https://openrouter.ai/api/v1/chat/completions"; +const OPENROUTER_KEY_URL = "https://openrouter.ai/api/v1/auth/key"; +export const OPENROUTER_FREE_MODEL = "openrouter/free"; + +export type OpenRouterClientDependencies = { + fetch?: typeof fetch; +}; + +export type OpenRouterProviderInspection = { + reachable: true; + modelAvailable: boolean; +}; + +export class OpenRouterClient implements AiClient { + private readonly fetchImplementation: typeof fetch; + + constructor( + private readonly apiKey: string, + dependencies: OpenRouterClientDependencies = {} + ) { + this.fetchImplementation = dependencies.fetch ?? fetch; + } + + async complete(request: AiCompletionRequest): Promise { + const response = await this.sendRequest(request, false); + if (!response.ok) throw await mapOpenRouterResponseError(response); + + const payload = await readCompletionPayload(response); + const content = payload.choices[0]?.message?.content?.trim(); + if (!content) { + throw new DevmapError( + "OpenRouter returned an empty response.", + "Try again or choose another model." + ); + } + + return { + content, + model: payload.model || request.model, + ...(payload.usage ? { usage: normalizeUsage(payload.usage) } : {}) + }; + } + + async stream( + request: AiCompletionRequest, + onDelta: AiDeltaHandler + ): Promise { + const response = await this.sendRequest(request, true); + if (!response.ok) throw await mapOpenRouterResponseError(response); + return readCompletionStream(response, request.model, onDelta); + } + + private async sendRequest( + request: AiCompletionRequest, + stream: boolean + ): Promise { + const models = resolveModels(request); + try { + return await this.fetchImplementation(OPENROUTER_CHAT_URL, { + method: "POST", + headers: { + Authorization: `Bearer ${this.apiKey}`, + "Content-Type": "application/json", + "HTTP-Referer": "https://github.com/itsflaid/devmap", + "X-OpenRouter-Title": "DevMap" + }, + body: JSON.stringify({ + ...(models.length > 1 ? { models } : { model: models[0] }), + messages: request.messages, + max_tokens: request.maxCompletionTokens ?? 1200, + temperature: request.temperature ?? 0.2, + ...(stream ? { stream: true } : {}) + }) + }); + } catch { + throw new DevmapError( + "Could not connect to OpenRouter.", + "Check your internet connection and run devmap doctor." + ); + } + } +} + +export async function validateOpenRouterApiKey( + apiKey: string, + dependencies: OpenRouterClientDependencies = {} +): Promise { + await inspectOpenRouterProvider(apiKey, undefined, dependencies); +} + +export async function inspectOpenRouterProvider( + apiKey: string, + _model?: string, + dependencies: OpenRouterClientDependencies = {} +): Promise { + const fetchImplementation = dependencies.fetch ?? fetch; + let response: Response; + try { + response = await fetchImplementation(OPENROUTER_KEY_URL, { + headers: { Authorization: `Bearer ${apiKey}` } + }); + } catch { + throw new DevmapError( + "Could not connect to OpenRouter.", + "Check your internet connection and run devmap init again." + ); + } + + if (response.status === 401 || response.status === 403) { + throw new DevmapError( + "The OpenRouter API key is invalid.", + "Create or copy a valid key from https://openrouter.ai/keys." + ); + } + if (!response.ok) { + throw new DevmapError( + `OpenRouter validation failed with HTTP ${response.status}.`, + "Try again shortly or check https://openrouter.ai/status." + ); + } + + return { reachable: true, modelAvailable: true }; +} + +type CompletionPayload = { + model?: string; + choices: Array<{ message?: { content?: string | null } }>; + usage?: RawUsage; +}; + +type RawUsage = { + prompt_tokens?: number; + completion_tokens?: number; + total_tokens?: number; +}; + +function resolveModels(request: AiCompletionRequest): string[] { + return Array.from(new Set([ + request.model, + ...(request.fallbackModels ?? []), + ...(request.fallbackModel ? [request.fallbackModel] : []) + ].filter((model) => model.trim().length > 0))); +} + +async function readCompletionPayload(response: Response): Promise { + try { + const payload = await response.json() as Partial; + return { + model: payload.model, + choices: Array.isArray(payload.choices) ? payload.choices : [], + usage: payload.usage + }; + } catch { + throw unreadableResponseError(); + } +} + +async function readCompletionStream( + response: Response, + requestedModel: string, + onDelta: AiDeltaHandler +): Promise { + if (!response.body) throw unreadableResponseError(); + const reader = response.body.getReader(); + const decoder = new TextDecoder(); + let buffer = ""; + let content = ""; + let model = requestedModel; + let usage: AiTokenUsage | undefined; + + const consume = (event: string): boolean => { + const data = event + .split(/\r?\n/) + .filter((line) => line.startsWith("data:")) + .map((line) => line.slice(5).trimStart()) + .join("\n"); + if (!data) return false; + if (data.trim() === "[DONE]") return true; + + try { + const payload = JSON.parse(data) as { + model?: string; + choices?: Array<{ delta?: { content?: string | null } }>; + usage?: RawUsage; + }; + model = payload.model || model; + const delta = payload.choices?.[0]?.delta?.content; + if (delta) { + content += delta; + onDelta(delta); + } + if (payload.usage) usage = normalizeUsage(payload.usage); + return false; + } catch { + throw unreadableResponseError(); + } + }; + + try { + let done = false; + while (!done) { + const chunk = await reader.read(); + buffer += decoder.decode(chunk.value, { stream: !chunk.done }); + const events = buffer.split(/\r?\n\r?\n/); + buffer = events.pop() ?? ""; + for (const event of events) { + if (consume(event)) { + done = true; + break; + } + } + if (chunk.done) { + if (!done && buffer.trim()) consume(buffer); + break; + } + } + } catch (error) { + if (error instanceof DevmapError) throw error; + throw unreadableResponseError(); + } + + if (!content.trim()) { + throw new DevmapError( + "OpenRouter returned an empty response.", + "Try again or choose another model." + ); + } + return { content: content.trim(), model, ...(usage ? { usage } : {}) }; +} + +async function mapOpenRouterResponseError(response: Response): Promise { + if (response.status === 401 || response.status === 403) { + return new DevmapError( + "The OpenRouter API key is invalid or no longer authorized.", + "Run devmap init again with a valid OpenRouter API key." + ); + } + if (response.status === 429) { + return new DevmapError( + "OpenRouter rate limit reached.", + "Wait briefly or choose another model." + ); + } + if (response.status >= 500) { + return new DevmapError( + "OpenRouter is temporarily unavailable.", + "Try again shortly or check https://openrouter.ai/status." + ); + } + return new DevmapError( + `OpenRouter could not complete the request (HTTP ${response.status}).`, + "Run devmap doctor and verify the selected model." + ); +} + +function unreadableResponseError(): DevmapError { + return new DevmapError( + "OpenRouter returned an unreadable response.", + "Try again shortly or choose another model." + ); +} + +function normalizeUsage(usage: RawUsage): AiTokenUsage { + return { + promptTokens: usage.prompt_tokens ?? 0, + completionTokens: usage.completion_tokens ?? 0, + totalTokens: usage.total_tokens ?? 0 + }; +} diff --git a/packages/cli/src/ai/provider.ts b/packages/cli/src/ai/provider.ts new file mode 100644 index 0000000..050cca5 --- /dev/null +++ b/packages/cli/src/ai/provider.ts @@ -0,0 +1,54 @@ +import type { DevmapConfig } from "../utils/config.js"; +import type { AiClient } from "./types.js"; +import { + DEFAULT_AI_FALLBACKS, + DEFAULT_AI_MODELS, + GroqClient, + inspectGroqProvider +} from "./groq.js"; +import { + inspectOpenRouterProvider, + OpenRouterClient, + OPENROUTER_FREE_MODEL +} from "./openrouter.js"; + +export type AiTask = "ask" | "analyze" | "deepAnalyze"; +export type ProviderInspection = { reachable: true; modelAvailable: boolean }; + +export function createAiClient(config: DevmapConfig): AiClient { + return config.provider === "openrouter" + ? new OpenRouterClient(config.apiKey ?? "") + : new GroqClient(config.apiKey ?? ""); +} + +export function resolveAiRouting( + config: DevmapConfig, + task: AiTask +): { model: string; fallbackModels: readonly string[] } { + if (config.model !== "auto") { + return { model: config.model, fallbackModels: [] }; + } + + if (config.provider === "openrouter") { + return { model: OPENROUTER_FREE_MODEL, fallbackModels: [] }; + } + + return { + model: DEFAULT_AI_MODELS[task], + fallbackModels: DEFAULT_AI_FALLBACKS[task] + }; +} + +export function providerDisplayName(provider: DevmapConfig["provider"]): string { + return provider === "openrouter" ? "OpenRouter" : "Groq"; +} + +export function inspectAiProvider( + provider: DevmapConfig["provider"], + apiKey: string, + model: string +): Promise { + return provider === "openrouter" + ? inspectOpenRouterProvider(apiKey, model) + : inspectGroqProvider(apiKey, model); +} diff --git a/packages/cli/src/analyzers/featureDetector.ts b/packages/cli/src/analyzers/featureDetector.ts index ee902e0..1b529dc 100644 --- a/packages/cli/src/analyzers/featureDetector.ts +++ b/packages/cli/src/analyzers/featureDetector.ts @@ -45,7 +45,7 @@ const FEATURE_SIGNALS: Array<{ }, { name: "AI Integration", - terms: ["openai", "groq", "gemini", "generative-ai"], + terms: ["openai", "groq", "openrouter", "gemini", "generative-ai"], purpose: "Handles AI providers, prompts, and model-facing context." }, { @@ -95,13 +95,15 @@ const ROLE_FEATURES: Array<{ role: "ai-integration", name: "AI Integration", purpose: "Handles AI providers, prompts, and model-facing context.", - terms: ["ai", "groq", "prompt", "context", "model"] + terms: ["ai", "groq", "openrouter", "prompt", "context", "model"] } ]; const FEATURE_FILE_PRIORITIES: Record = { "AI Integration": [ + /\/ai\/provider\.[cm]?[jt]s$/, /\/ai\/groq\.[cm]?[jt]s$/, + /\/ai\/openrouter\.[cm]?[jt]s$/, /\/ai\/contextbuilder\.[cm]?[jt]s$/, /\/ai\/prompts\.[cm]?[jt]s$/, /\/ai\/completion\.[cm]?[jt]s$/ diff --git a/packages/cli/src/analyzers/projectMap.ts b/packages/cli/src/analyzers/projectMap.ts index f884b68..6b836f8 100644 --- a/packages/cli/src/analyzers/projectMap.ts +++ b/packages/cli/src/analyzers/projectMap.ts @@ -612,7 +612,8 @@ function buildStructuralFeatureFlow(featureName: string, files: string[]): strin ? [ ["Build focused project context", find(/\/ai\/contextbuilder\.[cm]?[jt]s$/)], ["Construct grounded model prompts", find(/\/ai\/prompts\.[cm]?[jt]s$/)], - ["Call Groq with retry and model fallback", find(/\/ai\/groq\.[cm]?[jt]s$/)], + ["Select the configured AI provider", find(/\/ai\/provider\.[cm]?[jt]s$/)], + ["Call the provider with its model policy", find(/\/ai\/(groq|openrouter)\.[cm]?[jt]s$/)], ["Stream or return the completed response", find(/\/ai\/completion\.[cm]?[jt]s$/)] ] : []; diff --git a/packages/cli/src/analyzers/serviceDetector.ts b/packages/cli/src/analyzers/serviceDetector.ts index 3b70299..76aeb8e 100644 --- a/packages/cli/src/analyzers/serviceDetector.ts +++ b/packages/cli/src/analyzers/serviceDetector.ts @@ -11,12 +11,14 @@ const SERVICES: Array<[string[], string]> = [ [["cloudinary"], "Cloudinary"], [["firebase"], "Firebase"], [["openai"], "OpenAI"], - [["groq"], "Groq"] + [["groq"], "Groq"], + [["openrouter"], "OpenRouter"] ]; const SOURCE_SERVICE_SIGNALS: Array<[string[], string]> = [ [["api.groq.com", "console.groq.com", "groq api key", "groqclient"], "Groq"], - [["api.openai.com", "openai api key", "openaiclient"], "OpenAI"] + [["api.openai.com", "openai api key", "openaiclient"], "OpenAI"], + [["openrouter.ai", "openrouter api key", "openrouterclient"], "OpenRouter"] ]; export function detectExternalServices(files: ScannedFile[]): string[] { diff --git a/packages/cli/src/commands/analyze.ts b/packages/cli/src/commands/analyze.ts index d26b635..abdaceb 100644 --- a/packages/cli/src/commands/analyze.ts +++ b/packages/cli/src/commands/analyze.ts @@ -1,10 +1,9 @@ import { resolve } from "node:path"; import { completeWithOptionalStreaming } from "../ai/completion.js"; import { - DEFAULT_AI_FALLBACKS, - DEFAULT_AI_MODELS, - GroqClient -} from "../ai/groq.js"; + createAiClient as createDefaultAiClient, + resolveAiRouting +} from "../ai/provider.js"; import { buildAnalyzeMessages } from "../ai/prompts.js"; import { enrichSnapshotWithAi } from "../ai/snapshotEnrichment.js"; import type { AiClient } from "../ai/types.js"; @@ -93,21 +92,18 @@ async function enrichSnapshot( } const createAiClient = dependencies.createAiClient - ?? ((currentConfig: DevmapConfig) => new GroqClient(currentConfig.apiKey ?? "")); + ?? createDefaultAiClient; const client = createAiClient(config); - const defaultModel = options.deep - ? DEFAULT_AI_MODELS.deepAnalyze - : DEFAULT_AI_MODELS.analyze; - const model = config.model === "auto" ? defaultModel : config.model; - const fallbackModels = options.deep - ? DEFAULT_AI_FALLBACKS.deepAnalyze - : DEFAULT_AI_FALLBACKS.analyze; + const routing = resolveAiRouting( + config, + options.deep ? "deepAnalyze" : "analyze" + ); const enriched = await enrichSnapshotWithAi( snapshot, client, - model, - fallbackModels + routing.model, + routing.fallbackModels ); if (enriched !== snapshot) { @@ -188,15 +184,13 @@ async function printOrGenerateInterpretation( } const createAiClient = dependencies.createAiClient - ?? ((currentConfig: DevmapConfig) => new GroqClient(currentConfig.apiKey ?? "")); + ?? createDefaultAiClient; const client = createAiClient(config); - const defaultModel = options.deep - ? DEFAULT_AI_MODELS.deepAnalyze - : DEFAULT_AI_MODELS.analyze; - const model = config.model === "auto" ? defaultModel : config.model; - const fallbackModels = options.deep - ? DEFAULT_AI_FALLBACKS.deepAnalyze - : DEFAULT_AI_FALLBACKS.analyze; + const routing = resolveAiRouting( + config, + options.deep ? "deepAnalyze" : "analyze" + ); + const model = routing.model; output.step(`Interpreting architecture with ${model}`); @@ -204,7 +198,7 @@ async function printOrGenerateInterpretation( const execution = await completeWithOptionalStreaming(client, { messages: buildAnalyzeMessages(snapshot, options.deep), model, - fallbackModels, + fallbackModels: routing.fallbackModels, maxCompletionTokens: options.deep ? 1800 : 1000, temperature: 0.2 }, !options.json, () => output.section("Architecture")); diff --git a/packages/cli/src/commands/ask.ts b/packages/cli/src/commands/ask.ts index 8953daf..3925646 100644 --- a/packages/cli/src/commands/ask.ts +++ b/packages/cli/src/commands/ask.ts @@ -1,10 +1,10 @@ import { buildQuestionContext } from "../ai/contextBuilder.js"; import { completeWithOptionalStreaming } from "../ai/completion.js"; import { - DEFAULT_AI_FALLBACKS, - DEFAULT_AI_MODELS, - GroqClient -} from "../ai/groq.js"; + createAiClient as createDefaultAiClient, + providerDisplayName, + resolveAiRouting +} from "../ai/provider.js"; import { buildAskMessages, buildQueryExpansionMessages } from "../ai/prompts.js"; import type { AiClient } from "../ai/types.js"; import { inspectSnapshot, isSnapshotStale } from "../cache/snapshot.js"; @@ -73,10 +73,12 @@ async function runAsk( const loadConfig = dependencies.loadConfig ?? readConfig; const config = await loadConfig(); const createAiClient = dependencies.createAiClient - ?? ((currentConfig: DevmapConfig) => new GroqClient(currentConfig.apiKey ?? "")); - const model = config?.model === "auto" || !config?.model - ? DEFAULT_AI_MODELS.ask - : config.model; + ?? createDefaultAiClient; + const routing = resolveAiRouting(config ?? { + provider: "groq", + model: "auto" + }, "ask"); + const model = routing.model; const client = config?.apiKey ? createAiClient(config) : null; let context = await buildQuestionContext( projectRoot, @@ -84,7 +86,12 @@ async function runAsk( question ); if (client && context.topScore < MEDIUM_RELEVANCE_SCORE) { - const expandedTerms = await expandQuestionTerms(client, question, model); + const expandedTerms = await expandQuestionTerms( + client, + question, + model, + routing.fallbackModels + ); if (expandedTerms.length > 0) { context = await buildQuestionContext( projectRoot, @@ -123,7 +130,7 @@ async function runAsk( if (!config?.apiKey || !client) { output.warning("AI answering is not configured yet."); - output.note("Run devmap init to configure a Groq API key."); + output.note("Run devmap init to configure an AI provider API key."); printStaticContext(context.files); return { status: "static", @@ -140,13 +147,13 @@ async function runAsk( }; } - output.step(`Asking Groq with ${model}`); + output.step(`Asking ${providerDisplayName(config.provider)} with ${model}`); try { const execution = await completeWithOptionalStreaming(client, { messages: buildAskMessages(context, snapshot.project), model, - fallbackModels: DEFAULT_AI_FALLBACKS.ask, + fallbackModels: routing.fallbackModels, maxCompletionTokens: 1200, temperature: 0.2 }, !dependencies.json, () => output.section("Answer")); @@ -202,13 +209,14 @@ async function runAsk( async function expandQuestionTerms( client: AiClient, question: string, - model: string + model: string, + fallbackModels: readonly string[] ): Promise { try { const result = await client.complete({ messages: buildQueryExpansionMessages(question), model, - fallbackModels: DEFAULT_AI_FALLBACKS.ask, + fallbackModels, maxCompletionTokens: 180, temperature: 0 }); diff --git a/packages/cli/src/commands/config.ts b/packages/cli/src/commands/config.ts index 4eee3b6..c156514 100644 --- a/packages/cli/src/commands/config.ts +++ b/packages/cli/src/commands/config.ts @@ -56,7 +56,9 @@ async function updateModel( output.success( selectedModel === "auto" - ? "Restored automatic command-based model routing." + ? config.provider === "openrouter" + ? "Restored OpenRouter free model routing (openrouter/free)." + : "Restored automatic command-based model routing." : `Default model override set to ${selectedModel}.` ); @@ -64,6 +66,9 @@ async function updateModel( status: "ok", provider: config.provider, model: selectedModel, - automaticRouting: selectedModel === "auto" + automaticRouting: selectedModel === "auto", + ...(selectedModel === "auto" && config.provider === "openrouter" + ? { resolvedModel: "openrouter/free" } + : {}) }; } diff --git a/packages/cli/src/commands/doctor.ts b/packages/cli/src/commands/doctor.ts index f592063..2786639 100644 --- a/packages/cli/src/commands/doctor.ts +++ b/packages/cli/src/commands/doctor.ts @@ -1,10 +1,10 @@ import { arch, platform } from "node:os"; import { resolve } from "node:path"; import { - DEFAULT_AI_MODELS, - inspectGroqProvider, - type GroqProviderInspection -} from "../ai/groq.js"; + inspectAiProvider, + resolveAiRouting, + type ProviderInspection +} from "../ai/provider.js"; import { scanFiles } from "../analyzers/fileScanner.js"; import { detectFramework } from "../analyzers/frameworkDetector.js"; import { detectProjectMetadata } from "../analyzers/projectMetadata.js"; @@ -22,8 +22,9 @@ export type DoctorDependencies = { loadConfig?: () => Promise; inspectProvider?: ( apiKey: string, - model: string - ) => Promise; + model: string, + provider: DevmapConfig["provider"] + ) => Promise; }; export async function doctorCommand( @@ -44,7 +45,10 @@ async function runDoctor( ): Promise> { const projectRoot = resolve(dependencies.projectRoot ?? process.cwd()); const loadConfig = dependencies.loadConfig ?? readConfig; - const inspectProvider = dependencies.inspectProvider ?? inspectGroqProvider; + const inspectProvider = dependencies.inspectProvider + ?? ((apiKey: string, model: string, provider: DevmapConfig["provider"]) => ( + inspectAiProvider(provider, apiKey, model) + )); const [config, snapshotResult, files] = await Promise.all([ loadConfig(), inspectSnapshot(projectRoot), @@ -52,9 +56,9 @@ async function runDoctor( ]); const framework = detectFramework(files); const project = detectProjectMetadata(projectRoot, framework, files); - const selectedModel = config?.model === "auto" - ? DEFAULT_AI_MODELS.ask - : config?.model; + const selectedModel = config + ? resolveAiRouting(config, "ask").model + : undefined; const issues: string[] = []; const nodeSupported = readNodeMajor(process.version) >= MINIMUM_NODE_MAJOR; let apiKeyStatus = "not configured"; @@ -76,13 +80,17 @@ async function runDoctor( output.keyValue("API key", "not configured"); output.keyValue("Model", "not configured"); } else if (!config.apiKey || !selectedModel) { - issues.push("Run devmap init again to configure Groq."); + issues.push("Run devmap init again to configure the selected provider."); output.keyValue("API key", "missing"); output.keyValue("Model", selectedModel ?? "not configured"); apiKeyStatus = "missing"; } else { try { - const provider = await inspectProvider(config.apiKey, selectedModel); + const provider = await inspectProvider( + config.apiKey, + selectedModel, + config.provider + ); apiKeyStatus = provider.reachable ? "valid" : "unreachable"; modelStatus = provider.modelAvailable ? selectedModel @@ -91,7 +99,7 @@ async function runDoctor( output.keyValue("Model", modelStatus); if (!provider.modelAvailable) { - issues.push("Run devmap init or choose an available Groq model."); + issues.push("Run devmap init or choose an available provider model."); } } catch (error) { const message = error instanceof DevmapError diff --git a/packages/cli/src/commands/init.ts b/packages/cli/src/commands/init.ts index f375cb9..43bdb8e 100644 --- a/packages/cli/src/commands/init.ts +++ b/packages/cli/src/commands/init.ts @@ -3,6 +3,11 @@ import { resolve } from "node:path"; import { scanFiles } from "../analyzers/fileScanner.js"; import { detectFramework } from "../analyzers/frameworkDetector.js"; import { validateGroqApiKey } from "../ai/groq.js"; +import { + OPENROUTER_FREE_MODEL, + validateOpenRouterApiKey +} from "../ai/openrouter.js"; +import { providerDisplayName } from "../ai/provider.js"; import { readConfig, writeConfig, type DevmapConfig } from "../utils/config.js"; import { ensureAgentsFile, @@ -20,9 +25,13 @@ export type InitDependencies = { json?: boolean; projectRoot?: string; prompt?: Prompt; - validateApiKey?: (apiKey: string) => Promise; + validateApiKey?: ( + apiKey: string, + provider: DevmapConfig["provider"] + ) => Promise; isInteractive?: boolean; environmentApiKey?: string; + environmentOpenRouterApiKey?: string; loadConfig?: () => Promise; persistConfig?: (config: DevmapConfig) => Promise; }; @@ -48,8 +57,6 @@ async function runInit( const loadConfig = dependencies.loadConfig ?? readConfig; const persistConfig = dependencies.persistConfig ?? writeConfig; const existingConfig = await loadConfig(); - const environmentApiKey = dependencies.environmentApiKey ?? process.env.GROQ_API_KEY; - const validateApiKey = dependencies.validateApiKey ?? validateGroqApiKey; const prompt = dependencies.prompt ?? (interactive ? createPrompt() : null); if (!dependencies.json) { @@ -58,19 +65,42 @@ async function runInit( } output.section("DevMap Init"); - output.keyValue("Provider", "Groq"); try { + const provider = await resolveProvider(prompt, interactive, existingConfig); + const providerName = providerDisplayName(provider); + const environmentApiKey = dependencies.environmentApiKey + ?? (provider === "openrouter" + ? dependencies.environmentOpenRouterApiKey ?? process.env.OPENROUTER_API_KEY + : process.env.GROQ_API_KEY); + const validateApiKey = dependencies.validateApiKey + ?? ((key: string, selectedProvider: DevmapConfig["provider"]) => ( + selectedProvider === "openrouter" + ? validateOpenRouterApiKey(key) + : validateGroqApiKey(key) + )); + output.keyValue("Provider", providerName); + const apiKey = await resolveApiKey({ prompt, interactive, environmentApiKey, - existingApiKey: existingConfig?.apiKey + existingApiKey: existingConfig?.provider === provider + ? existingConfig.apiKey + : undefined, + provider }); - output.step("Validating Groq API key"); - await validateApiKey(apiKey); - output.success("Groq API key is valid"); + output.step(`Validating ${providerName} API key`); + await validateApiKey(apiKey, provider); + output.success(`${providerName} API key is valid`); + + const model = await resolveInitialModel({ + provider, + prompt, + interactive, + existingConfig + }); const agentsStatus = await inspectAgentsFile(projectRoot); const appendToExistingAgents = agentsStatus === "existing" @@ -86,9 +116,9 @@ async function runInit( await mkdir(resolve(projectRoot, ".devmap"), { recursive: true }); await persistConfig({ - provider: "groq", + provider, apiKey, - model: "auto" + model }); const ignored = await ensureDevmapIgnored(projectRoot); @@ -97,14 +127,18 @@ async function runInit( output.keyValue("Project", framework); output.success("Config saved to ~/.devmap/config.json"); + if (provider === "openrouter") { + output.note(`OpenRouter model: ${model}`); + output.note("Change it later with: devmap config model "); + } output.success(ignored ? "Added .devmap/ to .gitignore" : ".devmap/ already ignored"); output.success(devmapFileCreated ? "Created DEVMAP.md" : "DEVMAP.md already exists"); printAgentsResult(agentsResult); output.step("Next: devmap analyze"); return { status: "ok", - provider: "groq", - model: "auto", + provider, + model, framework, files: { gitignoreUpdated: ignored, @@ -146,6 +180,7 @@ type ResolveApiKeyOptions = { interactive: boolean; environmentApiKey?: string; existingApiKey?: string; + provider: DevmapConfig["provider"]; }; async function resolveApiKey(options: ResolveApiKeyOptions): Promise { @@ -159,23 +194,74 @@ async function resolveApiKey(options: ResolveApiKeyOptions): Promise { } throw new DevmapError( - "A Groq API key is required to initialize DevMap.", - "Run devmap init in an interactive terminal or set GROQ_API_KEY." + `An ${providerDisplayName(options.provider)} API key is required to initialize DevMap.`, + `Run devmap init in an interactive terminal or set ${readProviderEnvName(options.provider)}.` ); } + const providerName = providerDisplayName(options.provider); const keyPrompt = options.existingApiKey - ? "Groq API key [press Enter to keep existing]: " - : "Groq API key: "; + ? `${providerName} API key [press Enter to keep existing]: ` + : `${providerName} API key: `; const answer = (await options.prompt.ask(keyPrompt)).trim(); const apiKey = answer || options.existingApiKey?.trim(); if (!apiKey) { throw new DevmapError( - "A Groq API key is required.", - "Create one at https://console.groq.com/keys." + `An ${providerName} API key is required.`, + options.provider === "openrouter" + ? "Create one at https://openrouter.ai/keys." + : "Create one at https://console.groq.com/keys." ); } return apiKey; } + +async function resolveProvider( + prompt: Prompt | null, + interactive: boolean, + existingConfig: DevmapConfig | null +): Promise { + if (!interactive || !prompt) { + if (existingConfig) return existingConfig.provider; + return process.env.OPENROUTER_API_KEY && !process.env.GROQ_API_KEY + ? "openrouter" + : "groq"; + } + + return prompt.select("AI provider", [ + { label: "Groq", value: "groq" }, + { label: "OpenRouter", value: "openrouter" } + ], existingConfig?.provider ?? "groq"); +} + +type ResolveInitialModelOptions = { + provider: DevmapConfig["provider"]; + prompt: Prompt | null; + interactive: boolean; + existingConfig: DevmapConfig | null; +}; + +async function resolveInitialModel( + options: ResolveInitialModelOptions +): Promise { + if (options.provider === "groq") return "auto"; + + const existingModel = options.existingConfig?.provider === "openrouter" + ? options.existingConfig.model + : undefined; + const defaultModel = existingModel && existingModel !== "auto" + ? existingModel + : OPENROUTER_FREE_MODEL; + if (!options.interactive || !options.prompt) return defaultModel; + + const answer = await options.prompt.ask( + `OpenRouter model [${defaultModel}]: ` + ); + return answer.trim() || defaultModel; +} + +function readProviderEnvName(provider: DevmapConfig["provider"]): string { + return provider === "openrouter" ? "OPENROUTER_API_KEY" : "GROQ_API_KEY"; +} diff --git a/packages/cli/src/utils/config.ts b/packages/cli/src/utils/config.ts index 9b5f07b..5077334 100644 --- a/packages/cli/src/utils/config.ts +++ b/packages/cli/src/utils/config.ts @@ -3,7 +3,7 @@ import { homedir } from "node:os"; import { dirname, join } from "node:path"; export type DevmapConfig = { - provider: "groq"; + provider: "groq" | "openrouter"; apiKey?: string; model: "auto" | string; }; @@ -34,7 +34,7 @@ function isDevmapConfig(value: unknown): value is DevmapConfig { && value !== null && !Array.isArray(value) && "provider" in value - && value.provider === "groq" + && (value.provider === "groq" || value.provider === "openrouter") && "model" in value && typeof value.model === "string" ); diff --git a/packages/cli/src/utils/prompt.ts b/packages/cli/src/utils/prompt.ts index 22d4f9a..4287cbd 100644 --- a/packages/cli/src/utils/prompt.ts +++ b/packages/cli/src/utils/prompt.ts @@ -1,23 +1,122 @@ +import { + clearLine, + createInterface as createCallbackInterface, + cursorTo, + emitKeypressEvents, + moveCursor +} from "node:readline"; import { createInterface } from "node:readline/promises"; import { stdin, stdout } from "node:process"; +export type PromptOption = { + label: string; + value: T; +}; + export type Prompt = { ask(question: string): Promise; + select( + question: string, + options: Array>, + initialValue?: T + ): Promise; close(): void; }; export function createPrompt(): Prompt { - const readline = createInterface({ - input: stdin, - output: stdout - }); + let closed = false; return { - ask(question: string): Promise { - return readline.question(question); + async ask(question: string): Promise { + if (closed) return ""; + const readline = createInterface({ input: stdin, output: stdout }); + try { + return await readline.question(question); + } finally { + readline.close(); + } + }, + select( + question: string, + options: Array>, + initialValue?: T + ): Promise { + if (options.length === 0) { + throw new Error("Prompt options cannot be empty."); + } + + const initialIndex = Math.max( + 0, + options.findIndex((option) => option.value === initialValue) + ); + return selectWithArrowKeys(question, options, initialIndex); }, close(): void { - readline.close(); + closed = true; } }; } + +function selectWithArrowKeys( + question: string, + options: Array>, + initialIndex: number +): Promise { + if (!stdin.isTTY || typeof stdin.setRawMode !== "function") { + return Promise.resolve(options[initialIndex]!.value); + } + + return new Promise((resolve) => { + let selectedIndex = initialIndex; + const render = (replace = false): void => { + if (replace) { + moveCursor(stdout, 0, -options.length); + } + for (const [index, option] of options.entries()) { + cursorTo(stdout, 0); + clearLine(stdout, 0); + stdout.write(`${index === selectedIndex ? ">" : " "} ${option.label}\n`); + } + }; + + const readline = createCallbackInterface({ input: stdin, output: stdout }); + emitKeypressEvents(stdin, readline); + stdout.write(`${question}\n`); + render(); + stdin.setRawMode(true); + stdin.resume(); + + const finish = (): void => { + stdin.off("keypress", onKeypress); + stdin.setRawMode(false); + stdin.pause(); + readline.close(); + resolve(options[selectedIndex]!.value); + }; + + const onKeypress = ( + _value: string, + key: { name?: string; ctrl?: boolean } + ): void => { + if (key.name === "up") { + selectedIndex = (selectedIndex - 1 + options.length) % options.length; + render(true); + return; + } + if (key.name === "down") { + selectedIndex = (selectedIndex + 1) % options.length; + render(true); + return; + } + if (key.name === "return") { + finish(); + return; + } + if (key.ctrl && key.name === "c") { + finish(); + } + }; + + stdin.on("keypress", onKeypress); + }); +} diff --git a/packages/cli/test/analyzers.test.ts b/packages/cli/test/analyzers.test.ts index 174bf52..59cfc0c 100644 --- a/packages/cli/test/analyzers.test.ts +++ b/packages/cli/test/analyzers.test.ts @@ -182,6 +182,13 @@ test("service detector detects HTTP API providers without package dependencies", "export class GroqClient {}" ].join("\n") ); + await writeFile( + join(projectRoot, "src", "ai", "openrouter.ts"), + [ + "const OPENROUTER_URL = 'https://openrouter.ai/api/v1/chat/completions';", + "export class OpenRouterClient {}" + ].join("\n") + ); await mkdir(join(projectRoot, "src", "analyzers"), { recursive: true }); await writeFile( join(projectRoot, "src", "analyzers", "serviceDetector.ts"), @@ -190,7 +197,10 @@ test("service detector detects HTTP API providers without package dependencies", await mkdir(join(projectRoot, "docs"), { recursive: true }); await writeFile(join(projectRoot, "docs", "notes.md"), "Groq mentioned in docs only.\n"); - assert.deepEqual(detectExternalServices(await scanFiles(projectRoot)), ["Groq"]); + assert.deepEqual( + detectExternalServices(await scanFiles(projectRoot)), + ["Groq", "OpenRouter"] + ); } finally { await rm(projectRoot, { recursive: true, force: true }); } @@ -209,6 +219,10 @@ test("feature detection keeps documentation and landing UI out of technical feat "packages/cli/src/ai/groq.ts", 'import type { AiClient } from "./types.js"; export class GroqClient {}' ), + createScannedFile( + "packages/cli/src/ai/openrouter.ts", + 'import type { AiClient } from "./types.js"; export class OpenRouterClient {}' + ), createScannedFile( "packages/cli/src/ai/contextBuilder.ts", 'const aliases = ["auth", "login", "session", "jwt"]; export function buildContext() {}' @@ -250,6 +264,7 @@ test("feature detection keeps documentation and landing UI out of technical feat const aiFeature = features.find((feature) => feature.name === "AI Integration"); assert.ok(aiFeature?.files.includes("packages/cli/src/ai/groq.ts")); + assert.ok(aiFeature?.files.includes("packages/cli/src/ai/openrouter.ts")); assert.ok(aiFeature?.files.every((path) => path.startsWith("packages/cli/src/ai/"))); }); diff --git a/packages/cli/test/config-command.test.ts b/packages/cli/test/config-command.test.ts index 0dd83f6..4aab9b9 100644 --- a/packages/cli/test/config-command.test.ts +++ b/packages/cli/test/config-command.test.ts @@ -59,6 +59,31 @@ test("config model requires an existing initialized config", async () => { assert.match(logs, /Run devmap init/i); }); +test("config model auto explains OpenRouter free routing", async () => { + let saved: DevmapConfig | null = null; + const logs: string[] = []; + const originalLog = console.log; + console.log = (...values: unknown[]) => logs.push(values.join(" ")); + + try { + await configModelCommand("auto", { + loadConfig: async () => ({ + provider: "openrouter", + apiKey: "sk-or-fixture", + model: "qwen/qwen3-coder" + }), + persistConfig: async (config) => { + saved = config; + } + }); + + assert.equal(saved?.model, "auto"); + assert.match(logs.join("\n"), /openrouter\/free/i); + } finally { + console.log = originalLog; + } +}); + async function captureOutput(action: () => Promise): Promise { const logs: string[] = []; const originalLog = console.log; diff --git a/packages/cli/test/doctor.test.ts b/packages/cli/test/doctor.test.ts index 51e9083..cb681a2 100644 --- a/packages/cli/test/doctor.test.ts +++ b/packages/cli/test/doctor.test.ts @@ -104,6 +104,36 @@ test("doctor skips network diagnostics when config is missing", async () => { } }); +test("doctor inspects OpenRouter with the configured user model", async () => { + const projectRoot = await mkdtemp(join(tmpdir(), "devmap-doctor-openrouter-test-")); + let inspectedProvider = ""; + let inspectedModel = ""; + + try { + const logs = await captureOutput(() => doctorCommand({ + projectRoot, + loadConfig: async () => ({ + provider: "openrouter", + apiKey: "sk-or-fixture", + model: "qwen/qwen3-coder" + }), + inspectProvider: async (_apiKey, model, provider) => { + inspectedProvider = provider; + inspectedModel = model; + return { reachable: true, modelAvailable: true }; + } + })); + + assert.equal(inspectedProvider, "openrouter"); + assert.equal(inspectedModel, "qwen/qwen3-coder"); + assert.match(logs, /Provider\s+openrouter/); + assert.match(logs, /Model\s+qwen\/qwen3-coder/); + assert.doesNotMatch(logs, /sk-or-fixture/); + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } +}); + async function captureOutput(action: () => Promise): Promise { const logs: string[] = []; const originalLog = console.log; diff --git a/packages/cli/test/init-and-errors.test.ts b/packages/cli/test/init-and-errors.test.ts index 22b36d3..7808c05 100644 --- a/packages/cli/test/init-and-errors.test.ts +++ b/packages/cli/test/init-and-errors.test.ts @@ -214,6 +214,17 @@ test("readConfig returns null for invalid config schemas", async () => { model: "auto", apiKey: "gsk_fixture" }); + + await writeFile(configPath, JSON.stringify({ + provider: "openrouter", + model: "anthropic/claude-3.5-haiku", + apiKey: "sk-or-fixture" + }), "utf8"); + assert.deepEqual(await readConfig(), { + provider: "openrouter", + model: "anthropic/claude-3.5-haiku", + apiKey: "sk-or-fixture" + }); } finally { restoreEnvironment("HOME", originalHome); restoreEnvironment("USERPROFILE", originalUserProfile); @@ -221,9 +232,9 @@ test("readConfig returns null for invalid config schemas", async () => { } }); -test("interactive init uses Groq without prompting for a provider", async () => { +test("interactive init selects Groq with the provider menu", async () => { const projectRoot = await mkdtemp(join(tmpdir(), "devmap-init-provider-test-")); - const prompt = createFakePrompt(["gsk_fixture"]); + const prompt = createFakePrompt(["gsk_fixture"], ["groq"]); let savedConfig: DevmapConfig | null = null; try { @@ -244,7 +255,70 @@ test("interactive init uses Groq without prompting for a provider", async () => apiKey: "gsk_fixture", model: "auto" }); - assert.ok(prompt.questions.every((question) => !/^Provider\b/.test(question))); + assert.deepEqual(prompt.selections, ["AI provider"]); + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } +}); + +test("interactive init stores the OpenRouter model selected by the user", async () => { + const projectRoot = await mkdtemp(join(tmpdir(), "devmap-init-openrouter-test-")); + const prompt = createFakePrompt( + ["sk-or-fixture", "anthropic/claude-3.5-haiku"], + ["openrouter"] + ); + let savedConfig: DevmapConfig | null = null; + let validatedKey = ""; + + try { + await initCommand({ + projectRoot, + prompt, + isInteractive: true, + loadConfig: async () => null, + persistConfig: async (config) => { + savedConfig = config; + }, + validateApiKey: async (apiKey) => { + validatedKey = apiKey; + } + }); + + assert.equal(validatedKey, "sk-or-fixture"); + assert.deepEqual(savedConfig, { + provider: "openrouter", + apiKey: "sk-or-fixture", + model: "anthropic/claude-3.5-haiku" + }); + assert.deepEqual(prompt.selections, ["AI provider"]); + assert.ok(prompt.questions.some((question) => /OpenRouter model/i.test(question))); + } finally { + await rm(projectRoot, { recursive: true, force: true }); + } +}); + +test("interactive OpenRouter init defaults to the free router on Enter", async () => { + const projectRoot = await mkdtemp(join(tmpdir(), "devmap-init-openrouter-free-test-")); + const prompt = createFakePrompt(["sk-or-fixture", ""], ["openrouter"]); + let savedConfig: DevmapConfig | null = null; + + try { + await initCommand({ + projectRoot, + prompt, + isInteractive: true, + loadConfig: async () => null, + persistConfig: async (config) => { + savedConfig = config; + }, + validateApiKey: async () => undefined + }); + + assert.deepEqual(savedConfig, { + provider: "openrouter", + apiKey: "sk-or-fixture", + model: "openrouter/free" + }); } finally { await rm(projectRoot, { recursive: true, force: true }); } @@ -341,20 +415,32 @@ test("global error handler emits parseable JSON for machine output", () => { type FakePrompt = Prompt & { closed: boolean; questions: string[]; + selections: string[]; }; -function createFakePrompt(answers: string[]): FakePrompt { +function createFakePrompt(answers: string[], selectedValues: string[] = []): FakePrompt { let index = 0; + let selectionIndex = 0; return { closed: false, questions: [], + selections: [], async ask(question: string): Promise { this.questions.push(question); const answer = answers[index] ?? ""; index += 1; return answer; }, + async select( + question: string, + options: Array<{ label: string; value: T }> + ): Promise { + this.selections.push(question); + const selected = selectedValues[selectionIndex] as T | undefined; + selectionIndex += 1; + return selected ?? options[0]!.value; + }, close(): void { this.closed = true; } diff --git a/packages/cli/test/onboarding-command.test.ts b/packages/cli/test/onboarding-command.test.ts index c09a9ab..a46b8cb 100644 --- a/packages/cli/test/onboarding-command.test.ts +++ b/packages/cli/test/onboarding-command.test.ts @@ -128,6 +128,12 @@ function createFakePrompt(answers: string[]): Prompt & { closed: boolean; questi this.questions.push(question); return answers.shift() ?? ""; }, + async select( + _question: string, + options: Array<{ label: string; value: T }> + ): Promise { + return options[0]!.value; + }, close(): void { this.closed = true; } diff --git a/packages/cli/test/openrouter-client.test.ts b/packages/cli/test/openrouter-client.test.ts new file mode 100644 index 0000000..3888060 --- /dev/null +++ b/packages/cli/test/openrouter-client.test.ts @@ -0,0 +1,164 @@ +import assert from "node:assert/strict"; +import test from "node:test"; +import { + OpenRouterClient, + OPENROUTER_FREE_MODEL, + validateOpenRouterApiKey +} from "../src/ai/openrouter.js"; +import { createAiClient, resolveAiRouting } from "../src/ai/provider.js"; +import { DevmapError } from "../src/utils/errors.js"; + +test("OpenRouter client sends the user-selected model without replacing it", async () => { + const requests: Array<{ url: string; init?: RequestInit }> = []; + const client = new OpenRouterClient("sk-or-fixture", { + fetch: async (url, init) => { + requests.push({ url: String(url), init }); + return jsonResponse({ + model: "anthropic/claude-3.5-haiku", + choices: [{ message: { content: "Selected model answer." } }] + }); + } + }); + + const result = await client.complete({ + messages: [{ role: "user", content: "Explain the project." }], + model: "anthropic/claude-3.5-haiku" + }); + + const body = JSON.parse(String(requests[0]?.init?.body)) as { + model?: string; + models?: string[]; + }; + assert.equal(requests[0]?.url, "https://openrouter.ai/api/v1/chat/completions"); + assert.equal(body.model, "anthropic/claude-3.5-haiku"); + assert.equal(body.models, undefined); + assert.equal(result.model, "anthropic/claude-3.5-haiku"); +}); + +test("OpenRouter client sends ordered native fallbacks when explicitly provided", async () => { + let requestBody: { model?: string; models?: string[] } = {}; + const client = new OpenRouterClient("sk-or-fixture", { + fetch: async (_url, init) => { + requestBody = JSON.parse(String(init?.body)) as typeof requestBody; + return jsonResponse({ + model: OPENROUTER_FREE_MODEL, + choices: [{ message: { content: "Fallback answer." } }] + }); + } + }); + + await client.complete({ + messages: [{ role: "user", content: "Explain the project." }], + model: "custom/model", + fallbackModels: [OPENROUTER_FREE_MODEL] + }); + + assert.equal(requestBody.model, undefined); + assert.deepEqual(requestBody.models, ["custom/model", OPENROUTER_FREE_MODEL]); +}); + +test("OpenRouter client streams deltas and reports the routed model", async () => { + const deltas: string[] = []; + const encoder = new TextEncoder(); + const client = new OpenRouterClient("sk-or-fixture", { + fetch: async () => new Response(new ReadableStream({ + start(controller) { + controller.enqueue(encoder.encode( + 'data: {"model":"google/gemma-3-4b-it:free","choices":[{"delta":{"content":"Free "}}]}\n\n' + )); + controller.enqueue(encoder.encode( + 'data: {"choices":[{"delta":{"content":"answer."}}],"usage":{"prompt_tokens":8,"completion_tokens":2,"total_tokens":10}}\n\n' + )); + controller.enqueue(encoder.encode("data: [DONE]\n\n")); + controller.close(); + } + })) + }); + + const result = await client.stream({ + messages: [{ role: "user", content: "Explain the project." }], + model: OPENROUTER_FREE_MODEL + }, (delta) => deltas.push(delta)); + + assert.deepEqual(deltas, ["Free ", "answer."]); + assert.equal(result.content, "Free answer."); + assert.equal(result.model, "google/gemma-3-4b-it:free"); + assert.deepEqual(result.usage, { + promptTokens: 8, + completionTokens: 2, + totalTokens: 10 + }); +}); + +test("OpenRouter automatic routing defaults safely to the free router", () => { + assert.deepEqual(resolveAiRouting({ + provider: "openrouter", + apiKey: "sk-or-fixture", + model: "auto" + }, "ask"), { + model: OPENROUTER_FREE_MODEL, + fallbackModels: [] + }); +}); + +test("OpenRouter routing prioritizes an explicit user model", () => { + assert.deepEqual(resolveAiRouting({ + provider: "openrouter", + apiKey: "sk-or-fixture", + model: "qwen/qwen3-coder" + }, "deepAnalyze"), { + model: "qwen/qwen3-coder", + fallbackModels: [] + }); +}); + +test("provider factory creates an OpenRouter client from stored config", () => { + assert.ok(createAiClient({ + provider: "openrouter", + apiKey: "sk-or-fixture", + model: OPENROUTER_FREE_MODEL + }) instanceof OpenRouterClient); +}); + +test("OpenRouter validation maps invalid credentials without exposing the key", async () => { + await assert.rejects( + validateOpenRouterApiKey("secret-value", { + fetch: async () => jsonResponse({ error: { message: "invalid" } }, 401) + }), + (error: unknown) => error instanceof DevmapError + && /OpenRouter API key is invalid/i.test(error.message) + && !error.message.includes("secret-value") + ); +}); + +test("OpenRouter client does not expose raw provider errors", async () => { + const client = new OpenRouterClient("sk-or-fixture", { + fetch: async () => jsonResponse({ + error: { message: "internal provider detail that should remain hidden" } + }, 400) + }); + + await assert.rejects( + client.complete({ + messages: [{ role: "user", content: "Explain the project." }], + model: "custom/model" + }), + (error: unknown) => error instanceof DevmapError + && /HTTP 400/.test(error.message) + && !error.message.includes("internal provider detail") + ); +}); + +function jsonResponse( + body: unknown, + status = 200, + headers: Record = {} +): Response { + return new Response(JSON.stringify(body), { + status, + headers: { + "content-type": "application/json", + ...headers + } + }); +}