Instrument the OpenAI client, and stop shipping raw prompts in span events

.changeset/ts-gen-ai-openai.md

@@ -0,0 +1,11 @@
+---
+'@smooai/observability': minor
+---
+
+Instrument the OpenAI Node SDK, and stop leaking prompt content into GenAI span events.
+
+`wrapOpenAI(client, options)` returns a proxy of an OpenAI client whose `chat.completions.create` emits OTel GenAI semantic-convention spans — request model / sampling params / tool names on the way out, response model, id, finish reason, and token usage on the way back. Streaming is handled: the span stays open until the stream drains (or the consumer breaks out early), and picks up the usage chunk emitted under `stream_options.include_usage`. The client is duck-typed, so this adds no dependency on `openai` and works against Groq, Together, Fireworks, DeepSeek, Azure OpenAI, and any OpenAI-compatible gateway via `{ system: 'groq' }`.
+
+Cost has a seam now: nothing in the platform computes an LLM price on its own, which is why the dashboard's cost column is empty. Pass `costUsd({ requestModel, responseModel, inputTokens, outputTokens, cachedTokens })` and it lands on `gen_ai.usage.cost_usd`.
+
+`recordGenAIMessage` now routes content through the SDK's PII scrub before it leaves the process. Prompts are the most PII-dense payload this SDK can touch, and `wrapOpenAI` keeps content recording **off** by default (`{ recordContent: true }` opts in).

README.md

@@ -158,6 +158,46 @@ The same ingest contract (`POST /webhooks/observability/{org_id}/{token}` with `
 - 🐹 **Go** — `github.com/smooai/observability-go` (tracked in SMOODEV-1067 follow-ups)
 - 💠 **.NET** — `SmooAI.Observability` on NuGet (tracked in SMOODEV-1067 follow-ups)
 
+### 🤖 GenAI telemetry (`gen_ai.*`)
+
+LLM and agent spans carry the [OTel GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/), so any semconv-aware backend reads them — Smoo's LLM dashboard routes on `gen_ai.system` alone.
+
+```ts
+import OpenAI from 'openai';
+import { wrapOpenAI } from '@smooai/observability';
+
+// Instruments chat.completions.create — the original client is untouched.
+const openai = wrapOpenAI(new OpenAI(), {
+    conversationId: conversation.id,
+    // Providers don't return a price. Supply one and the cost column fills in.
+    costUsd: ({ inputTokens = 0, outputTokens = 0 }) => inputTokens * 2.5e-6 + outputTokens * 1e-5,
+});
+```
+
+The same wrapper covers Groq, Together, Fireworks, DeepSeek, Azure OpenAI, and any OpenAI-compatible gateway — pass `{ system: 'groq' }` so spans attribute to the real provider. Prompt and completion **content is off by default**; `{ recordContent: true }` records it as `gen_ai.*.message` span events, PII-scrubbed on the way out.
+
+For hand-rolled calls, set the attributes directly:
+
+```ts
+import { setGenAIAttributes, recordGenAIMessage } from '@smooai/observability';
+
+setGenAIAttributes(span, { system: 'anthropic', operationName: 'chat', requestModel: 'claude-opus-4-7', usageInputTokens: 812, usageOutputTokens: 96 });
+```
+
+> `gen_ai.operation.name` is a straight passthrough on ingest with **no fallback** — leave it unset and the operation column lands `NULL`. Always set it.
+
+**Parity across the five SDKs:**
+
+| SDK            | Attribute helper              | Message events                | Content PII-scrubbed | Framework integration                               |
+| -------------- | ----------------------------- | ----------------------------- | -------------------- | --------------------------------------------------- |
+| **TypeScript** | `setGenAIAttributes`          | `recordGenAIMessage`          | ✅                   | ✅ `wrapOpenAI` — OpenAI Node SDK + compatible APIs |
+| **Rust**       | `set_gen_ai_attributes`       | `record_gen_ai_message`       | ❌                   | —                                                   |
+| **Python**     | `set_gen_ai_attributes`       | `record_gen_ai_message`       | ❌                   | ✅ `SmooAICallbackHandler` — LangChain / LangGraph  |
+| **Go**         | `SetGenAIAttributes`          | `RecordGenAIMessage`          | ❌                   | —                                                   |
+| **.NET**       | `GenAIActivity.SetAttributes` | `GenAIActivity.RecordMessage` | ❌                   | —                                                   |
+
+Known divergences: TypeScript, Python, Go and .NET emit `gen_ai.tool.names` as a **string array**; Rust emits a comma-joined string. Only TypeScript scrubs recorded message content today.
+
 ## 📖 Architecture
 
 The SDK is intentionally thin. It captures, batches, redacts PII, and POSTs to a Smoo ingest endpoint. All of the heavy lifting — fingerprint grouping, source-map symbolication, dashboards, alerts, retention — lives in the Smoo platform.

packages/core/README.md

@@ -103,6 +103,30 @@ deterministic, stable for a page's lifetime, and reproduced byte-identically by
 the Rust / Python / Go / .NET SDKs against
 [`parity/sampling-corpus.json`](../../parity/README.md).
 
+## GenAI spans
+
+```ts
+import OpenAI from 'openai';
+import { wrapOpenAI, setGenAIAttributes } from '@smooai/observability';
+
+const openai = wrapOpenAI(new OpenAI(), { conversationId: convo.id });
+await openai.chat.completions.create({ model: 'gpt-4o', messages });
+```
+
+`wrapOpenAI` proxies `chat.completions.create` (streaming included — the span
+stays open until the stream drains) and emits the OTel
+[GenAI semconv](https://opentelemetry.io/docs/specs/semconv/gen-ai/) attributes.
+It needs no dependency on `openai`; the client is duck-typed, so the same
+wrapper covers Groq / DeepSeek / Azure / any OpenAI-compatible gateway via
+`{ system: 'groq' }`.
+
+- **Cost**: nothing computes a price on its own. Pass `costUsd(...)` to fill
+  `gen_ai.usage.cost_usd`.
+- **Content**: prompts and completions are **not** recorded unless you pass
+  `{ recordContent: true }`, and are PII-scrubbed when you do.
+- **Hand-rolled calls**: `setGenAIAttributes(span, attrs)` /
+  `recordGenAIMessage(span, role, content)`.
+
 ## What it does NOT do
 
 - Does not capture `console.log` / `console.info` / `console.warn`

packages/core/src/__tests__/gen-ai-attributes.test.ts

@@ -0,0 +1,200 @@
+import { trace } from '@opentelemetry/api';
+import { BasicTracerProvider, InMemorySpanExporter, SimpleSpanProcessor } from '@opentelemetry/sdk-trace-base';
+import { beforeEach, describe, expect, it } from 'vitest';
+import { recordGenAIMessage, setGenAIAttributes, type GenAIAttributes } from '../gen-ai-attributes';
+
+const exporter = new InMemorySpanExporter();
+trace.setGlobalTracerProvider(new BasicTracerProvider({ spanProcessors: [new SimpleSpanProcessor(exporter)] }));
+const tracer = trace.getTracer('test');
+
+/** Run `fn` against a fresh span and hand back the exported span. */
+function onSpan(fn: (span: ReturnType<typeof tracer.startSpan>) => void) {
+    const span = tracer.startSpan('llm');
+    fn(span);
+    span.end();
+    const finished = exporter.getFinishedSpans();
+    return finished[finished.length - 1]!;
+}
+
+/**
+ * Every attribute `setGenAIAttributes` can emit, and the key it MUST emit it
+ * under. A typo in any of these produces a span the platform cannot route, so
+ * the keys are asserted literally rather than derived from the source.
+ */
+const EXPECTED_KEYS: Record<keyof GenAIAttributes, string> = {
+    system: 'gen_ai.system',
+    operationName: 'gen_ai.operation.name',
+    requestModel: 'gen_ai.request.model',
+    responseModel: 'gen_ai.response.model',
+    responseId: 'gen_ai.response.id',
+    temperature: 'gen_ai.request.temperature',
+    topP: 'gen_ai.request.top_p',
+    topK: 'gen_ai.request.top_k',
+    maxTokens: 'gen_ai.request.max_tokens',
+    seed: 'gen_ai.request.seed',
+    usageInputTokens: 'gen_ai.usage.input_tokens',
+    usageOutputTokens: 'gen_ai.usage.output_tokens',
+    usageCachedTokens: 'gen_ai.usage.cached_tokens',
+    usageCostUsd: 'gen_ai.usage.cost_usd',
+    toolNames: 'gen_ai.tool.names',
+    truncated: 'gen_ai.response.truncated',
+    finishReason: 'gen_ai.response.finish_reason',
+    endUserId: 'gen_ai.end_user.id',
+    conversationId: 'gen_ai.conversation.id',
+};
+
+const FULL_ATTRS: Required<GenAIAttributes> = {
+    system: 'anthropic',
+    operationName: 'chat',
+    requestModel: 'claude-opus-4-7',
+    responseModel: 'claude-opus-4-7-20260101',
+    responseId: 'msg_123',
+    temperature: 0.7,
+    topP: 0.9,
+    topK: 40,
+    maxTokens: 1024,
+    seed: 42,
+    usageInputTokens: 100,
+    usageOutputTokens: 250,
+    usageCachedTokens: 80,
+    usageCostUsd: 0.0123,
+    toolNames: ['search', 'calendar'],
+    truncated: false,
+    finishReason: 'stop',
+    endUserId: 'user_abc',
+    conversationId: 'conv_xyz',
+};
+
+describe('setGenAIAttributes', () => {
+    beforeEach(() => exporter.reset());
+
+    it('emits every attribute under its exact semantic-convention key', () => {
+        const span = onSpan((s) => setGenAIAttributes(s, FULL_ATTRS));
+
+        expect(span.attributes['gen_ai.system']).toBe('anthropic');
+        expect(span.attributes['gen_ai.operation.name']).toBe('chat');
+        expect(span.attributes['gen_ai.request.model']).toBe('claude-opus-4-7');
+        expect(span.attributes['gen_ai.response.model']).toBe('claude-opus-4-7-20260101');
+        expect(span.attributes['gen_ai.response.id']).toBe('msg_123');
+        expect(span.attributes['gen_ai.request.temperature']).toBe(0.7);
+        expect(span.attributes['gen_ai.request.top_p']).toBe(0.9);
+        expect(span.attributes['gen_ai.request.top_k']).toBe(40);
+        expect(span.attributes['gen_ai.request.max_tokens']).toBe(1024);
+        expect(span.attributes['gen_ai.request.seed']).toBe(42);
+        expect(span.attributes['gen_ai.usage.input_tokens']).toBe(100);
+        expect(span.attributes['gen_ai.usage.output_tokens']).toBe(250);
+        expect(span.attributes['gen_ai.usage.cached_tokens']).toBe(80);
+        expect(span.attributes['gen_ai.usage.cost_usd']).toBe(0.0123);
+        expect(span.attributes['gen_ai.tool.names']).toEqual(['search', 'calendar']);
+        expect(span.attributes['gen_ai.response.truncated']).toBe(false);
+        expect(span.attributes['gen_ai.response.finish_reason']).toBe('stop');
+        expect(span.attributes['gen_ai.end_user.id']).toBe('user_abc');
+        expect(span.attributes['gen_ai.conversation.id']).toBe('conv_xyz');
+    });
+
+    it('emits nothing outside the gen_ai.* vocabulary, and covers every field', () => {
+        const span = onSpan((s) => setGenAIAttributes(s, FULL_ATTRS));
+        expect(new Set(Object.keys(span.attributes))).toEqual(new Set(Object.values(EXPECTED_KEYS)));
+    });
+
+    it('skips undefined fields so partial calls are additive', () => {
+        const span = onSpan((s) => {
+            setGenAIAttributes(s, { system: 'openai', operationName: 'chat', requestModel: 'gpt-4o' });
+            setGenAIAttributes(s, { usageInputTokens: 5, usageOutputTokens: 9 });
+        });
+        expect(Object.keys(span.attributes).sort()).toEqual([
+            'gen_ai.operation.name',
+            'gen_ai.request.model',
+            'gen_ai.system',
+            'gen_ai.usage.input_tokens',
+            'gen_ai.usage.output_tokens',
+        ]);
+    });
+
+    it('omits gen_ai.tool.names for an empty tool list', () => {
+        const span = onSpan((s) => setGenAIAttributes(s, { toolNames: [] }));
+        expect(span.attributes['gen_ai.tool.names']).toBeUndefined();
+    });
+
+    it('accepts an arbitrary system string for providers outside the known set', () => {
+        const span = onSpan((s) => setGenAIAttributes(s, { system: 'my-private-gateway' }));
+        expect(span.attributes['gen_ai.system']).toBe('my-private-gateway');
+    });
+});
+
+/**
+ * Routing contract with `rust/api-prime/src/handlers/observability/ingest_traces.rs`.
+ * That handler forwards a span to `gen_ai_events` iff it carries `gen_ai.system`,
+ * then reads exactly these columns off the merged attribute map. Every one is a
+ * straight passthrough — `gen_ai.operation.name` in particular has NO fallback,
+ * so a caller that leaves it unset writes a NULL operation column.
+ */
+const INGEST_READS = [
+    'gen_ai.system',
+    'gen_ai.operation.name',
+    'gen_ai.request.model',
+    'gen_ai.request.temperature',
+    'gen_ai.request.top_p',
+    'gen_ai.request.max_tokens',
+    'gen_ai.response.model',
+    'gen_ai.response.id',
+    'gen_ai.response.finish_reason',
+    'gen_ai.usage.input_tokens',
+    'gen_ai.usage.output_tokens',
+    'gen_ai.usage.cached_tokens',
+    'gen_ai.usage.cost_usd',
+    'gen_ai.end_user.id',
+    'gen_ai.conversation.id',
+] as const;
+
+describe('ingest routing contract', () => {
+    beforeEach(() => exporter.reset());
+
+    it('produces every column ingest_traces.rs reads', () => {
+        const span = onSpan((s) => setGenAIAttributes(s, FULL_ATTRS));
+        for (const key of INGEST_READS) {
+            expect(span.attributes[key], `ingest reads ${key} but the SDK never emitted it`).toBeDefined();
+        }
+    });
+
+    it('carries the gen_ai.system trigger that routes the span at all', () => {
+        // Without this exact key the span lands in ClickHouse traces only and
+        // never reaches gen_ai_events — the whole LLM dashboard goes dark.
+        const span = onSpan((s) => setGenAIAttributes(s, { system: 'openai' }));
+        expect(Object.keys(span.attributes)).toContain('gen_ai.system');
+    });
+});
+
+describe('recordGenAIMessage', () => {
+    beforeEach(() => exporter.reset());
+
+    it('names the event gen_ai.{role}.message and keys content exactly', () => {
+        const span = onSpan((s) => recordGenAIMessage(s, 'assistant', 'hello there'));
+        expect(span.events[0]!.name).toBe('gen_ai.assistant.message');
+        expect(span.events[0]!.attributes!['gen_ai.message.content']).toBe('hello there');
+    });
+
+    it.each(['user', 'assistant', 'system', 'tool'] as const)('supports the %s role', (role) => {
+        const span = onSpan((s) => recordGenAIMessage(s, role, 'x'));
+        expect(span.events[0]!.name).toBe(`gen_ai.${role}.message`);
+    });
+
+    it('attaches tool linkage under the spec keys', () => {
+        const span = onSpan((s) => recordGenAIMessage(s, 'tool', 'result', { toolCallId: 'call_1', toolName: 'search' }));
+        expect(span.events[0]!.attributes!['gen_ai.tool_call.id']).toBe('call_1');
+        expect(span.events[0]!.attributes!['gen_ai.tool.name']).toBe('search');
+    });
+
+    it('omits tool linkage keys when not supplied', () => {
+        const span = onSpan((s) => recordGenAIMessage(s, 'user', 'hi'));
+        expect(Object.keys(span.events[0]!.attributes!)).toEqual(['gen_ai.message.content']);
+    });
+
+    it('PII-scrubs message content before it leaves the process', () => {
+        const span = onSpan((s) => recordGenAIMessage(s, 'user', 'call the api with Bearer abc123token and password=hunter2'));
+        const content = span.events[0]!.attributes!['gen_ai.message.content'] as string;
+        expect(content).not.toContain('abc123token');
+        expect(content).not.toContain('hunter2');
+        expect(content).toContain('[redacted]');
+    });
+});