Expand the anonymous product event catalog

docs/TELEMETRY.md

@@ -6,13 +6,26 @@ includes prompts, code, file contents, or paths.
 
 ## What's collected
 
-Three events, each with a small set of properties:
+Each event carries a small set of properties:
 
 | Event | When | Properties |
 |---|---|---|
 | `cli_start` | Once per used session (see First-run disclosure) | (none beyond common properties) |
 | `session_end` | When a TUI session finishes | `status`, `turn_count`, `duration_ms`, `session_mode`, `exit_reason` |
 | `inference_turn` | Once per completed turn | `provider_id`, `model_id`, `input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_write_tokens`, `thinking_tokens`, `duration_ms` |
+| `slash_command` | A slash command is dispatched in the TUI | `command_name` |
+| `skill_used` | `use_skill` loads a skill that resolved | (none beyond common properties) |
+| `plugin_loaded` | A plugin is discovered and loaded at startup | `origin` |
+| `subagent_start` | A `task` dispatch begins | `agent_name` |
+| `subagent_end` | A `task` dispatch finishes | `agent_name`, `status`, `duration_ms` |
+| `permission_prompt` | An approval prompt is answered (or abandoned) | `decision`, `permission_kind` |
+| `compaction` | The compactor actually folds turns away | `mode`, `duration_ms`, `turns_before`, `turns_after` |
+| `crash` | A fatal error reaches the process-level handler | `kind`, `error_class` |
+| `auth_failure` | A provider rejects the stored credentials | `auth_provider` |
+
+`compaction` is deliberately silent on the runs where the compactor decides
+there is nothing to compact — an event that also fires on no-ops makes its own
+duration and turn-count averages meaningless.
 
 Common properties attached to every event: a random installation UUID
 (`distinct_id`), `session_id`, `service_version`, `os_type`, `os_arch`, and a
@@ -30,10 +43,45 @@ onboarding or settings. `model_id` is the model identifier exactly as
 configured — it is the one user-entered string that is sent, so do not put
 anything identifying in a model name.
 
+## Names are never sent, only categories
+
+Most of the things a usage event would naturally want to name are named by
+someone other than us: an MCP server key is a key in your settings, a skill is
+a directory in your repo, a plugin id is chosen by its author, an agent profile
+and a plugin's slash commands are project-local. On a private repo those names
+are your employer, your internal services, or fragments of your paths.
+
+So none of them are transmitted. Each is matched against a fixed list of names
+this project itself ships and reported as that name, or as `custom` when it
+matches nothing — with `mcp` as its own bucket for `permission_kind`, so the
+share of prompts driven by MCP stays visible without the server key coming
+with it. `skill_used` and `plugin_loaded` go further: there is no first-party
+list of skills or plugins to match against, so `skill_used` carries no name at
+all and `plugin_loaded` carries only `origin`, the discovery tier
+(`repo`, `user`, `project`, `path`).
+
+`error_class` is bucketed the same way: only the error types defined by the
+language are reported by name, because an error subclass defined in
+application or plugin code is as author-chosen as any other string. It appears
+on `crash` and nowhere else, so the column means one thing everywhere it is
+recorded.
+
+`auth_provider` is a separate property for that reason: it names which
+provider's sign-in was rejected (`codex`, `xai`), chosen from a fixed
+first-party set in `src/tui-opentui/session-chrome.ts`. No part of the
+provider's rejection message is sent.
+
+The mapping is `src/telemetry/classify.ts`, and the tests that feed each
+emission site a deliberately identifying name and assert it reaches no part of
+the payload are in `tests/unit/telemetry-product-events.test.ts`.
+
 ## What's never collected
 
 - Prompts, model output, or any conversation content
 - File paths, file contents, or repo/project names
+- Names anyone but this project chose: MCP servers, skills, plugins, agent
+  profiles, plugin-registered slash commands, error subclasses (see above)
+- Shell commands, tool arguments, or tool results
 - API keys, tokens, or any other credential
 - Anything not in the allowlist above
 
@@ -46,6 +94,12 @@ Any of the following disables telemetry entirely:
 - `CORBITS_TELEMETRY` set to any falsy value: `0`, `false`, `off`, `no`, or empty
 - `DO_NOT_TRACK=1` (the standard [Console Do Not Track](https://consoledonottrack.com/) convention)
 
+Turning telemetry off also discards whatever is still queued and unsent.
+Events captured earlier in the session but not yet transmitted are thrown
+away at the moment you opt out, not sent on the way out — opting out covers
+the activity you have already generated, not just the activity still to
+come.
+
 Re-enable from the same Telemetry tab or by removing the env var / settings
 override. While an env kill is active the Telemetry tab cannot re-enable —
 the env override always wins, and the attempt is refused rather than
@@ -92,9 +146,30 @@ Events are sent to PostHog. PostHog derives an approximate country from the
 request IP server-side; the client sends no location data itself. No
 self-hosted or third-party analytics beyond PostHog are used.
 
+## On the wire
+
+Events are not sent one at a time. Each captured event is stamped with its
+capture time and held in an in-memory queue, which is posted to PostHog's
+`/batch/` endpoint when it reaches the batch size or when the batch
+interval elapses, whichever comes first. At most one request is ever in
+flight: events captured while a request is open wait for it rather than
+opening another connection. Exit paths flush the queue, bounded by a short
+deadline so a slow endpoint cannot delay quitting.
+
+The queue has a hard depth limit. Once it is full — which in practice means
+the endpoint is unreachable, as on a captive portal or behind a hung proxy
+— the oldest queued events are dropped to make room for new ones. Telemetry
+is therefore lossy by design: it never grows memory without bound, never
+retries indefinitely, and never blocks or reports failures to the user.
+Nothing is written to disk, so dropped events are gone rather than deferred
+to a later run.
+
+See `src/telemetry/index.ts` for the batch size, interval, and queue limit
+in force.
+
 ## Not this document
 
 Local performance tracing and optional OpenTelemetry export to an operator-owned
 collector (Phoenix, PostHog OTEL, Jaeger, generic OTLP) are documented in
-`docs/PERFTRACE.md`. That pipe is separate: it does not expand these three
-events, and product telemetry opt-out does not control OTEL export.
+`docs/PERFTRACE.md`. That pipe is separate: it does not expand the events
+above, and product telemetry opt-out does not control OTEL export.

src/agent/tools.ts

@@ -16,6 +16,7 @@ import {
   type ShellTimeoutConfig,
 } from "../plugins/shell-guard-plugin.js";
 import { advertiseEditFileLineRange } from "../plugins/edit-file-line-range.js";
+import type { Telemetry } from "../telemetry/index.js";
 import type { PermissionGate } from "../permission/gate.js";
 import { buildCorePosixToolPlugins } from "./posix-tool-plugins.js";
 import { createLazyBlobReader } from "./lazy-blob-reader.js";
@@ -112,6 +113,9 @@ export type AgentToolsetArgs = {
   // Real sessions always pass their detected values — see tool-search.ts for
   // why these must be fixed for the session's life.
   toolAvailability?: ToolAvailability;
+  // Records skill loads and sub-agent dispatch. Omitted (tests, ad-hoc
+  // toolsets) means those events are never emitted.
+  telemetry?: Telemetry;
   // When provided, the agent gets a `task` tool that delegates to autonomous
   // sub-agents. Omitted in contexts that cannot spawn sub-agents (e.g. tests).
   subAgent?: {
@@ -210,7 +214,7 @@ export async function createAgentToolset(args: AgentToolsetArgs): Promise<AgentT
       ),
     })),
     createListDirTool(cwd),
-    createUseSkillTool(cwd, skillDirs),
+    createUseSkillTool(cwd, skillDirs, args.telemetry),
     createWebFetchTool(),
     createWebSearchTool(),
     ...(subAgentsEnabled && args.subAgent !== undefined
@@ -237,6 +241,7 @@ export async function createAgentToolset(args: AgentToolsetArgs): Promise<AgentT
             ...(args.subAgent.useWorktree !== undefined
               ? { useWorktree: args.subAgent.useWorktree }
               : {}),
+            ...(args.telemetry !== undefined ? { telemetry: args.telemetry } : {}),
           }),
           ...(args.subAgent.profiles !== undefined
             ? [

src/agent/use-skill.ts

@@ -4,6 +4,7 @@ import type { ToolDefinition } from "@intx/types/runtime";
 import { type } from "arktype";
 
 import { resolveSkillBody } from "../extensions/skills.js";
+import { NOOP_TELEMETRY, type Telemetry } from "../telemetry/index.js";
 
 // Lazy skill loading: the available skills are listed by name + description in
 // the system prompt, but their full instructions are pulled into context only
@@ -24,7 +25,11 @@ const useSkillDefinition: ToolDefinition = {
 
 const UseSkillArgs = type({ name: "string" });
 
-export function createUseSkillTool(cwd: string, skillDirs: string[] = []): AgentTool {
+export function createUseSkillTool(
+  cwd: string,
+  skillDirs: string[] = [],
+  telemetry: Telemetry = NOOP_TELEMETRY,
+): AgentTool {
   return stringTool({
     definition: useSkillDefinition,
     handler: async (rawArgs: Record<string, unknown>): Promise<string> => {
@@ -34,6 +39,10 @@ export function createUseSkillTool(cwd: string, skillDirs: string[] = []): Agent
       if (name.length === 0) return "Error: use_skill requires a non-empty name.";
       const body = await resolveSkillBody(cwd, name, skillDirs);
       if (body === undefined) return `No skill named "${name}" is available.`;
+      // Skills are project- or plugin-authored, so the name is as identifying
+      // as any other user-written string and never leaves the process; the
+      // event records only that a skill was loaded.
+      telemetry.capture("skill_used");
       return `Skill "${name}" — follow these instructions for this task:\n\n${body}`;
     },
   });

src/exec/runner.ts

@@ -53,6 +53,7 @@ import type {
   PermissionRequest,
 } from "../permission/types.js";
 import { createAgentToolset, type AgentToolset, type OperatorResult } from "../agent/tools.js";
+import { liveTelemetry } from "../telemetry/singleton.js";
 import { collectToolPlugins, resolveToolPlugins } from "../plugins/tool-plugins.js";
 import {
   expandExistingPluginMembers,
@@ -262,6 +263,7 @@ export async function runExec(config: Config): Promise<ExecResult> {
       isProjectPluginTrusted,
       isRegisteredPathTrusted,
       diagnostics: pluginLoadDiag,
+      telemetry: liveTelemetry,
     });
     emitPluginWarningSummary(pluginLoadDiag, (line) => logger.warn(line));
     // Metadata-only (untrusted) modules stay out of executable plugins.
@@ -298,6 +300,7 @@ export async function runExec(config: Config): Promise<ExecResult> {
 
     const permissionGate = createPermissionGate({
       approvals: seededApprovals,
+      telemetry: liveTelemetry,
       cwd: config.cwd,
       rootsProvider: createWorktreeRootsProvider(config.cwd),
       providerName: config.providerName,
@@ -326,6 +329,7 @@ export async function runExec(config: Config): Promise<ExecResult> {
       cwd: config.cwd,
       permissionGate,
       skillDirs,
+      telemetry: liveTelemetry,
       ...(shellTimeout !== undefined ? { shellTimeout } : {}),
       ...(toolWatchdog !== undefined ? { toolWatchdog } : {}),
       ...(localSettingsForMode?.env !== undefined ? { shellEnv: localSettingsForMode.env } : {}),
@@ -544,6 +548,7 @@ export async function runExec(config: Config): Promise<ExecResult> {
           "pruning-compactor": createSessionPruningCompactor({
             compactionMode: liveCompactionMode,
             summarize: summarizeForCompaction,
+            telemetry: liveTelemetry,
           }),
         },
       });

src/index.ts

@@ -9,6 +9,7 @@ import { ensureTelemetrySettings, globalSettingsPath } from "./config/settings.j
 import { installFileLogSink } from "./logging/sink.js";
 import { flushPerfToOtel } from "./perf/index.js";
 import { createTelemetry, telemetryDisabledByEnv } from "./telemetry/index.js";
+import { classifyErrorClass } from "./telemetry/classify.js";
 import { getTelemetry, setTelemetry } from "./telemetry/singleton.js";
 import { runExec } from "./exec/runner.js";
 import { runOnboarding } from "./tui/onboarding.js";
@@ -154,6 +155,12 @@ export async function handleFatal(kind: CrashKind, error: unknown): Promise<void
     process.stderr.write("failed to write crash report\n");
   }
   await finalizeActiveRunOnCrash(error);
+  // kind is one of the two process-level handler names. A constructor name is
+  // author-chosen — an application or plugin error subclass can be as
+  // identifying as any other free text — so only the language's own error
+  // types are reported by name.
+  getTelemetry().capture("crash", { kind, error_class: classifyErrorClass(error) });
+  await getTelemetry().flush();
   process.exit(1);
 }
 

src/permission/gate.ts

@@ -28,6 +28,25 @@ import {
 import type { MCPClient } from "../mcp/client.js";
 import { end, start } from "../perf/index.js";
 import { currentTurnId } from "../perf/reactor-spans.js";
+import { classifyPermissionKind } from "../telemetry/classify.js";
+import { NOOP_TELEMETRY, type Telemetry } from "../telemetry/index.js";
+
+// Closes out an operator prompt: ends the wait span and records the outcome.
+// buildRequests yields at most one request per tool call, and the two prompt
+// sites below are mutually exclusive, so this runs once per prompt shown.
+function finishApprovalWait(
+  telemetry: Telemetry,
+  waitSpanId: string,
+  tool: string,
+  outcome: ApprovalOutcome | undefined,
+): void {
+  const decision = outcome !== undefined && outcome.allow ? "allow" : "deny";
+  end(waitSpanId, outcome !== undefined ? { decision } : undefined);
+  telemetry.capture("permission_prompt", {
+    decision,
+    permission_kind: classifyPermissionKind(tool),
+  });
+}
 
 export type GateVerdict = { allowed: true } | { allowed: false; reason: string };
 
@@ -211,6 +230,10 @@ export type PermissionGateOptions = {
   // restriction anchored to the session cwd; a caller resolving a sub-agent
   // request's own cwd would clear restrictions the gate still enforces.
   onGrant?: (approval: Approval, covers: (request: PermissionRequest) => boolean) => void;
+  // Records that a prompt was shown and how it was answered. Injected rather
+  // than read from the process-wide handle so a gate built without one is
+  // silent by construction.
+  telemetry?: Telemetry;
 };
 
 export type PermissionGate = {
@@ -246,6 +269,7 @@ export type PermissionGate = {
 
 export function createPermissionGate(options: PermissionGateOptions): PermissionGate {
   const { requestApproval, persist, interactive, skipPermissions, providerName, model, cwd } = options;
+  const telemetry = options.telemetry ?? NOOP_TELEMETRY;
   const mcpTiers = options.mcpTiers ?? createMcpToolPermissionRegistry();
   const resolvedCwd = cwd ?? process.cwd();
   const rootsProvider = options.rootsProvider ?? createWorktreeRootsProvider(resolvedCwd);
@@ -463,12 +487,7 @@ export function createPermissionGate(options: PermissionGateOptions): Permission
         try {
           outcome = await requestApproval(requestForOperator);
         } finally {
-          end(
-            waitSpanId,
-            outcome !== undefined
-              ? { decision: outcome.allow ? "allow" : "deny" }
-              : undefined,
-          );
+          finishApprovalWait(telemetry, waitSpanId, request.tool, outcome);
         }
         if (outcome === undefined || !outcome.allow) {
           const suffix =
@@ -516,12 +535,7 @@ export function createPermissionGate(options: PermissionGateOptions): Permission
       try {
         outcome = await requestApproval(request);
       } finally {
-        end(
-          waitSpanId,
-          outcome !== undefined
-            ? { decision: outcome.allow ? "allow" : "deny" }
-            : undefined,
-        );
+        finishApprovalWait(telemetry, waitSpanId, request.tool, outcome);
       }
       if (outcome === undefined || !outcome.allow) {
         const suffix =