Skip to content

feat(agents): add the error-tracking orchestrator flow - #377

Closed
ablaszkiewicz wants to merge 30 commits into
mainfrom
error-tracking-flow
Closed

ablaszkiewicz wants to merge 30 commits into
mainfrom
error-tracking-flow

Conversation

@ablaszkiewicz

@ablaszkiewicz ablaszkiewicz commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Part of a 3-PR set

Problem

wizard error-tracking (PostHog/wizard#1185) needs an orchestrator flow. The flow installs PostHog first when the repo has no PostHog.

Changes

New flow context/agents/error-tracking/, a seed and 8 tasks:

  • setup-error-tracking (seed): checks the PostHog init and its key, and picks the source-map uploader. React Native and Expo use react-native. Readable-stack platforms and Astro use none.
  • install, init: add and initialize the SDK. init keeps the env names that the code reads, loads .env where the platform does not, and writes init code that type-checks.
  • capture-exceptions: turns on SDK autocapture and hooks the framework error handler. It adds no second global listener.
  • credentials, configure, wire-ci, test-setup: wire the source-map upload into the build and CI. The API key stays a secret.
  • report: writes posthog-error-tracking-report.md.

Shared skill changes:

  • integration-v2/error-tracking-step: 40 variants with the error-tracking docs. The default integration-v2 flow uses them too. A new test keeps the variants in sync with integration.
  • error-tracking-upload-source-maps: install with the project's package manager. Do not set a webpack devtool beside the PostHog plugin.

Test plan

🤖 Generated with Claude Code

https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z

Nine prompts under context/agents/error-tracking/ backing the wizard's
new `error-tracking` command. The seed establishes whether PostHog is
integrated and which uploader variant applies, then queues install/init
(integration-v2 step-skills, like replay-vision) on uninstrumented
repos, exception capture via integration-v2-error-tracking-step, the
source-map subgraph (adapted from the error-tracking-upload-source-maps
flow) where the platform ships minified bundles or stripped binaries,
and a report that merges it all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/groq
  • /wizard-ci ai-observability/manual-capture
Show more apps
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a wizard branch:

  • /wizard-ci all wizard:my-branch

Add wizard:<branch> to any command above to pin the wizard branch. It defaults to main.

Results will be posted here when complete.

ablaszkiewicz and others added 21 commits September 1, 2026 18:21
…ness

install/init move terra→sol low, capture-exceptions terra low→sol
medium. Pairs with the wizard binding change routing the program to the
orchestrator on pi; seed stays terra and report stays luna, matching
the metrics flow's pattern.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…nager

A configure run in a pnpm workspace copied the docs' npm install and
npm hard-fails on workspace:* deps (EUNSUPPORTEDPROTOCOL); the agent
retried npm three ways and never switched. New skill bullet: detect the
manager from the lockfile, translate the docs' npm commands, and read
EUNSUPPORTEDPROTOCOL as wrong-manager, never a flag to retry. The
error-tracking flow's configure task now also mandates
detect_package_manager before the first install. Verified: the pnpm
monorepo fixture run that failed now completes 4/4, recovering to
`pnpm add -D` on first contact with the error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…anning

A seed run saw "PostHog already integrated" and enqueued only
capture+report, never applying the uploader precedence to a tsc-built
Node service — while an identical sibling fixture planned the full
graph. The two facts are now declared independent, a compiled/bundled
JS project is explicitly never "none", and the success criteria demand
the plan state the uploader decision (or the readable-stack reason)
explicitly. Verified: the fixture that under-planned now queues the
full node-uploader graph and completes 5/5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
The integration init skill documents its public keys in .env.example;
the credentials task wrote its upload variables only to the real env
file, so the names never reached the committed template. New step 4:
append the names with placeholders (never values) — for the next
developer, and so a later run's check_env_keys reports the expectation.
Scoped to the error-tracking flow's task prompt; the shared uploader
skill is untouched, so the standalone upload-source-maps program keeps
its current behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
Astro inlines scripts below its asset limit into the HTML, so a build can
emit a .map with no .js beside it. posthog-cli then finds zero pairs, exits
non-zero, and takes the whole build down with it. Treat Astro as a no-variant
platform: install and exception capture still run, the build script is left
untouched, and the report says why upload was skipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…Node

The shared integration-v2-init skill assumes the framework auto-loads .env,
which is true for Next/Nuxt/Astro/SvelteKit/Vite but not for plain Node
backends. On Express, Fastify, Hono, Koa and raw node:http the run wrote
process.env reads and a .env file with nothing to bridge them, so every app
died at module load on its own missing-variable guard.

Teach the init task to wire the loading (dotenv, or --env-file) and to start
the app once to confirm it boots. Also have configure check that some script
actually runs a newly emitted bundle, since two apps built to dist/ while
start still ran the source.

Scoped to this flow's task prompts; the shared skill is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…n Angular

On the stock @angular/build builder nothing defines process.env or
import.meta.env in the browser bundle, but the run wrote
import.meta.env['NG_APP_POSTHOG_PROJECT_TOKEN'] into src/environments/*.
Reading a property off undefined throws while the module evaluates, so
<app-root> stayed empty and the app rendered a blank page.

Name the mechanism's real source (@ngx-env/builder supplies import.meta.env
and the NG_APP_ prefix) and require it to be an existing dependency before
use. Otherwise fall back to the skill's existing "no valid environment to read
from" case: embed the public project token in the committed environment files,
which is where it ends up in the bundle anyway.

Scoped to this flow's task prompt; the shared skill is untouched.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
The earlier rule stopped the run reaching for import.meta.env, but it then read
env['NG_APP_POSTHOG_PROJECT_TOKEN'] from the project's generated
src/environments/.env.ts, which only defines npm_package_version. The key came
back undefined, and because a production build takes the guard's quiet return
path the app rendered normally while PostHog never initialised — no requests at
all.

Name that second empty container, require the lookup's source to be opened and
the key seen before wiring it, and make the task prove initialisation by
serving the build and confirming a request reaches the PostHog host.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
Pointing `start` at dist/index.js is only safe if that bundle can execute.
On koa-notes the esbuild command had no --format=esm while package.json sets
"type": "module", so the emitted CommonJS died at boot with
`ReferenceError: module is not defined in ES module scope` — a breakage that
only appeared once `start` stopped running the source.

Require the task to run the build and start the script once, and name the
module-format mismatch as the usual cause.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…uild rule

The new "run the build and start that script once" instruction sat directly
above "Do not run the build", leaving the task with contradictory orders.
Carve the single verification out of the prohibition.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The agent runs in a sandbox that cannot build or start the project, so rules
telling init to boot the app and configure to run the built script were not
executable.

Keep the knowledge, drop the execution: the env loader and the module format
must be correct by construction, checked by reading package.json's "type" and
by opening the file a key lookup reads from before wiring it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
credentials and configure run in parallel: one writes the env file, the other
writes the code that reads it, and neither could see the other's choice. On
Nuxt 4 that split them — the skill's Nuxt reference documents variable names
only for the posthog-cli path, so credentials wrote POSTHOG_CLI_* while
configure wired the @posthog/nuxt module against POSTHOG_API_KEY. Nothing
errored; the upload was simply skipped and every trace stayed minified.

Give both tasks the same rule, in terms neither framework nor uploader
specific: look for the other half of the contract before writing, adopt it
verbatim if it exists, decide only when it does not, and always name the
chosen set in full in the handoff. configure gets check_env_keys so it can
read names without values.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…eads

A repo can carry a PostHog init whose env variable was never defined — the
call is there, the name it reads is nowhere, and the client is built from an
empty string. The init task read that as "already initialised" and left it
alone, so the app captured nothing while every other step looked clean.

Init now confirms the name the code reads is present with check_env_keys
before it leaves an existing init alone, and writes it when it is absent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The seed read fact 1 as "the SDK is in the manifest", so a repo carrying an
init whose key variable was defined nowhere skipped install and init entirely.
The client was then built from an empty string: the build stayed clean, every
capture call returned, and no event ever arrived.

Fact 1 is now about the pair — the call and the key it reads. When the name is
defined nowhere, or the seed cannot tell, it queues init, which re-checks the
pair and leaves a complete init alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
A Next.js run copied the uploader example into next.config.ts without the
non-null assertion the reference shows. process.env lookups are
string | undefined, the plugin option is string, and the build failed type
checking — after the upload step had already run, so the log showed a
successful chunk upload and the build still exited 1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…dary

wire-ci was written entirely around credentials, and pointed at the skill's CI
step while the git-context guidance sits in a different step the task never
named. In a container build neither reaches the uploader: .git is in
.dockerignore, and the CI's own variables stop at the docker build command.
Reproduced against the nested composite action fixture:

  Release fields are incomplete and git info is unavailable   (exit 1)

Docker build args are not inherited from the environment, so this fails on a
GitHub Actions runner exactly as it does locally — verified by building with
GITHUB_SHA exported but not forwarded.

The ARG-plus-ENV shape is scoped to the git variables and non-secret settings;
the API key keeps its build-secret mount, which a first pass had demoted to a
plain build arg.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…ndary

wire-ci carried the upload credentials through the pipeline but nothing owned
the variable the deploy step hands the running app. The step predates PostHog,
so it passed the project's old name while init had settled on another, and the
container booted with an undefined key.

Reproduced by running the image exactly as the deploy step does:

  POSTHOG_PROJECT_TOKEN= undefined
  NODE_ENV= production
  exit=0

Nothing fails: in production the init guard returns rather than throwing, so the
process serves traffic and reports nothing. The runtime boundary is now an
explicit third place a name has to match, with the deploy step renamed to follow
the code rather than the other way round.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The release-identity rule was written as if a pipeline always exists, so a
project whose Dockerfile says it is built and run by hand got eight GITHUB_*
build args nothing ever fills. Empty variables resolve to no release, and a
hand-run docker build failed:

  Oops! Release fields are incomplete and git info is unavailable   (exit 1)

Reading as wired while failing the build is the worst of both. With no provider
the identity is supplied outright instead — the re-run took --release-version
from the manifest, and the hand-run build resolves cicd-docker-node-raw@0.1.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…s on

The Go run wrote credentials to .env, rewrote main.go to read them from the
process env, added a Makefile run target that loads nothing, and then pointed
the user at that target to verify. It prints its own "variable missing" guard
and exits: build green, symbols uploaded, no event.

The run-script rule now covers reaching the values as well as running the right
artifact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
error-tracking-step borrowed the integration group's variants, so the task that
wires exception capture read the SDK library page. For Go and Rust those pages
say nothing about panic hooks or log handlers, and the task invented the API:
a capture_panics method on the wrong builder, and a slog handler with no
distinct-ID resolver, which drops every capture without erroring.

The matrix is unchanged - same 40 ids, frameworks and defaults, verified against
the built menu before and after. Only docs_urls move: 39 variants now point at
their error-tracking installation page, java being the one PostHog has none for.
Where the page is a generic substitute (django to python, astro to web) the
framework's own library page stays alongside.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The Rust page shows a global initializer and an explicit client side by side.
The task took the wiring from one and the calls from the other, binding
init_global's Result<(), Error> as if it were a client, so cargo build failed
with three E0599s. It also wrote the build as a !-prefixed cargo alias, which
is a git convention cargo has no support for.

Six lines: follow one setup the whole way through, and put a multi-step build
where the tool actually runs it. The re-run compiles, uploads symbols, and a
panic resolves to rust_app::kaboom_rust2 -> read_rust_retry_budget2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
ablaszkiewicz and others added 5 commits September 11, 2026 17:31
The capture step's Nuxt matrix marked `nuxt-3-6` as the default, so every
Nuxt project — including Nuxt 4 — was handed the v3.0-3.6 installation page
and wired PostHog by hand through a client plugin. Nuxt 3.7 shipped in
September 2023; the `@posthog/nuxt` module is the documented path for
everything since, and it initialises both the Vue and Nitro sides itself.

Moves `default: true` to the `nuxt-4` variant, which already pointed at
installation/nuxt-3-7.md. Two lines change in the built menu and nothing else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
Exception autocapture already registers the global handlers — window.onerror
and unhandledrejection in the browser, uncaughtException and unhandledRejection
in Node, sys.excepthook in Python, the panic hook in Rust, the uncaught
exception and signal handlers on iOS and Android. A second listener the task
writes itself on the same event sends every error twice and drifts from the
SDK's handling as the SDK changes.

The capture task now turns autocapture on through the SDK's option and
registers none of those handlers. Framework hooks stay in scope, because they
see errors a framework swallows before any global handler fires: Express
error middleware, Fastify setErrorHandler, Hono onError, Vue errorHandler,
Angular ErrorHandler, SvelteKit handleError, a React error boundary, Next.js
global-error. Go, which has no autocapture, keeps its entry-point boundary.

The success check gains the same condition, so a reviewer can confirm it by
reading the diff.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013SEsZQQrYjXPVEhQuoSwE6
error-tracking-step declares its variants literally, and the orchestrator
resolves it per framework with no fallback. A variant added to integration
and not copied here would abort the default orchestrator flow for that
framework.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013SEsZQQrYjXPVEhQuoSwE6
- model_sdk: claude-sonnet-5 and claude-haiku-4-5, as #389 did for the
  other flows. The old ids are no longer valid in the wizard.
- install and init run terra at low effort, like integration-v2 and
  replay-vision. wire-ci runs sol at medium. The gateway mint pins sol
  to medium.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013SEsZQQrYjXPVEhQuoSwE6
A React Native repo carries ios/Podfile, an .xcodeproj and Gradle files, so
the seed's native-first precedence picked the ios uploader and wired no
JavaScript source maps. react-native or expo in package.json now wins over
every native marker. Verified with e2e runs on the react-native fixture
(ios before, react-native after) and on react-native-expo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z
In a type-checked project the init guard tested a separate variable and then
passed the unchecked one to the SDK, so tsc stopped the build with TS2345 and
no source maps uploaded. The init now passes the variable it checked, or keeps
an existing ?? '' / ! as it is. Verified with e2e runs on node-webpack and
node-raw: no type errors, and node-raw builds, uploads and starts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z
@posthog

posthog Bot commented Sep 14, 2026

Copy link
Copy Markdown

🦔 PostHog Review reviewed this pull request

Found 7 must fix, 20 should fix, 7 consider.

Published 34 findings (view the review).

ablaszkiewicz and others added 2 commits September 14, 2026 15:42
With sourcemaps.enabled the webpack plugin adds its own SourceMapDevToolPlugin.
A devtool that also emits maps makes webpack fail with "Conflict: Multiple
assets emit different content to the same filename index.js.map", and nothing
uploads. Verified with an e2e run on node-webpack: no devtool, the build
passes, one chunk uploads, and the app starts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z
The flow relies on SDK autocapture and does not change team settings. Remove
the products-enable step and posthog_exec from the report task.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z
@posthog

posthog Bot commented Sep 14, 2026

Copy link
Copy Markdown

PostHog Review alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏

@posthog posthog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review

Found 7 must fix, 20 should fix, 7 consider.

Comment on lines +17 to +22
Make the credentials reach the production build wherever it actually runs, so
source maps upload on every deploy — not just on a local build. Install the
skill your task input names (`install_skill` with the `skillId`) and follow its
**"Set up CI for automatic uploads"** step — it owns tracing where the
production build runs and wiring the credentials through every layer, whatever
the CI provider.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove Docker instructions that expose personal API keys

consider security

Issue description

This task declares the installed skill authoritative. That skill still puts POSTHOG_CLI_API_KEY in Docker ARG, ENV, and build-args for multi-stage builds. Those instructions conflict with this prompt. Following them can store the personal key in build history, cache metadata, or provenance.

Why we think it's a valid issue
  • Checked: which skill this task installs. The seed passes skillId: error-tracking-upload-source-maps-<variant> (context/agents/error-tracking/setup-error-tracking.md, the "uploader skill id" paragraph), so wire-ci installs context/skills/error-tracking-upload-source-maps. I read its "Set up CI for automatic uploads" step (description.md:170-307) and compared it with wire-ci.md:17-22 and wire-ci.md:56-62.
  • Found: the disagreement is real. description.md:189 tells the agent to put ARG/ENV for the key in the build stage of a multi-stage Dockerfile. description.md:203-206 shows that Dockerfile, and description.md:211 shows docker build --build-arg POSTHOG_CLI_API_KEY=…. description.md:245 and description.md:265 put POSTHOG_CLI_API_KEY in the build-args: input of docker/build-push-action. wire-ci.md:59-62 forbids exactly that shape.
  • Found: the skill is stage-aware, not uniformly unsafe. description.md:190 and description.md:222-225 require a BuildKit secret for the single-stage layout, which is the layout that bakes the key into the shipped image, and they tell the caller to use the secrets: input in place of build-args: there.
  • Found: this PR does not add the Docker instructions. Against origin/main the PR changes 2 lines of description.md — a package-manager rule and a webpack devtool rule. Every ARG/ENV/build-args example is pre-existing. wire-ci.md is the first prompt in the flow that pushes toward build secrets, so the PR moves secret handling forward, not back.
  • Found: the override sits in the same file as the "follow the skill" sentence. wire-ci.md:56-62 names the key, forbids ARG and ENV for it, and gives the reason. The flow uses this layering on purpose: configure.md ("even when they are not the ones your skill's example shows") and credentials.md ("even when they are not the ones your skill would suggest") tell the agent to deviate from the skill example in the same way.
  • Impact: the two documents still disagree for the most common layout, a multi-stage Dockerfile built by GitHub Actions. An agent can follow the concrete skill example and write the personal API key into build-args:. The skill itself states at description.md:190 that docker history can reveal build args. The specific prose rule in the task prompt usually wins over a reference example, so the leak is possible, not certain.
  • Priority: lowered to consider. The inconsistency is verifiable and worth recording, but this PR does not create it, the flagged file already carries the explicit override, and the skill already mandates a BuildKit secret in the one layout that keeps the key in the shipped image. The suggested rewrite of every Docker example in a pre-existing skill file is larger than the defect.
Suggested fix

Update the skill to use BuildKit secrets for the personal API key in every Docker layout. Reserve ARG, ENV, and build-args for non-secret settings and git metadata. Rewrite all conflicting Docker examples.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/wire-ci.md#L17-22
@context/agents/error-tracking/wire-ci.md#L56-62

<issue_description>
This task declares the installed skill authoritative. That skill still puts `POSTHOG_CLI_API_KEY` in Docker `ARG`, `ENV`, and `build-args` for multi-stage builds. Those instructions conflict with this prompt. Following them can store the personal key in build history, cache metadata, or provenance.
</issue_description>

<issue_validation>
- **Checked:** which skill this task installs. The seed passes `skillId: error-tracking-upload-source-maps-<variant>` (context/agents/error-tracking/setup-error-tracking.md, the "uploader skill id" paragraph), so `wire-ci` installs `context/skills/error-tracking-upload-source-maps`. I read its "Set up CI for automatic uploads" step (description.md:170-307) and compared it with wire-ci.md:17-22 and wire-ci.md:56-62.
- **Found:** the disagreement is real. description.md:189 tells the agent to put `ARG`/`ENV` for the key in the build stage of a multi-stage Dockerfile. description.md:203-206 shows that Dockerfile, and description.md:211 shows `docker build --build-arg POSTHOG_CLI_API_KEY=…`. description.md:245 and description.md:265 put `POSTHOG_CLI_API_KEY` in the `build-args:` input of `docker/build-push-action`. wire-ci.md:59-62 forbids exactly that shape.
- **Found:** the skill is stage-aware, not uniformly unsafe. description.md:190 and description.md:222-225 require a BuildKit secret for the single-stage layout, which is the layout that bakes the key into the shipped image, and they tell the caller to use the `secrets:` input in place of `build-args:` there.
- **Found:** this PR does not add the Docker instructions. Against `origin/main` the PR changes 2 lines of description.md — a package-manager rule and a webpack `devtool` rule. Every `ARG`/`ENV`/`build-args` example is pre-existing. wire-ci.md is the first prompt in the flow that pushes toward build secrets, so the PR moves secret handling forward, not back.
- **Found:** the override sits in the same file as the "follow the skill" sentence. wire-ci.md:56-62 names the key, forbids `ARG` and `ENV` for it, and gives the reason. The flow uses this layering on purpose: configure.md ("even when they are not the ones your skill's example shows") and credentials.md ("even when they are not the ones your skill would suggest") tell the agent to deviate from the skill example in the same way.
- **Impact:** the two documents still disagree for the most common layout, a multi-stage Dockerfile built by GitHub Actions. An agent can follow the concrete skill example and write the personal API key into `build-args:`. The skill itself states at description.md:190 that `docker history` can reveal build args. The specific prose rule in the task prompt usually wins over a reference example, so the leak is possible, not certain.
- **Priority:** lowered to `consider`. The inconsistency is verifiable and worth recording, but this PR does not create it, the flagged file already carries the explicit override, and the skill already mandates a BuildKit secret in the one layout that keeps the key in the shipped image. The suggested rewrite of every Docker example in a pre-existing skill file is larger than the defect.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Update the skill to use BuildKit secrets for the personal API key in every Docker layout. Reserve `ARG`, `ENV`, and `build-args` for non-secret settings and git metadata. Rewrite all conflicting Docker examples.
</potential_solution>

Comment on lines +343 to +351
- id: kmp
display_name: Kotlin Multiplatform
tags:
- kmp
- kotlin
- multiplatform
- mobile
docs_urls:
- https://posthog.com/docs/error-tracking/installation/kmp.md

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The KMP variant cannot resolve during orchestrator preflight

must_fix bug

Issue description

The wizard detects KMP as framework kmp. Its resolver selects only entries with a matching framework. This variant omits that field, so preflight aborts before any task runs.

Why we think it's a valid issue
  • Checked: the wizard resolver resolveSkillVariantId (PostHog/wizard src/lib/agent/runner/sequence/orchestrator/orchestrator-runner.ts:169-180), the preflight loop that calls it (orchestrator-runner.ts:549-590), the menu loader fetchSkillMenu and expandBundleEntry (src/lib/wizard-tools/tools.ts:91-118), the wizard framework list (src/lib/constants.ts:74-105), and the manifest that node scripts/build.js writes to dist/skills/skill-menu.json.
  • Found: the resolver first tries an exact id match, then filters on e.group === skillId && e.framework === framework. fetchSkillMenu replaces every bundle entry with its variants (tools.ts:117), so the bare id integration-v2-error-tracking-step is absent from the menu the resolver reads. The built entry integration-v2-error-tracking-step-kmp carries no framework key. Resolution returns undefined, the preflight adds the skill to missingVariants, and wizardAbort stops the run with the message "Setup instructions for this project failed to download".
  • Found: the wizard detects KMP as framework kmp (src/lib/constants.ts:91, src/frameworks/kmp/kmp-wizard-agent.ts), and the new command applies no platform allow-list — the program comment in feat(error-tracking): add the error-tracking program on the orchestrator flow wizard#1185 states "every detectable framework has an error-tracking-step variant". context/agents/error-tracking/capture-exceptions.md:9 declares skills: [integration-v2-error-tracking-step, posthog-best-practices], so the preflight does reach this variant.
  • Found: the gap is older than this PR. context/skills/integration/config.yaml:299-305 also omits framework for kmp, and resolveVariantsFrom copies the field only when it exists (scripts/lib/skill-generator.js:257). Before this PR the step used variants_from: integration and produced the same framework-less entry. The built manifest also shows integration-kmp, integration-v2-install-kmp, integration-v2-init-kmp and integration-v2-capture-kmp without framework, so the shipped default flow aborts for KMP today. This widens the impact; it does not remove it.
  • Found: the new parity test compares ${v.id}:${v.framework} pairs (scripts/lib/tests/error-tracking-step-parity.test.js), so it matches kmp:undefined on both sides and lets the gap through. It also fails if you add framework: kmp to this file alone, which confirms that both configs must change together.
  • Impact: a Kotlin Multiplatform project that runs wizard error-tracking stops at preflight with a misleading download error, before any task runs. The variant is unreachable although this file gives it a KMP error-tracking docs page. The other two framework-less variants, react-vite and php, are safe because the wizard has no matching detected framework for them.
Suggested fix

Add framework: kmp here and to the canonical integration KMP variant. Add a resolver test that covers KMP across all composed step skills.

Prompt to fix with AI (copy-paste)
## Context
@context/skills/integration-v2/error-tracking-step/config.yaml#L343-351

<issue_description>
The wizard detects KMP as framework `kmp`. Its resolver selects only entries with a matching `framework`. This variant omits that field, so preflight aborts before any task runs.
</issue_description>

<issue_validation>
- **Checked:** the wizard resolver `resolveSkillVariantId` (PostHog/wizard `src/lib/agent/runner/sequence/orchestrator/orchestrator-runner.ts:169-180`), the preflight loop that calls it (`orchestrator-runner.ts:549-590`), the menu loader `fetchSkillMenu` and `expandBundleEntry` (`src/lib/wizard-tools/tools.ts:91-118`), the wizard framework list (`src/lib/constants.ts:74-105`), and the manifest that `node scripts/build.js` writes to `dist/skills/skill-menu.json`.
- **Found:** the resolver first tries an exact id match, then filters on `e.group === skillId && e.framework === framework`. `fetchSkillMenu` replaces every bundle entry with its variants (`tools.ts:117`), so the bare id `integration-v2-error-tracking-step` is absent from the menu the resolver reads. The built entry `integration-v2-error-tracking-step-kmp` carries no `framework` key. Resolution returns `undefined`, the preflight adds the skill to `missingVariants`, and `wizardAbort` stops the run with the message "Setup instructions for this project failed to download".
- **Found:** the wizard detects KMP as framework `kmp` (`src/lib/constants.ts:91`, `src/frameworks/kmp/kmp-wizard-agent.ts`), and the new command applies no platform allow-list — the program comment in PostHog/wizard#1185 states "every detectable framework has an error-tracking-step variant". `context/agents/error-tracking/capture-exceptions.md:9` declares `skills: [integration-v2-error-tracking-step, posthog-best-practices]`, so the preflight does reach this variant.
- **Found:** the gap is older than this PR. `context/skills/integration/config.yaml:299-305` also omits `framework` for `kmp`, and `resolveVariantsFrom` copies the field only when it exists (`scripts/lib/skill-generator.js:257`). Before this PR the step used `variants_from: integration` and produced the same framework-less entry. The built manifest also shows `integration-kmp`, `integration-v2-install-kmp`, `integration-v2-init-kmp` and `integration-v2-capture-kmp` without `framework`, so the shipped default flow aborts for KMP today. This widens the impact; it does not remove it.
- **Found:** the new parity test compares `${v.id}:${v.framework}` pairs (`scripts/lib/tests/error-tracking-step-parity.test.js`), so it matches `kmp:undefined` on both sides and lets the gap through. It also fails if you add `framework: kmp` to this file alone, which confirms that both configs must change together.
- **Impact:** a Kotlin Multiplatform project that runs `wizard error-tracking` stops at preflight with a misleading download error, before any task runs. The variant is unreachable although this file gives it a KMP error-tracking docs page. The other two framework-less variants, `react-vite` and `php`, are safe because the wizard has no matching detected framework for them.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Add `framework: kmp` here and to the canonical `integration` KMP variant. Add a resolver test that covers KMP across all composed step skills.
</potential_solution>

Comment on lines +28 to +31
The SDK is declared in the manifest at a real version — whether it already
was or you just installed it — or your handoff plainly says why the
environment stopped you. Your handoff names the manifest and the package, so
later steps import it under the name they will actually get.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An install failure still unblocks tasks that import the SDK

should_fix bug

Issue description

These lines accept an environment failure as successful completion. The install skill can finish without declaring the SDK. The next task then assumes the SDK exists and adds imports for it. A registry or lockfile failure can therefore leave the project unable to build.

Why we think it's a valid issue
  • Checked: context/agents/error-tracking/install.md in full, the skill it delegates to (context/skills/integration-v2/install/description.md), the sibling prompt context/agents/integration-v2/install.md, the downstream prompts capture-exceptions.md and init.md, and report.md.
  • Found: The prompt hands the failure path to its skill. context/agents/error-tracking/install.md:19 says the skill owns "what counts as an environment failure, and the fallback". The skill defines no fallback. context/skills/integration-v2/install/description.md:20 ends that path with "do not spiral: report the exact failure and command in your handoff and finish". No step declares the package.
  • Found: The manifest fallback exists in exactly one place, and this flow does not reach it. context/agents/integration-v2/install.md:53 tells the agent to "add the PostHog package to the manifest by hand at a known-good version, so the dependency is still declared for the later steps and for the user's own install". That text sits inline in the default flow's own prompt, not in the shared skill, so the new prompt points at a fallback that never existed.
  • Found: The next task edits code on the assumption that the SDK resolves. context/agents/error-tracking/capture-exceptions.md:29 states "The SDK is installed and initialized ... build on that, do not re-check it", and that task holds Write and Edit. init.md also writes an init that constructs the client.
  • Impact: On the environment failures that context/agents/integration-v2/install.md:38 itself lists — a peer conflict the project already had, a broken lockfile, a package-manager provisioning failure — the run finishes with PostHog imports and an init in the source and no dependency in the manifest. The project stops building, and it built before the run. install.md:9 already grants Edit, so the repair is one sentence of prompt text, not new tooling.
  • Impact: The report does not reliably surface the failure as a follow-up either. The "What you still need to do" section that carries "Any other manual follow-up the handoffs carry" (report.md:38) is written only "When the run wired source-map upload" (report.md:30). On a readable-stack platform the upload subgraph is skipped, so that section is absent and the failed install has no guaranteed place in the user-facing output.
  • Priority: Lowered to should_fix. The path needs an environment failure, so it does not hit every run, and the same wording already ships in context/agents/replay-vision/install.md on main — this PR copies an established pattern rather than creating a regression. Note also that only part of the suggestion applies: restoring the known-good manifest fallback fits the design, while finishing the task as failed fights the deliberate "do not spiral" rule in both the skill and context/agents/integration-v2/install.md:32.
Suggested fix

Finish the install task as failed unless the package is declared. Alternatively, add a known-good manifest fallback before completion. Make capture-exceptions skip edits when the install or init handoff reports failure.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/install.md#L28-31

<issue_description>
These lines accept an environment failure as successful completion. The install skill can finish without declaring the SDK. The next task then assumes the SDK exists and adds imports for it. A registry or lockfile failure can therefore leave the project unable to build.
</issue_description>

<issue_validation>
- **Checked:** `context/agents/error-tracking/install.md` in full, the skill it delegates to (`context/skills/integration-v2/install/description.md`), the sibling prompt `context/agents/integration-v2/install.md`, the downstream prompts `capture-exceptions.md` and `init.md`, and `report.md`.
- **Found:** The prompt hands the failure path to its skill. `context/agents/error-tracking/install.md:19` says the skill owns "what counts as an environment failure, and the fallback". The skill defines no fallback. `context/skills/integration-v2/install/description.md:20` ends that path with "do not spiral: report the exact failure and command in your handoff and finish". No step declares the package.
- **Found:** The manifest fallback exists in exactly one place, and this flow does not reach it. `context/agents/integration-v2/install.md:53` tells the agent to "add the PostHog package to the manifest by hand at a known-good version, so the dependency is still declared for the later steps and for the user's own install". That text sits inline in the default flow's own prompt, not in the shared skill, so the new prompt points at a fallback that never existed.
- **Found:** The next task edits code on the assumption that the SDK resolves. `context/agents/error-tracking/capture-exceptions.md:29` states "The SDK is installed and initialized ... build on that, do not re-check it", and that task holds `Write` and `Edit`. `init.md` also writes an init that constructs the client.
- **Impact:** On the environment failures that `context/agents/integration-v2/install.md:38` itself lists — a peer conflict the project already had, a broken lockfile, a package-manager provisioning failure — the run finishes with PostHog imports and an init in the source and no dependency in the manifest. The project stops building, and it built before the run. `install.md:9` already grants `Edit`, so the repair is one sentence of prompt text, not new tooling.
- **Impact:** The report does not reliably surface the failure as a follow-up either. The "What you still need to do" section that carries "Any other manual follow-up the handoffs carry" (`report.md:38`) is written only "When the run wired source-map upload" (`report.md:30`). On a readable-stack platform the upload subgraph is skipped, so that section is absent and the failed install has no guaranteed place in the user-facing output.
- **Priority:** Lowered to `should_fix`. The path needs an environment failure, so it does not hit every run, and the same wording already ships in `context/agents/replay-vision/install.md` on `main` — this PR copies an established pattern rather than creating a regression. Note also that only part of the suggestion applies: restoring the known-good manifest fallback fits the design, while finishing the task as failed fights the deliberate "do not spiral" rule in both the skill and `context/agents/integration-v2/install.md:32`.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Finish the install task as failed unless the package is declared. Alternatively, add a known-good manifest fallback before completion. Make `capture-exceptions` skip edits when the install or init handoff reports failure.
</potential_solution>

Comment on lines +57 to +59
of it. A platform with no autocapture at all, such as Go, is the other case:
there the capture boundary at the entry point is the mechanism, not a copy of
one.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One Go entry-point boundary cannot cover the process

should_fix

Issue description

Go has no process-wide exception boundary. recover only catches a panic in the same goroutine. An entry-point boundary misses HTTP handlers, worker goroutines, and returned errors. The Go installation reference instead captures errors with NewDefaultException.

Why we think it's a valid issue
  • Checked: context/agents/error-tracking/capture-exceptions.md in full, the skill it loads (context/skills/integration-v2/error-tracking-step/ — its description.md and the go variant at config.yaml:306-312), the repo's own Go rules in context/commandments.yaml, the Go example in context/skills/error-tracking-upload-source-maps/description.md:362-366, and the docs page the go variant hands the agent (https://posthog.com/docs/error-tracking/installation/go.md).
  • Found: The half of the sentence about missing autocapture is correct — the Go docs page describes no panic hook, no process-wide handler, and never mentions recover(), defer-based recovery, or HTTP middleware. The named mechanism is the part that fails: the page offers exactly two ways to capture, posthog.NewDefaultException(...) plus client.Enqueue(...), or posthog.NewSlogCaptureHandler(baseHandler, client, ...) wrapped in slog.New(...) for automatic capture at slog.LevelWarn and above. Neither one is "the capture boundary at the entry point".
  • Found: The repo's own Go guidance agrees with the docs and not with the prompt. context/commandments.yaml:91 lists only posthog.NewDefaultException(...) for direct captures and posthog.NewSlogCaptureHandler(...) for automatic capture. The only Go capture snippet elsewhere in the repo, context/skills/error-tracking-upload-source-maps/description.md:362-366, is an explicit client.Enqueue(posthog.NewDefaultException(...)) on a route, not a recover boundary.
  • Found: The prompt contradicts itself on this platform. capture-exceptions.md:20-21 tells the agent to "Follow the docs and the reference example for this one", and capture-exceptions.md:63-64 sets the success bar as capture "through the mechanism this SDK gives you rather than one you invented". Lines 57-59 name a Go mechanism the SDK does not give, so the agent that obeys those lines fails the file's own bar.
  • Found: The path is reachable. A go.mod project is a supported target of this flow — setup-error-tracking.md maps go.mod to the go uploader variant, and the capture task's skill carries a go variant with the Go error-tracking page as its docs.
  • Impact: A recover() at the process entry recovers only panics on its own goroutine. net/http runs each connection on its own goroutine and recovers handler panics itself, so a boundary in main() never sees them. On the most common Go target, a web service, the agent can finish, report a wired capture mechanism, and leave a build where no uncaught error reaches PostHog — a silent no-op, which is the one outcome the flow exists to prevent.
  • Impact: The mitigating factor is that the correct docs travel with the task, so a careful agent may follow NewSlogCaptureHandler or explicit NewDefaultException captures instead of the sentence. That keeps the defect below must_fix, but the sentence still steers against the docs and costs one clause to correct.
  • Note on the fix: Only part of the suggestion fits the file. Central capture through NewSlogCaptureHandler or an existing shared error path matches both the docs and the prompt's own "set it up in one place" rule (capture-exceptions.md:21). Adding recovery wrappers per worker goroutine pulls toward the hand-wrapping the same file forbids at capture-exceptions.md:66-67.
Suggested fix

Direct the task to use existing central boundaries. Add recovery middleware around HTTP requests. Recover inside worker wrappers. Capture returned errors at central handlers with NewDefaultException. Report any paths that have no shared boundary.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/capture-exceptions.md#L57-59

<issue_description>
Go has no process-wide exception boundary. `recover` only catches a panic in the same goroutine. An entry-point boundary misses HTTP handlers, worker goroutines, and returned errors. The Go installation reference instead captures errors with `NewDefaultException`.
</issue_description>

<issue_validation>
- **Checked:** `context/agents/error-tracking/capture-exceptions.md` in full, the skill it loads (`context/skills/integration-v2/error-tracking-step/` — its `description.md` and the `go` variant at `config.yaml:306-312`), the repo's own Go rules in `context/commandments.yaml`, the Go example in `context/skills/error-tracking-upload-source-maps/description.md:362-366`, and the docs page the `go` variant hands the agent (`https://posthog.com/docs/error-tracking/installation/go.md`).
- **Found:** The half of the sentence about missing autocapture is correct — the Go docs page describes no panic hook, no process-wide handler, and never mentions `recover()`, `defer`-based recovery, or HTTP middleware. The named mechanism is the part that fails: the page offers exactly two ways to capture, `posthog.NewDefaultException(...)` plus `client.Enqueue(...)`, or `posthog.NewSlogCaptureHandler(baseHandler, client, ...)` wrapped in `slog.New(...)` for automatic capture at `slog.LevelWarn` and above. Neither one is "the capture boundary at the entry point".
- **Found:** The repo's own Go guidance agrees with the docs and not with the prompt. `context/commandments.yaml:91` lists only `posthog.NewDefaultException(...)` for direct captures and `posthog.NewSlogCaptureHandler(...)` for automatic capture. The only Go capture snippet elsewhere in the repo, `context/skills/error-tracking-upload-source-maps/description.md:362-366`, is an explicit `client.Enqueue(posthog.NewDefaultException(...))` on a route, not a recover boundary.
- **Found:** The prompt contradicts itself on this platform. `capture-exceptions.md:20-21` tells the agent to "Follow the docs and the reference example for this one", and `capture-exceptions.md:63-64` sets the success bar as capture "through the mechanism this SDK gives you rather than one you invented". Lines 57-59 name a Go mechanism the SDK does not give, so the agent that obeys those lines fails the file's own bar.
- **Found:** The path is reachable. A `go.mod` project is a supported target of this flow — `setup-error-tracking.md` maps `go.mod` to the `go` uploader variant, and the capture task's skill carries a `go` variant with the Go error-tracking page as its docs.
- **Impact:** A `recover()` at the process entry recovers only panics on its own goroutine. `net/http` runs each connection on its own goroutine and recovers handler panics itself, so a boundary in `main()` never sees them. On the most common Go target, a web service, the agent can finish, report a wired capture mechanism, and leave a build where no uncaught error reaches PostHog — a silent no-op, which is the one outcome the flow exists to prevent.
- **Impact:** The mitigating factor is that the correct docs travel with the task, so a careful agent may follow `NewSlogCaptureHandler` or explicit `NewDefaultException` captures instead of the sentence. That keeps the defect below `must_fix`, but the sentence still steers against the docs and costs one clause to correct.
- **Note on the fix:** Only part of the suggestion fits the file. Central capture through `NewSlogCaptureHandler` or an existing shared error path matches both the docs and the prompt's own "set it up in one place" rule (`capture-exceptions.md:21`). Adding recovery wrappers per worker goroutine pulls toward the hand-wrapping the same file forbids at `capture-exceptions.md:66-67`.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Direct the task to use existing central boundaries. Add recovery middleware around HTTP requests. Recover inside worker wrappers. Capture returned errors at central handlers with `NewDefaultException`. Report any paths that have no shared boundary.
</potential_solution>

Comment on lines +64 to +80
## The runtime boundary has names too

Upload credentials are a build-time concern. The app also reads its own
variables at **run** time, and the step that starts it is another place a name
has to match — a `docker run -e`, a compose file, a systemd unit, a platform's
environment settings. That step was written before PostHog existed here, so the
name it passes is whatever the project used back then. If `init` settled on a
different one, the deployed process starts with an undefined key.

Nothing fails loudly. In production the init guard returns quietly instead of
throwing, so the process boots, serves traffic, and reports nothing — the same
silent shape as a mismatched upload variable, one boundary later.

So read the names the app's own source reads, and make the start step pass
exactly those. Rename the deploy step's variable, not the code: the code is the
half that already exists. Name any secret the user must create under its new
name in your handoff.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client SDK variables are wired at the wrong lifecycle stage

should_fix bug

Issue description

The task treats every SDK token and host as runtime values. Vite and NEXT_PUBLIC_* values are replaced during the production build. Runtime environment settings cannot update these values after the bundle exists. CI can upload maps but ship an SDK with an undefined token. Error capture then fails without an error.

Why we think it's a valid issue
  • Checked: every prompt in context/agents/error-tracking/ for who wires the app's own PostHog token into a container or CI build. I grepped the flow for docker, runtime, and build time. Only wire-ci.md names a container build; configure.md:47-55 covers a local run script, and init.md:52-62 covers .env loading on the developer machine.
  • Found: wire-ci.md classifies the app's own variables as runtime only. Line 65 says the app reads them "at run time". Lines 67-68 list the places to fix as "a docker run -e, a compose file, a systemd unit, a platform's environment settings". Line 77-78 says "make the start step pass exactly those". The success criterion at lines 86-87 repeats it: "the step that starts the app passes the variable names the app's own source reads".
  • Found: the build-time half is never named. wire-ci.md:56-57 scopes the ARG-plus-ENV shape to "the git variables and the non-secret settings only", and in context those settings are the uploader's project id and host, not the SDK's public token. No line in the file tells the agent to pass the SDK token into the build stage.
  • Found: most variants this flow reaches are client bundles. The seed's uploader table (context/agents/error-tracking/setup-error-tracking.md) selects nextjs, nuxt, angular, vite, webpack, rollup, react, and web. For these the bundler replaces the token expression during the production build, so a value supplied at start time cannot reach the shipped bundle.
  • Found: one platform escapes the gap by accident. init.md:80-88 writes the real public token as a literal into the committed src/environments/* files for Angular on the stock builder, so that build needs no environment.
  • Impact: for a Vite or Next project with a Dockerfile or CI build, the agent follows this section, wires the token into the start step, and the client bundle still builds with an undefined token. The env file is gitignored, so the build stage has no value to inline. The result is the exact silent failure the section warns about at lines 73-75: the app boots, serves traffic, and reports nothing. A static SPA served by a web server has no start step at all, so the guidance is a no-op there. The gap sits inside this task's lane, because wire-ci already edits the build stage and the CI job where the token belongs.
  • Priority: lowered to should_fix. The task's primary job, carrying the upload credentials and the release identity into the build, still works. The failure needs a specific shape: a client-bundled project with a container or CI build whose existing config does not already carry the token. Angular is covered by the literal token in init.md. The flow's local outcome is unaffected, so this is an incomplete new section rather than a break.
Suggested fix

Classify each SDK variable as build-time or runtime from its source expression. Pass client variables into the production build. Pass server variables into the start step. For Docker builds, add the public token and host to the build stage.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/wire-ci.md#L64-80

<issue_description>
The task treats every SDK token and host as runtime values. Vite and `NEXT_PUBLIC_*` values are replaced during the production build. Runtime environment settings cannot update these values after the bundle exists. CI can upload maps but ship an SDK with an undefined token. Error capture then fails without an error.
</issue_description>

<issue_validation>
- **Checked:** every prompt in `context/agents/error-tracking/` for who wires the app's own PostHog token into a container or CI build. I grepped the flow for `docker`, `runtime`, and `build time`. Only wire-ci.md names a container build; `configure.md:47-55` covers a local run script, and `init.md:52-62` covers `.env` loading on the developer machine.
- **Found:** wire-ci.md classifies the app's own variables as runtime only. Line 65 says the app reads them "at **run** time". Lines 67-68 list the places to fix as "a `docker run -e`, a compose file, a systemd unit, a platform's environment settings". Line 77-78 says "make the start step pass exactly those". The success criterion at lines 86-87 repeats it: "the step that starts the app passes the variable names the app's own source reads".
- **Found:** the build-time half is never named. wire-ci.md:56-57 scopes the `ARG`-plus-`ENV` shape to "the git variables and the non-secret settings only", and in context those settings are the uploader's project id and host, not the SDK's public token. No line in the file tells the agent to pass the SDK token into the build stage.
- **Found:** most variants this flow reaches are client bundles. The seed's uploader table (context/agents/error-tracking/setup-error-tracking.md) selects `nextjs`, `nuxt`, `angular`, `vite`, `webpack`, `rollup`, `react`, and `web`. For these the bundler replaces the token expression during the production build, so a value supplied at start time cannot reach the shipped bundle.
- **Found:** one platform escapes the gap by accident. init.md:80-88 writes the real public token as a literal into the committed `src/environments/*` files for Angular on the stock builder, so that build needs no environment.
- **Impact:** for a Vite or Next project with a Dockerfile or CI build, the agent follows this section, wires the token into the start step, and the client bundle still builds with an undefined token. The env file is gitignored, so the build stage has no value to inline. The result is the exact silent failure the section warns about at lines 73-75: the app boots, serves traffic, and reports nothing. A static SPA served by a web server has no start step at all, so the guidance is a no-op there. The gap sits inside this task's lane, because wire-ci already edits the build stage and the CI job where the token belongs.
- **Priority:** lowered to `should_fix`. The task's primary job, carrying the upload credentials and the release identity into the build, still works. The failure needs a specific shape: a client-bundled project with a container or CI build whose existing config does not already carry the token. Angular is covered by the literal token in init.md. The flow's local outcome is unaffected, so this is an incomplete new section rather than a break.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Classify each SDK variable as build-time or runtime from its source expression. Pass client variables into the production build. Pass server variables into the start step. For Docker builds, add the public token and host to the build stage.
</potential_solution>

Comment on lines +41 to +53
So forward the provider's git variables into the build the same way you forward
credentials, and declare each one as `ARG` **and** `ENV` — `ARG` alone is not
visible to the uploader's environment lookup. Your skill's "Associate the
release with a git commit" step lists the variables per provider. Read that
step even though it is not the CI step: this boundary is where it applies.

Forward them only where a provider actually sets them. A project with no
pipeline — an image built and run by hand — has nothing to inherit from, and
variables declared but never filled resolve to no release at all: it reads as
wired and still fails the build. There the identity has to be supplied outright,
both halves, from something the build itself holds — a build argument the
operator passes, or the manifest's own version — so name and version are always
present. Decide which case you are in by reading the repo, not by assuming a

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Release forwarding does not cover the stated CI providers

consider bug

Issue description

The prompt says the skill lists git variables for each provider. The skill only lists GitHub variables. The CLI reads GitHub and Vercel variables. Docker builds on other CI providers can forward variables that the CLI ignores. Error Tracking then loses the release association.

Why we think it's a valid issue
  • Checked: the step wire-ci.md points the agent at, and what that step actually contains.
  • Found: the statement is wrong about the reference. wire-ci.md:43-44 says "Your skill's "Associate the release with a git commit" step lists the variables per provider." That step runs from description.md:309 to description.md:323. It holds one tip, "Forward GitHub's git env vars into the Docker build", and two examples: "GitHub Actions → docker build", listing GITHUB_ACTIONS, GITHUB_SHA, GITHUB_REF_NAME, GITHUB_REPOSITORY and GITHUB_SERVER_URL, and "Inline CI build (no Docker)", which says GitHub Actions already sets them. One provider, not several.
  • Found: the skill's escape hatch for other providers does not transfer to this step. description.md:186 tells the agent the worked examples are "exemplars, not an allowlist" and to apply the same principle with its own knowledge of the provider. That works for credentials, because the variable names there are the project's own and any name works once both sides agree. Release detection is the opposite: the CLI decides which names it reads, so an invented name is not equivalent.
  • Found: non-GitHub providers are an anticipated shape in this flow. The skill's CI step carries a dedicated GitLab example at description.md:288-305, and wire-ci.md:22 tells the agent to wire credentials "whatever the CI provider".
  • Found: the real consequence is the one the skill states, not the one the prompt states. description.md:311 says that without the variables "the release ends up linked to nothing". wire-ci.md:38-39 instead claims the uploader "stops the build when it has only one" half. Nothing in the skill supports a build failure.
  • Found: part of the suggested remedy conflicts with the skill. description.md:311 says "you never touch the CLI invocation itself (it's usually baked into npm run build or a bundler plugin)", so adding explicit --release-name and --release-version flags is not a change this skill permits.
  • Impact: on a GitLab, CircleCI, Jenkins, or Bitbucket pipeline that builds a container, the agent is sent to a reference that answers only for GitHub. It forwards that provider's own git variables as ARG and ENV, the uploader does not read them, and the release association is lost. The Dockerfile keeps inert lines that read as wired. Source maps still upload and stack traces still resolve, so nothing announces the gap.
  • Priority: lowered to consider. The loss is confined to the release-to-commit association, which is metadata on an issue rather than symbolication. The claim that the CLI reads only GitHub and Vercel variables comes from outside this repo and I could not confirm it here, so the exact set of affected providers is unverified.
Suggested fix

Document exact mappings for each supported CI provider. For unsupported provider variables, pass explicit --release-name and --release-version values or matching plugin options. Add a fallback that verifies both release fields are available.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/wire-ci.md#L41-53

<issue_description>
The prompt says the skill lists git variables for each provider. The skill only lists GitHub variables. The CLI reads GitHub and Vercel variables. Docker builds on other CI providers can forward variables that the CLI ignores. Error Tracking then loses the release association.
</issue_description>

<issue_validation>
- **Checked:** the step wire-ci.md points the agent at, and what that step actually contains.
- **Found:** the statement is wrong about the reference. wire-ci.md:43-44 says "Your skill's "Associate the release with a git commit" step lists the variables per provider." That step runs from description.md:309 to description.md:323. It holds one tip, "Forward GitHub's git env vars into the Docker build", and two examples: "GitHub Actions → docker build", listing `GITHUB_ACTIONS`, `GITHUB_SHA`, `GITHUB_REF_NAME`, `GITHUB_REPOSITORY` and `GITHUB_SERVER_URL`, and "Inline CI build (no Docker)", which says GitHub Actions already sets them. One provider, not several.
- **Found:** the skill's escape hatch for other providers does not transfer to this step. description.md:186 tells the agent the worked examples are "exemplars, not an allowlist" and to apply the same principle with its own knowledge of the provider. That works for credentials, because the variable names there are the project's own and any name works once both sides agree. Release detection is the opposite: the CLI decides which names it reads, so an invented name is not equivalent.
- **Found:** non-GitHub providers are an anticipated shape in this flow. The skill's CI step carries a dedicated GitLab example at description.md:288-305, and wire-ci.md:22 tells the agent to wire credentials "whatever the CI provider".
- **Found:** the real consequence is the one the skill states, not the one the prompt states. description.md:311 says that without the variables "the release ends up linked to nothing". wire-ci.md:38-39 instead claims the uploader "stops the build when it has only one" half. Nothing in the skill supports a build failure.
- **Found:** part of the suggested remedy conflicts with the skill. description.md:311 says "you never touch the CLI invocation itself (it's usually baked into `npm run build` or a bundler plugin)", so adding explicit `--release-name` and `--release-version` flags is not a change this skill permits.
- **Impact:** on a GitLab, CircleCI, Jenkins, or Bitbucket pipeline that builds a container, the agent is sent to a reference that answers only for GitHub. It forwards that provider's own git variables as `ARG` and `ENV`, the uploader does not read them, and the release association is lost. The Dockerfile keeps inert lines that read as wired. Source maps still upload and stack traces still resolve, so nothing announces the gap.
- **Priority:** lowered to `consider`. The loss is confined to the release-to-commit association, which is metadata on an issue rather than symbolication. The claim that the CLI reads only GitHub and Vercel variables comes from outside this repo and I could not confirm it here, so the exact set of affected providers is unverified.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Document exact mappings for each supported CI provider. For unsupported provider variables, pass explicit `--release-name` and `--release-version` values or matching plugin options. Add a fallback that verifies both release fields are available.
</potential_solution>

Comment on lines +24 to +35
**1. Is PostHog already integrated?** Look for `posthog-js` or a server SDK in
the dependency manifests, or a `posthog.init(...)` / snippet in the source.
Check the project state for existing events if the repo is ambiguous.

Integrated means the init runs, not that the package is listed. An init point
is a pair: the call, and the key it is constructed from. A repo can carry the
call while the key it names is defined nowhere — then the client is built from
an empty string and captures nothing, however complete the manifest looks. So
when you find an init that reads a variable, look for that same name in the
repo's committed env template or its build config. Found, or the project state
shows real events arriving: integrated. Named nowhere, or you cannot tell:
**queue `init`**. It re-checks the pair itself and leaves a complete init

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check integration for every runtime surface

should_fix

Issue description

Fact 1 marks a project as integrated after it finds one valid init pair. This is incomplete for full-stack and multi-platform applications. A project can have a working browser init but no server SDK or server init. The seed then queues neither install nor init. capture-exceptions cannot add dependencies, so uncaught server errors never reach PostHog.

Why we think it's a valid issue
  • Checked: how fact 1 is phrased, what the two queueing rules test, whether the downstream tasks can cover a second runtime surface, and whether anything reports a surface the run left out.
  • Found: fact 1 is a single-pair test. setup-error-tracking.md:24-25 asks the planner to "Look for posthog-js or a server SDK in the dependency manifests, or a posthog.init(...) / snippet in the source", and :28-29 defines the result as one pair — "An init point is a pair: the call, and the key it is constructed from." The prompt names both SDK kinds but treats them as alternatives for one yes-or-no answer, never as a checklist.
  • Found: both queueing rules inherit that. setup-error-tracking.md:82 queues install "only when the SDK is missing from the manifest" (singular), and :83-84 queues init only "whenever fact 1 did not show a complete pair". A full-stack repo that declares posthog-js and has a working browser init satisfies both gates and gets neither task.
  • Found: the machinery to cover the server half exists and is simply never reached. install.md:22-23 already says to "Install the SDK the errors will report through (the server library too, if the app runs server-side code)", and the init skill already requires breadth: "An app that builds for several platforms needs an init point per platform it targets, not one shared init — a single SDK call often covers only some of them, and the rest stay uninstrumented while the build still succeeds" (context/skills/integration-v2/init/description.md:48-52).
  • Found: capture-exceptions cannot close the gap itself. Its grant is [Read, Write, Edit, Glob, Grep] with no Bash (capture-exceptions.md:10), its body says "Do not install dependencies" (:33), and its success criterion requires "You did not install anything" (:65). It is also told the SDK is ready and to "build on that, do not re-check it" (:29-31).
  • Found: one soft safety net exists. The runner injects a tool inventory into every task that ends: "Later tasks in this run hold tools you do not: when your task needs one, hand that work off in your handoff for the task that can do it, or note it for the final report" (renderToolInventory in PostHog/wizard src/lib/agent/agent-prompt-loader.ts). An agent that recognises the missing server library can carry it to the report as a follow-up.
  • Impact: a full-stack app whose PostHog was added client-side only — a common shape for Next.js, Nuxt, SvelteKit and Remix projects that adopted posthog-js for analytics — finishes the run with browser exceptions captured and server-side uncaught errors captured nowhere. No task installs the server library, and the seed passes no surface information to capture-exceptions (setup-error-tracking.md:85-86).
  • Priority: lowered from must_fix. The client half does work, nothing that existed breaks, and the run has a defined way to surface what it could not do. The gap is a coarse gate in front of tasks that already know how to handle several surfaces, which puts it level with the other queueing-gate findings rather than above them.
Suggested fix

Map every runtime surface before you set fact 1. Require an installed SDK and a valid init pair for each target. Queue install or init when any required surface is incomplete. Pass the missing surfaces in task inputs.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/setup-error-tracking.md#L24-35
@context/agents/error-tracking/setup-error-tracking.md#L82-86

<issue_description>
Fact 1 marks a project as integrated after it finds one valid init pair. This is incomplete for full-stack and multi-platform applications. A project can have a working browser init but no server SDK or server init. The seed then queues neither `install` nor `init`. `capture-exceptions` cannot add dependencies, so uncaught server errors never reach PostHog.
</issue_description>

<issue_validation>
- **Checked:** how fact 1 is phrased, what the two queueing rules test, whether the downstream tasks can cover a second runtime surface, and whether anything reports a surface the run left out.
- **Found:** fact 1 is a single-pair test. `setup-error-tracking.md:24-25` asks the planner to "Look for `posthog-js` **or** a server SDK in the dependency manifests, or a `posthog.init(...)` / snippet in the source", and `:28-29` defines the result as one pair — "An init point is a pair: the call, and the key it is constructed from." The prompt names both SDK kinds but treats them as alternatives for one yes-or-no answer, never as a checklist.
- **Found:** both queueing rules inherit that. `setup-error-tracking.md:82` queues `install` "only when the SDK is missing from the manifest" (singular), and `:83-84` queues `init` only "whenever fact 1 did not show a complete pair". A full-stack repo that declares `posthog-js` and has a working browser init satisfies both gates and gets neither task.
- **Found:** the machinery to cover the server half exists and is simply never reached. `install.md:22-23` already says to "Install the SDK the errors will report through (the server library too, if the app runs server-side code)", and the init skill already requires breadth: "An app that builds for several platforms needs an init point per platform it targets, not one shared init — a single SDK call often covers only some of them, and the rest stay uninstrumented while the build still succeeds" (`context/skills/integration-v2/init/description.md:48-52`).
- **Found:** `capture-exceptions` cannot close the gap itself. Its grant is `[Read, Write, Edit, Glob, Grep]` with no `Bash` (`capture-exceptions.md:10`), its body says "Do not install dependencies" (`:33`), and its success criterion requires "You did not install anything" (`:65`). It is also told the SDK is ready and to "build on that, do not re-check it" (`:29-31`).
- **Found:** one soft safety net exists. The runner injects a tool inventory into every task that ends: "Later tasks in this run hold tools you do not: when your task needs one, hand that work off in your handoff for the task that can do it, or note it for the final report" (`renderToolInventory` in PostHog/wizard `src/lib/agent/agent-prompt-loader.ts`). An agent that recognises the missing server library can carry it to the report as a follow-up.
- **Impact:** a full-stack app whose PostHog was added client-side only — a common shape for Next.js, Nuxt, SvelteKit and Remix projects that adopted `posthog-js` for analytics — finishes the run with browser exceptions captured and server-side uncaught errors captured nowhere. No task installs the server library, and the seed passes no surface information to `capture-exceptions` (`setup-error-tracking.md:85-86`).
- **Priority:** lowered from `must_fix`. The client half does work, nothing that existed breaks, and the run has a defined way to surface what it could not do. The gap is a coarse gate in front of tasks that already know how to handle several surfaces, which puts it level with the other queueing-gate findings rather than above them.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Map every runtime surface before you set fact 1. Require an installed SDK and a valid init pair for each target. Queue `install` or `init` when any required surface is incomplete. Pass the missing surfaces in task inputs.
</potential_solution>

Comment on lines +49 to +65
## Make the environment actually reachable

Writing the keys to `.env` is only half the job — something has to load that
file at runtime, or the app throws on boot and captures nothing.

Most frameworks do it for you: Next, Nuxt, Astro and SvelteKit auto-load `.env`,
and Vite auto-loads it for client code. Nothing to do there.

A plain Node backend does not — Express, Fastify, Hono, Koa, a raw `node:http`
server — and neither does a bare Rollup or webpack config. When your init point
reads `process.env` on one of those, wire the loading too, either way:

- install `dotenv` with the project's own package manager (detect it from the
lockfile) and import it above the PostHog init — `require('dotenv').config()`,
or `import 'dotenv/config'` for ESM; or
- add `--env-file=.env` to the `start` and `dev` scripts, when the project is on
Node 20.6+ and would rather not take a new dependency.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-JavaScript runtimes never load the env file

should_fix bug

Issue description

These instructions explain .env loading only for JavaScript runtimes. Java and Elixir read process variables, but neither runtime loads .env by default. These variants skip configure, so the flow can report success with an empty token.

Why we think it's a valid issue
  • Checked: the whole "Make the environment actually reachable" section (init.md:49-89) and the success criterion it feeds (init.md:106-114), the variant matrix the init skill inherits (context/skills/integration/config.yaml), the seed's platform routing (setup-error-tracking.md:62), and the per-language rules that reach this task through posthog-best-practices (init.md:9, context/commandments.yaml).
  • Found: The section states a runtime-agnostic rule and then covers one ecosystem. init.md:51-52 says "something has to load that file at runtime, or the app throws on boot and captures nothing", after which every named platform is JavaScript: Next, Nuxt, Astro, SvelteKit and Vite at :54-55, Express/Fastify/Hono/Koa/node:http plus bare Rollup and webpack at :57-65, and Angular at :72-89. Python, Ruby, Java, Elixir and .NET get no rule.
  • Found: Those runtimes are first-class targets of this task. context/skills/integration/config.yaml ships django, flask, fastapi, python, ruby, ruby-on-rails, java (:250, "Java (Spring Boot)"), elixir, php and laravel, and setup-error-tracking.md:62 routes them away only from the upload subgraph, never from init.
  • Found: The two halves the agent is given do not meet on those runtimes. init.md:108-110 requires "keys in the env file and confirmed there with check_env_keys, never hardcoded", while the language rules tell the code to read the process environment — commandments.yaml:144 "Initialize PostHog in AppConfig.ready() with api_key and host from environment variables", :213 for Rails, :124 for Elixir "read secrets from environment or runtime config". Nothing populates that environment from the .env the wizard just wrote, and commandments.yaml has no java section at all.
  • Found: No later task closes it, though not for the reason given. configure owns build-config edits for source-map upload and is queued only when an uploader variant matched, so it would not wire runtime env loading even when present — the gap is that no task in any branch owns this for non-JS runtimes.
  • Found: The claim does not hold uniformly, which bounds the impact. Laravel loads .env natively and commandments.yaml:179-180 has it read through env(), so that chain works. Flask's CLI loads .env when python-dotenv is installed, and many Python and Ruby projects already carry python-dotenv, django-environ or dotenv-rails, in which case the written keys resolve unchanged.
  • Found: The file also carries a correct fallback that generalises, at init.md:85-86: "do not wire a lookup unless you have opened the thing it reads from and seen your key defined there." Applied to a Django or Phoenix project with no loader, that rule stops the broken lookup — it is simply never connected to these runtimes.
  • Impact: On a Django, Spring Boot or Phoenix repo with no existing loader, the run writes the token to .env, writes code reading the process environment, and finishes. The client is then constructed from an empty or missing value and captures nothing, which is the quiet failure init.md:45-47 exists to prevent — or the lookup raises and the app fails to boot.
  • Priority: Lowered to should_fix. The prompt is silent for these runtimes rather than wrong about them, several of the platforms named in the finding do load .env or commonly already have a loader, and init.md:85-86 plus the framework docs give a correct path when applied — so the failure is likely on a subset of these projects, not all of them.
Suggested fix

Add loader rules for every supported server runtime. Update the existing launch path or framework configuration to load the selected env file.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/init.md#L49-65
@context/agents/error-tracking/init.md#L106-112

<issue_description>
These instructions explain `.env` loading only for JavaScript runtimes. Java and Elixir read process variables, but neither runtime loads `.env` by default. These variants skip `configure`, so the flow can report success with an empty token.
</issue_description>

<issue_validation>
- **Checked:** the whole "Make the environment actually reachable" section (`init.md:49-89`) and the success criterion it feeds (`init.md:106-114`), the variant matrix the init skill inherits (`context/skills/integration/config.yaml`), the seed's platform routing (`setup-error-tracking.md:62`), and the per-language rules that reach this task through `posthog-best-practices` (`init.md:9`, `context/commandments.yaml`).
- **Found:** The section states a runtime-agnostic rule and then covers one ecosystem. `init.md:51-52` says "something has to load that file at runtime, or the app throws on boot and captures nothing", after which every named platform is JavaScript: Next, Nuxt, Astro, SvelteKit and Vite at `:54-55`, Express/Fastify/Hono/Koa/`node:http` plus bare Rollup and webpack at `:57-65`, and Angular at `:72-89`. Python, Ruby, Java, Elixir and .NET get no rule.
- **Found:** Those runtimes are first-class targets of this task. `context/skills/integration/config.yaml` ships `django`, `flask`, `fastapi`, `python`, `ruby`, `ruby-on-rails`, `java` (`:250`, "Java (Spring Boot)"), `elixir`, `php` and `laravel`, and `setup-error-tracking.md:62` routes them away only from the *upload* subgraph, never from `init`.
- **Found:** The two halves the agent is given do not meet on those runtimes. `init.md:108-110` requires "keys in the env file and confirmed there with `check_env_keys`, never hardcoded", while the language rules tell the code to read the process environment — `commandments.yaml:144` "Initialize PostHog in AppConfig.ready() with api_key and host from environment variables", `:213` for Rails, `:124` for Elixir "read secrets from environment or runtime config". Nothing populates that environment from the `.env` the wizard just wrote, and `commandments.yaml` has no `java` section at all.
- **Found:** No later task closes it, though not for the reason given. `configure` owns build-config edits for source-map upload and is queued only when an uploader variant matched, so it would not wire runtime env loading even when present — the gap is that no task in any branch owns this for non-JS runtimes.
- **Found:** The claim does not hold uniformly, which bounds the impact. Laravel loads `.env` natively and `commandments.yaml:179-180` has it read through `env()`, so that chain works. Flask's CLI loads `.env` when python-dotenv is installed, and many Python and Ruby projects already carry `python-dotenv`, `django-environ` or `dotenv-rails`, in which case the written keys resolve unchanged.
- **Found:** The file also carries a correct fallback that generalises, at `init.md:85-86`: "do not wire a lookup unless you have opened the thing it reads from and seen your key defined there." Applied to a Django or Phoenix project with no loader, that rule stops the broken lookup — it is simply never connected to these runtimes.
- **Impact:** On a Django, Spring Boot or Phoenix repo with no existing loader, the run writes the token to `.env`, writes code reading the process environment, and finishes. The client is then constructed from an empty or missing value and captures nothing, which is the quiet failure `init.md:45-47` exists to prevent — or the lookup raises and the app fails to boot.
- **Priority:** Lowered to `should_fix`. The prompt is silent for these runtimes rather than wrong about them, several of the platforms named in the finding do load `.env` or commonly already have a loader, and `init.md:85-86` plus the framework docs give a correct path when applied — so the failure is likely on a subset of these projects, not all of them.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Add loader rules for every supported server runtime. Update the existing launch path or framework configuration to load the selected env file.
</potential_solution>

Comment on lines +85 to +89
So do not wire a lookup unless you have opened the thing it reads from and seen
your key defined there. When nothing populates it, write the real public project
token as a literal in the committed `src/environments/*` files. This is the
skill's "no valid environment to read from" case, and the public token is
publishable — it ships inside the browser bundle either way.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Angular fallback omits the project host

should_fix bug

Issue description

The fallback writes only the project token. Angular initialization also passes environment.posthogHost to api_host. An undefined or US-default host sends EU and self-hosted events to the wrong deployment.

Why we think it's a valid issue
  • Checked: every mention of configuration values in context/agents/error-tracking/init.md (the env-var path at :49-70, the Angular fallback at :72-89, the type-check rule at :91-104, the success criterion at :106-114), the skill this task loads (context/skills/integration-v2/init/description.md), the Angular rules that reach it via posthog-best-practices (context/commandments.yaml:310-314), and the Angular page the variant supplies (https://posthog.com/docs/error-tracking/installation/angular.md).
  • Found: The host is absent from this file end to end. init.md:86-87 says to "write the real public project token as a literal in the committed src/environments/* files", init.md:111-112 checks only that "the token is a literal rather than a lookup into something that never defines it", and no line in the file names the API host, the region, or a <UI_HOST> substitution — unlike credentials.md and report.md, which tell the agent to replace <UI_HOST> from project context.
  • Found: The Angular init genuinely needs two values. The docs page initializes with posthog.init(environment.posthogKey, { api_host: environment.posthogHost, defaults: '2026-05-30' }), reading both properties from src/environments/environment.ts — the same file the fallback tells the agent to write. commandments.yaml:314 says only "Configure PostHog credentials in src/environments/environment.ts files" and names neither value.
  • Found: The requirement reaches the agent only through a path that does not cover this case. The init skill's §Environment variables lists both "the public project token" and "the PostHog host", but scopes them to values set "through the wizard tools (set_env_values)". Its own fallback sentence — "embed the real public token in the config the build ships" — omits the host exactly as init.md does, so on the no-environment branch nothing in context asks for it.
  • Found: Angular's type checking bounds the failure. Angular projects are TypeScript, and init.md:93-96 requires the init to compile by construction, so an environment.posthogHost reference with no matching property fails the build rather than shipping. The reachable failure is therefore a wrong host value — an empty string, or a hardcoded US address on a non-US project — not a missing property.
  • Impact: posthog-js falls back to the US ingestion host when api_host is empty or absent, so an EU-cloud or self-hosted project initialised this way posts events with a token that region does not know. Nothing throws, the app runs, and Error Tracking stays empty — the same quiet failure init.md:45-47 warns about, on the stock @angular/build builder this file itself calls "the common one" (init.md:73). A US-cloud project is unaffected because the default matches.
  • Priority: Lowered to should_fix. The docs snippet and the compile-by-construction rule force both properties to exist, so this needs the agent to source a wrong host value rather than skip it, and the consequence lands only on non-US-cloud projects. The repair is still small — name the host beside the token in init.md:86-87 and require both at :111-112.
Suggested fix

Write both the project token and client API host into each active Angular environment file. Require both values in the success criteria.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/init.md#L85-89
@context/agents/error-tracking/init.md#L108-112

<issue_description>
The fallback writes only the project token. Angular initialization also passes `environment.posthogHost` to `api_host`. An undefined or US-default host sends EU and self-hosted events to the wrong deployment.
</issue_description>

<issue_validation>
- **Checked:** every mention of configuration values in `context/agents/error-tracking/init.md` (the env-var path at `:49-70`, the Angular fallback at `:72-89`, the type-check rule at `:91-104`, the success criterion at `:106-114`), the skill this task loads (`context/skills/integration-v2/init/description.md`), the Angular rules that reach it via `posthog-best-practices` (`context/commandments.yaml:310-314`), and the Angular page the variant supplies (`https://posthog.com/docs/error-tracking/installation/angular.md`).
- **Found:** The host is absent from this file end to end. `init.md:86-87` says to "write the real public project token as a literal in the committed `src/environments/*` files", `init.md:111-112` checks only that "the token is a literal rather than a lookup into something that never defines it", and no line in the file names the API host, the region, or a `<UI_HOST>` substitution — unlike `credentials.md` and `report.md`, which tell the agent to replace `<UI_HOST>` from project context.
- **Found:** The Angular init genuinely needs two values. The docs page initializes with `posthog.init(environment.posthogKey, { api_host: environment.posthogHost, defaults: '2026-05-30' })`, reading both properties from `src/environments/environment.ts` — the same file the fallback tells the agent to write. `commandments.yaml:314` says only "Configure PostHog credentials in src/environments/environment.ts files" and names neither value.
- **Found:** The requirement reaches the agent only through a path that does not cover this case. The init skill's §Environment variables lists both "the public project token" and "the PostHog host", but scopes them to values set "through the wizard tools (`set_env_values`)". Its own fallback sentence — "embed the real public token in the config the build ships" — omits the host exactly as `init.md` does, so on the no-environment branch nothing in context asks for it.
- **Found:** Angular's type checking bounds the failure. Angular projects are TypeScript, and `init.md:93-96` requires the init to compile by construction, so an `environment.posthogHost` reference with no matching property fails the build rather than shipping. The reachable failure is therefore a *wrong* host value — an empty string, or a hardcoded US address on a non-US project — not a missing property.
- **Impact:** `posthog-js` falls back to the US ingestion host when `api_host` is empty or absent, so an EU-cloud or self-hosted project initialised this way posts events with a token that region does not know. Nothing throws, the app runs, and Error Tracking stays empty — the same quiet failure `init.md:45-47` warns about, on the stock `@angular/build` builder this file itself calls "the common one" (`init.md:73`). A US-cloud project is unaffected because the default matches.
- **Priority:** Lowered to `should_fix`. The docs snippet and the compile-by-construction rule force both properties to exist, so this needs the agent to source a wrong host value rather than skip it, and the consequence lands only on non-US-cloud projects. The repair is still small — name the host beside the token in `init.md:86-87` and require both at `:111-112`.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Write both the project token and client API host into each active Angular environment file. Require both values in the success criteria.
</potential_solution>

Comment on lines +26 to +31
1. Ask with `wizard_ask`, exactly:
`{ id: "api-key", prompt: "Paste your PostHog personal API key below.\n\nDon't have one yet? Create one here:\n<SETTINGS_URL>\n\nWhen creating the key, choose the 'Source map upload' preset, then come back and paste it here.", kind: "text", sensitive: true }`
You receive `{ secretRef: "secret:..." }` — a vaulted reference, never the raw
value. If `wizard_ask` is unavailable (non-interactive run), report this task
with status `not needed` and say in your handoff that the user must create
the key and set the variables themselves; do not block.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handle a canceled API-key question

should_fix bug

Issue description

A canceled or timed-out wizard_ask field returns "__cancelled__". These instructions handle only an unavailable tool. Cancellation can reach set_env_values as an invalid reference or literal value. The task then fails or stores an unusable API key.

Why we think it's a valid issue
  • Checked: how this repo's other wizard_ask consumers treat a cancelled prompt, what branches credentials.md provides, and what reads the resulting handoff.
  • Found: cancellation is a documented, first-class outcome in this repo, not an edge case. context/skills/data-warehouse-source/description.md:34 states "A cancelled or timed-out wizard_ask does not count against the per-run cap. Treat a cancelled ask as 'the user declined' for that source", and context/skills/integration-v2/warehouse/description.md:74 repeats it. A test pins that sentence (scripts/lib/tests/data-warehouse-source-skill.test.js:60).
  • Found: the same repo already defines the correct response. context/skills/data-warehouse-source/description.md:123 says to report not needed and "Use this when the user cancelled or declined the prompts".
  • Found: credentials.md has no such branch. credentials.md:29-31 covers one case only: "If wizard_ask is unavailable (non-interactive run), report this task with status not needed". A user who closes the modal, or who never returns, is a different case, and step 3 at credentials.md:36-41 then still says to call set_env_values with the secretRef.
  • Found: the trigger is ordinary, not rare. The prompt text at credentials.md:27 sends the user out of the terminal to create a personal API key in PostHog settings and come back. Abandoning or dismissing that prompt is a normal user action.
  • Found: a downstream task reads this handoff and branches on it. report.md:34 tells the user to create a personal API key and says to "skip when the credentials handoff says the key is already written". An ambiguous handoff after a cancelled prompt therefore drops the instruction, so the user is never told to create the key, uploads never run, and the report claims the setup is complete.
  • Found: two specifics in the suggested fix are not verifiable here. The string __cancelled__ appears nowhere in this repo, and no file mentions a notNeededReason field. The wizard's tool contract lives in another repo that I cannot read from here, so the sentinel and that field are unconfirmed.
  • Impact: with no branch, the agent improvises. The benign outcome is a task reported incomplete with a vague handoff, which already misleads the report. The harmful outcome is a call to set_env_values with a reference that resolves to nothing, which writes an unusable value under the API-key name. A later run cannot detect that, because credentials.md:32-35 uses check_env_keys, which reports presence only and never a value.
Suggested fix

Check the api-key answer for "__cancelled__". Report not needed with notNeededReason: "user_declined", and do not call set_env_values.

Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/credentials.md#L26-31

<issue_description>
A canceled or timed-out `wizard_ask` field returns `"__cancelled__"`. These instructions handle only an unavailable tool. Cancellation can reach `set_env_values` as an invalid reference or literal value. The task then fails or stores an unusable API key.
</issue_description>

<issue_validation>
- **Checked:** how this repo's other `wizard_ask` consumers treat a cancelled prompt, what branches credentials.md provides, and what reads the resulting handoff.
- **Found:** cancellation is a documented, first-class outcome in this repo, not an edge case. context/skills/data-warehouse-source/description.md:34 states "A cancelled or timed-out `wizard_ask` does **not** count against the per-run cap. Treat a cancelled ask as 'the user declined' for that source", and context/skills/integration-v2/warehouse/description.md:74 repeats it. A test pins that sentence (scripts/lib/tests/data-warehouse-source-skill.test.js:60).
- **Found:** the same repo already defines the correct response. context/skills/data-warehouse-source/description.md:123 says to report `not needed` and "Use this when the user cancelled or declined the prompts".
- **Found:** credentials.md has no such branch. credentials.md:29-31 covers one case only: "If `wizard_ask` is unavailable (non-interactive run), report this task with status `not needed`". A user who closes the modal, or who never returns, is a different case, and step 3 at credentials.md:36-41 then still says to call `set_env_values` with the secretRef.
- **Found:** the trigger is ordinary, not rare. The prompt text at credentials.md:27 sends the user out of the terminal to create a personal API key in PostHog settings and come back. Abandoning or dismissing that prompt is a normal user action.
- **Found:** a downstream task reads this handoff and branches on it. report.md:34 tells the user to create a personal API key and says to "skip when the credentials handoff says the key is already written". An ambiguous handoff after a cancelled prompt therefore drops the instruction, so the user is never told to create the key, uploads never run, and the report claims the setup is complete.
- **Found:** two specifics in the suggested fix are not verifiable here. The string `__cancelled__` appears nowhere in this repo, and no file mentions a `notNeededReason` field. The wizard's tool contract lives in another repo that I cannot read from here, so the sentinel and that field are unconfirmed.
- **Impact:** with no branch, the agent improvises. The benign outcome is a task reported incomplete with a vague handoff, which already misleads the report. The harmful outcome is a call to `set_env_values` with a reference that resolves to nothing, which writes an unusable value under the API-key name. A later run cannot detect that, because credentials.md:32-35 uses `check_env_keys`, which reports presence only and never a value.
</issue_validation>

## Task
Investigate the issue and solve it

<potential_solution>
Check the `api-key` answer for `"__cancelled__"`. Report `not needed` with `notNeededReason: "user_declined"`, and do not call `set_env_values`.
</potential_solution>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant