feat(agents): add the error-tracking orchestrator flow - #377
ablaszkiewicz wants to merge 30 commits into
Conversation
Nine prompts under context/agents/error-tracking/ backing the wizard's new `error-tracking` command. The seed establishes whether PostHog is integrated and which uploader variant applies, then queues install/init (integration-v2 step-skills, like replay-vision) on uninstrumented repos, exception capture via integration-v2-error-tracking-step, the source-map subgraph (adapted from the error-tracking-upload-source-maps flow) where the platform ships minified bundles or stripped binaries, and a report that merges it all. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Test against a wizard branch:
Add Results will be posted here when complete. |
…ness install/init move terra→sol low, capture-exceptions terra low→sol medium. Pairs with the wizard binding change routing the program to the orchestrator on pi; seed stays terra and report stays luna, matching the metrics flow's pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…nager A configure run in a pnpm workspace copied the docs' npm install and npm hard-fails on workspace:* deps (EUNSUPPORTEDPROTOCOL); the agent retried npm three ways and never switched. New skill bullet: detect the manager from the lockfile, translate the docs' npm commands, and read EUNSUPPORTEDPROTOCOL as wrong-manager, never a flag to retry. The error-tracking flow's configure task now also mandates detect_package_manager before the first install. Verified: the pnpm monorepo fixture run that failed now completes 4/4, recovering to `pnpm add -D` on first contact with the error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…anning A seed run saw "PostHog already integrated" and enqueued only capture+report, never applying the uploader precedence to a tsc-built Node service — while an identical sibling fixture planned the full graph. The two facts are now declared independent, a compiled/bundled JS project is explicitly never "none", and the success criteria demand the plan state the uploader decision (or the readable-stack reason) explicitly. Verified: the fixture that under-planned now queues the full node-uploader graph and completes 5/5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
The integration init skill documents its public keys in .env.example; the credentials task wrote its upload variables only to the real env file, so the names never reached the committed template. New step 4: append the names with placeholders (never values) — for the next developer, and so a later run's check_env_keys reports the expectation. Scoped to the error-tracking flow's task prompt; the shared uploader skill is untouched, so the standalone upload-source-maps program keeps its current behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
Astro inlines scripts below its asset limit into the HTML, so a build can emit a .map with no .js beside it. posthog-cli then finds zero pairs, exits non-zero, and takes the whole build down with it. Treat Astro as a no-variant platform: install and exception capture still run, the build script is left untouched, and the report says why upload was skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…Node The shared integration-v2-init skill assumes the framework auto-loads .env, which is true for Next/Nuxt/Astro/SvelteKit/Vite but not for plain Node backends. On Express, Fastify, Hono, Koa and raw node:http the run wrote process.env reads and a .env file with nothing to bridge them, so every app died at module load on its own missing-variable guard. Teach the init task to wire the loading (dotenv, or --env-file) and to start the app once to confirm it boots. Also have configure check that some script actually runs a newly emitted bundle, since two apps built to dist/ while start still ran the source. Scoped to this flow's task prompts; the shared skill is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
…n Angular On the stock @angular/build builder nothing defines process.env or import.meta.env in the browser bundle, but the run wrote import.meta.env['NG_APP_POSTHOG_PROJECT_TOKEN'] into src/environments/*. Reading a property off undefined throws while the module evaluates, so <app-root> stayed empty and the app rendered a blank page. Name the mechanism's real source (@ngx-env/builder supplies import.meta.env and the NG_APP_ prefix) and require it to be an existing dependency before use. Otherwise fall back to the skill's existing "no valid environment to read from" case: embed the public project token in the committed environment files, which is where it ends up in the bundle anyway. Scoped to this flow's task prompt; the shared skill is untouched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PVXmLWB31CBujD2FJ2ogvY
The earlier rule stopped the run reaching for import.meta.env, but it then read env['NG_APP_POSTHOG_PROJECT_TOKEN'] from the project's generated src/environments/.env.ts, which only defines npm_package_version. The key came back undefined, and because a production build takes the guard's quiet return path the app rendered normally while PostHog never initialised — no requests at all. Name that second empty container, require the lookup's source to be opened and the key seen before wiring it, and make the task prove initialisation by serving the build and confirming a request reaches the PostHog host. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
Pointing `start` at dist/index.js is only safe if that bundle can execute. On koa-notes the esbuild command had no --format=esm while package.json sets "type": "module", so the emitted CommonJS died at boot with `ReferenceError: module is not defined in ES module scope` — a breakage that only appeared once `start` stopped running the source. Require the task to run the build and start the script once, and name the module-format mismatch as the usual cause. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…uild rule The new "run the build and start that script once" instruction sat directly above "Do not run the build", leaving the task with contradictory orders. Carve the single verification out of the prohibition. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The agent runs in a sandbox that cannot build or start the project, so rules telling init to boot the app and configure to run the built script were not executable. Keep the knowledge, drop the execution: the env loader and the module format must be correct by construction, checked by reading package.json's "type" and by opening the file a key lookup reads from before wiring it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
credentials and configure run in parallel: one writes the env file, the other writes the code that reads it, and neither could see the other's choice. On Nuxt 4 that split them — the skill's Nuxt reference documents variable names only for the posthog-cli path, so credentials wrote POSTHOG_CLI_* while configure wired the @posthog/nuxt module against POSTHOG_API_KEY. Nothing errored; the upload was simply skipped and every trace stayed minified. Give both tasks the same rule, in terms neither framework nor uploader specific: look for the other half of the contract before writing, adopt it verbatim if it exists, decide only when it does not, and always name the chosen set in full in the handoff. configure gets check_env_keys so it can read names without values. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…eads A repo can carry a PostHog init whose env variable was never defined — the call is there, the name it reads is nowhere, and the client is built from an empty string. The init task read that as "already initialised" and left it alone, so the app captured nothing while every other step looked clean. Init now confirms the name the code reads is present with check_env_keys before it leaves an existing init alone, and writes it when it is absent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The seed read fact 1 as "the SDK is in the manifest", so a repo carrying an init whose key variable was defined nowhere skipped install and init entirely. The client was then built from an empty string: the build stayed clean, every capture call returned, and no event ever arrived. Fact 1 is now about the pair — the call and the key it reads. When the name is defined nowhere, or the seed cannot tell, it queues init, which re-checks the pair and leaves a complete init alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
A Next.js run copied the uploader example into next.config.ts without the non-null assertion the reference shows. process.env lookups are string | undefined, the plugin option is string, and the build failed type checking — after the upload step had already run, so the log showed a successful chunk upload and the build still exited 1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…dary wire-ci was written entirely around credentials, and pointed at the skill's CI step while the git-context guidance sits in a different step the task never named. In a container build neither reaches the uploader: .git is in .dockerignore, and the CI's own variables stop at the docker build command. Reproduced against the nested composite action fixture: Release fields are incomplete and git info is unavailable (exit 1) Docker build args are not inherited from the environment, so this fails on a GitHub Actions runner exactly as it does locally — verified by building with GITHUB_SHA exported but not forwarded. The ARG-plus-ENV shape is scoped to the git variables and non-secret settings; the API key keeps its build-secret mount, which a first pass had demoted to a plain build arg. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…ndary wire-ci carried the upload credentials through the pipeline but nothing owned the variable the deploy step hands the running app. The step predates PostHog, so it passed the project's old name while init had settled on another, and the container booted with an undefined key. Reproduced by running the image exactly as the deploy step does: POSTHOG_PROJECT_TOKEN= undefined NODE_ENV= production exit=0 Nothing fails: in production the init guard returns rather than throwing, so the process serves traffic and reports nothing. The runtime boundary is now an explicit third place a name has to match, with the deploy step renamed to follow the code rather than the other way round. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The release-identity rule was written as if a pipeline always exists, so a project whose Dockerfile says it is built and run by hand got eight GITHUB_* build args nothing ever fills. Empty variables resolve to no release, and a hand-run docker build failed: Oops! Release fields are incomplete and git info is unavailable (exit 1) Reading as wired while failing the build is the worst of both. With no provider the identity is supplied outright instead — the re-run took --release-version from the manifest, and the hand-run build resolves cicd-docker-node-raw@0.1.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
…s on The Go run wrote credentials to .env, rewrote main.go to read them from the process env, added a Makefile run target that loads nothing, and then pointed the user at that target to verify. It prints its own "variable missing" guard and exits: build green, symbols uploaded, no event. The run-script rule now covers reaching the values as well as running the right artifact. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
error-tracking-step borrowed the integration group's variants, so the task that wires exception capture read the SDK library page. For Go and Rust those pages say nothing about panic hooks or log handlers, and the task invented the API: a capture_panics method on the wrong builder, and a slog handler with no distinct-ID resolver, which drops every capture without erroring. The matrix is unchanged - same 40 ids, frameworks and defaults, verified against the built menu before and after. Only docs_urls move: 39 variants now point at their error-tracking installation page, java being the one PostHog has none for. Where the page is a generic substitute (django to python, astro to web) the framework's own library page stays alongside. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The Rust page shows a global initializer and an explicit client side by side. The task took the wiring from one and the calls from the other, binding init_global's Result<(), Error> as if it were a client, so cargo build failed with three E0599s. It also wrote the build as a !-prefixed cargo alias, which is a git convention cargo has no support for. Six lines: follow one setup the whole way through, and put a multi-step build where the tool actually runs it. The re-run compiles, uploads symbols, and a panic resolves to rust_app::kaboom_rust2 -> read_rust_retry_budget2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
The capture step's Nuxt matrix marked `nuxt-3-6` as the default, so every Nuxt project — including Nuxt 4 — was handed the v3.0-3.6 installation page and wired PostHog by hand through a client plugin. Nuxt 3.7 shipped in September 2023; the `@posthog/nuxt` module is the documented path for everything since, and it initialises both the Vue and Nitro sides itself. Moves `default: true` to the `nuxt-4` variant, which already pointed at installation/nuxt-3-7.md. Two lines change in the built menu and nothing else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X5Kw3YahYz9RiFeJ8r2hxY
Exception autocapture already registers the global handlers — window.onerror and unhandledrejection in the browser, uncaughtException and unhandledRejection in Node, sys.excepthook in Python, the panic hook in Rust, the uncaught exception and signal handlers on iOS and Android. A second listener the task writes itself on the same event sends every error twice and drifts from the SDK's handling as the SDK changes. The capture task now turns autocapture on through the SDK's option and registers none of those handlers. Framework hooks stay in scope, because they see errors a framework swallows before any global handler fires: Express error middleware, Fastify setErrorHandler, Hono onError, Vue errorHandler, Angular ErrorHandler, SvelteKit handleError, a React error boundary, Next.js global-error. Go, which has no autocapture, keeps its entry-point boundary. The success check gains the same condition, so a reviewer can confirm it by reading the diff. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SEsZQQrYjXPVEhQuoSwE6
error-tracking-step declares its variants literally, and the orchestrator resolves it per framework with no fallback. A variant added to integration and not copied here would abort the default orchestrator flow for that framework. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SEsZQQrYjXPVEhQuoSwE6
- model_sdk: claude-sonnet-5 and claude-haiku-4-5, as #389 did for the other flows. The old ids are no longer valid in the wizard. - install and init run terra at low effort, like integration-v2 and replay-vision. wire-ci runs sol at medium. The gateway mint pins sol to medium. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013SEsZQQrYjXPVEhQuoSwE6
A React Native repo carries ios/Podfile, an .xcodeproj and Gradle files, so the seed's native-first precedence picked the ios uploader and wired no JavaScript source maps. react-native or expo in package.json now wins over every native marker. Verified with e2e runs on the react-native fixture (ios before, react-native after) and on react-native-expo. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z
In a type-checked project the init guard tested a separate variable and then passed the unchecked one to the SDK, so tsc stopped the build with TS2345 and no source maps uploaded. The init now passes the variable it checked, or keeps an existing ?? '' / ! as it is. Verified with e2e runs on node-webpack and node-raw: no type errors, and node-raw builds, uploads and starts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z
🦔 PostHog Review reviewed this pull requestFound 7 must fix, 20 should fix, 7 consider. Published 34 findings (view the review). |
With sourcemaps.enabled the webpack plugin adds its own SourceMapDevToolPlugin. A devtool that also emits maps makes webpack fail with "Conflict: Multiple assets emit different content to the same filename index.js.map", and nothing uploads. Verified with an e2e run on node-webpack: no devtool, the build passes, one chunk uploads, and the app starts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z
The flow relies on SDK autocapture and does not change team settings. Remove the products-enable step and posthog_exec from the report task. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z
|
PostHog Review alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏 |
| Make the credentials reach the production build wherever it actually runs, so | ||
| source maps upload on every deploy — not just on a local build. Install the | ||
| skill your task input names (`install_skill` with the `skillId`) and follow its | ||
| **"Set up CI for automatic uploads"** step — it owns tracing where the | ||
| production build runs and wiring the credentials through every layer, whatever | ||
| the CI provider. |
There was a problem hiding this comment.
Remove Docker instructions that expose personal API keys
Issue description
This task declares the installed skill authoritative. That skill still puts POSTHOG_CLI_API_KEY in Docker ARG, ENV, and build-args for multi-stage builds. Those instructions conflict with this prompt. Following them can store the personal key in build history, cache metadata, or provenance.
Why we think it's a valid issue
- Checked: which skill this task installs. The seed passes
skillId: error-tracking-upload-source-maps-<variant>(context/agents/error-tracking/setup-error-tracking.md, the "uploader skill id" paragraph), sowire-ciinstallscontext/skills/error-tracking-upload-source-maps. I read its "Set up CI for automatic uploads" step (description.md:170-307) and compared it with wire-ci.md:17-22 and wire-ci.md:56-62. - Found: the disagreement is real. description.md:189 tells the agent to put
ARG/ENVfor the key in the build stage of a multi-stage Dockerfile. description.md:203-206 shows that Dockerfile, and description.md:211 showsdocker build --build-arg POSTHOG_CLI_API_KEY=…. description.md:245 and description.md:265 putPOSTHOG_CLI_API_KEYin thebuild-args:input ofdocker/build-push-action. wire-ci.md:59-62 forbids exactly that shape. - Found: the skill is stage-aware, not uniformly unsafe. description.md:190 and description.md:222-225 require a BuildKit secret for the single-stage layout, which is the layout that bakes the key into the shipped image, and they tell the caller to use the
secrets:input in place ofbuild-args:there. - Found: this PR does not add the Docker instructions. Against
origin/mainthe PR changes 2 lines of description.md — a package-manager rule and a webpackdevtoolrule. EveryARG/ENV/build-argsexample is pre-existing. wire-ci.md is the first prompt in the flow that pushes toward build secrets, so the PR moves secret handling forward, not back. - Found: the override sits in the same file as the "follow the skill" sentence. wire-ci.md:56-62 names the key, forbids
ARGandENVfor it, and gives the reason. The flow uses this layering on purpose: configure.md ("even when they are not the ones your skill's example shows") and credentials.md ("even when they are not the ones your skill would suggest") tell the agent to deviate from the skill example in the same way. - Impact: the two documents still disagree for the most common layout, a multi-stage Dockerfile built by GitHub Actions. An agent can follow the concrete skill example and write the personal API key into
build-args:. The skill itself states at description.md:190 thatdocker historycan reveal build args. The specific prose rule in the task prompt usually wins over a reference example, so the leak is possible, not certain. - Priority: lowered to
consider. The inconsistency is verifiable and worth recording, but this PR does not create it, the flagged file already carries the explicit override, and the skill already mandates a BuildKit secret in the one layout that keeps the key in the shipped image. The suggested rewrite of every Docker example in a pre-existing skill file is larger than the defect.
Suggested fix
Update the skill to use BuildKit secrets for the personal API key in every Docker layout. Reserve ARG, ENV, and build-args for non-secret settings and git metadata. Rewrite all conflicting Docker examples.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/wire-ci.md#L17-22
@context/agents/error-tracking/wire-ci.md#L56-62
<issue_description>
This task declares the installed skill authoritative. That skill still puts `POSTHOG_CLI_API_KEY` in Docker `ARG`, `ENV`, and `build-args` for multi-stage builds. Those instructions conflict with this prompt. Following them can store the personal key in build history, cache metadata, or provenance.
</issue_description>
<issue_validation>
- **Checked:** which skill this task installs. The seed passes `skillId: error-tracking-upload-source-maps-<variant>` (context/agents/error-tracking/setup-error-tracking.md, the "uploader skill id" paragraph), so `wire-ci` installs `context/skills/error-tracking-upload-source-maps`. I read its "Set up CI for automatic uploads" step (description.md:170-307) and compared it with wire-ci.md:17-22 and wire-ci.md:56-62.
- **Found:** the disagreement is real. description.md:189 tells the agent to put `ARG`/`ENV` for the key in the build stage of a multi-stage Dockerfile. description.md:203-206 shows that Dockerfile, and description.md:211 shows `docker build --build-arg POSTHOG_CLI_API_KEY=…`. description.md:245 and description.md:265 put `POSTHOG_CLI_API_KEY` in the `build-args:` input of `docker/build-push-action`. wire-ci.md:59-62 forbids exactly that shape.
- **Found:** the skill is stage-aware, not uniformly unsafe. description.md:190 and description.md:222-225 require a BuildKit secret for the single-stage layout, which is the layout that bakes the key into the shipped image, and they tell the caller to use the `secrets:` input in place of `build-args:` there.
- **Found:** this PR does not add the Docker instructions. Against `origin/main` the PR changes 2 lines of description.md — a package-manager rule and a webpack `devtool` rule. Every `ARG`/`ENV`/`build-args` example is pre-existing. wire-ci.md is the first prompt in the flow that pushes toward build secrets, so the PR moves secret handling forward, not back.
- **Found:** the override sits in the same file as the "follow the skill" sentence. wire-ci.md:56-62 names the key, forbids `ARG` and `ENV` for it, and gives the reason. The flow uses this layering on purpose: configure.md ("even when they are not the ones your skill's example shows") and credentials.md ("even when they are not the ones your skill would suggest") tell the agent to deviate from the skill example in the same way.
- **Impact:** the two documents still disagree for the most common layout, a multi-stage Dockerfile built by GitHub Actions. An agent can follow the concrete skill example and write the personal API key into `build-args:`. The skill itself states at description.md:190 that `docker history` can reveal build args. The specific prose rule in the task prompt usually wins over a reference example, so the leak is possible, not certain.
- **Priority:** lowered to `consider`. The inconsistency is verifiable and worth recording, but this PR does not create it, the flagged file already carries the explicit override, and the skill already mandates a BuildKit secret in the one layout that keeps the key in the shipped image. The suggested rewrite of every Docker example in a pre-existing skill file is larger than the defect.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Update the skill to use BuildKit secrets for the personal API key in every Docker layout. Reserve `ARG`, `ENV`, and `build-args` for non-secret settings and git metadata. Rewrite all conflicting Docker examples.
</potential_solution>
| - id: kmp | ||
| display_name: Kotlin Multiplatform | ||
| tags: | ||
| - kmp | ||
| - kotlin | ||
| - multiplatform | ||
| - mobile | ||
| docs_urls: | ||
| - https://posthog.com/docs/error-tracking/installation/kmp.md |
There was a problem hiding this comment.
The KMP variant cannot resolve during orchestrator preflight
Issue description
The wizard detects KMP as framework kmp. Its resolver selects only entries with a matching framework. This variant omits that field, so preflight aborts before any task runs.
Why we think it's a valid issue
- Checked: the wizard resolver
resolveSkillVariantId(PostHog/wizardsrc/lib/agent/runner/sequence/orchestrator/orchestrator-runner.ts:169-180), the preflight loop that calls it (orchestrator-runner.ts:549-590), the menu loaderfetchSkillMenuandexpandBundleEntry(src/lib/wizard-tools/tools.ts:91-118), the wizard framework list (src/lib/constants.ts:74-105), and the manifest thatnode scripts/build.jswrites todist/skills/skill-menu.json. - Found: the resolver first tries an exact id match, then filters on
e.group === skillId && e.framework === framework.fetchSkillMenureplaces every bundle entry with its variants (tools.ts:117), so the bare idintegration-v2-error-tracking-stepis absent from the menu the resolver reads. The built entryintegration-v2-error-tracking-step-kmpcarries noframeworkkey. Resolution returnsundefined, the preflight adds the skill tomissingVariants, andwizardAbortstops the run with the message "Setup instructions for this project failed to download". - Found: the wizard detects KMP as framework
kmp(src/lib/constants.ts:91,src/frameworks/kmp/kmp-wizard-agent.ts), and the new command applies no platform allow-list — the program comment in feat(error-tracking): add the error-tracking program on the orchestrator flow wizard#1185 states "every detectable framework has an error-tracking-step variant".context/agents/error-tracking/capture-exceptions.md:9declaresskills: [integration-v2-error-tracking-step, posthog-best-practices], so the preflight does reach this variant. - Found: the gap is older than this PR.
context/skills/integration/config.yaml:299-305also omitsframeworkforkmp, andresolveVariantsFromcopies the field only when it exists (scripts/lib/skill-generator.js:257). Before this PR the step usedvariants_from: integrationand produced the same framework-less entry. The built manifest also showsintegration-kmp,integration-v2-install-kmp,integration-v2-init-kmpandintegration-v2-capture-kmpwithoutframework, so the shipped default flow aborts for KMP today. This widens the impact; it does not remove it. - Found: the new parity test compares
${v.id}:${v.framework}pairs (scripts/lib/tests/error-tracking-step-parity.test.js), so it matcheskmp:undefinedon both sides and lets the gap through. It also fails if you addframework: kmpto this file alone, which confirms that both configs must change together. - Impact: a Kotlin Multiplatform project that runs
wizard error-trackingstops at preflight with a misleading download error, before any task runs. The variant is unreachable although this file gives it a KMP error-tracking docs page. The other two framework-less variants,react-viteandphp, are safe because the wizard has no matching detected framework for them.
Suggested fix
Add framework: kmp here and to the canonical integration KMP variant. Add a resolver test that covers KMP across all composed step skills.
Prompt to fix with AI (copy-paste)
## Context
@context/skills/integration-v2/error-tracking-step/config.yaml#L343-351
<issue_description>
The wizard detects KMP as framework `kmp`. Its resolver selects only entries with a matching `framework`. This variant omits that field, so preflight aborts before any task runs.
</issue_description>
<issue_validation>
- **Checked:** the wizard resolver `resolveSkillVariantId` (PostHog/wizard `src/lib/agent/runner/sequence/orchestrator/orchestrator-runner.ts:169-180`), the preflight loop that calls it (`orchestrator-runner.ts:549-590`), the menu loader `fetchSkillMenu` and `expandBundleEntry` (`src/lib/wizard-tools/tools.ts:91-118`), the wizard framework list (`src/lib/constants.ts:74-105`), and the manifest that `node scripts/build.js` writes to `dist/skills/skill-menu.json`.
- **Found:** the resolver first tries an exact id match, then filters on `e.group === skillId && e.framework === framework`. `fetchSkillMenu` replaces every bundle entry with its variants (`tools.ts:117`), so the bare id `integration-v2-error-tracking-step` is absent from the menu the resolver reads. The built entry `integration-v2-error-tracking-step-kmp` carries no `framework` key. Resolution returns `undefined`, the preflight adds the skill to `missingVariants`, and `wizardAbort` stops the run with the message "Setup instructions for this project failed to download".
- **Found:** the wizard detects KMP as framework `kmp` (`src/lib/constants.ts:91`, `src/frameworks/kmp/kmp-wizard-agent.ts`), and the new command applies no platform allow-list — the program comment in PostHog/wizard#1185 states "every detectable framework has an error-tracking-step variant". `context/agents/error-tracking/capture-exceptions.md:9` declares `skills: [integration-v2-error-tracking-step, posthog-best-practices]`, so the preflight does reach this variant.
- **Found:** the gap is older than this PR. `context/skills/integration/config.yaml:299-305` also omits `framework` for `kmp`, and `resolveVariantsFrom` copies the field only when it exists (`scripts/lib/skill-generator.js:257`). Before this PR the step used `variants_from: integration` and produced the same framework-less entry. The built manifest also shows `integration-kmp`, `integration-v2-install-kmp`, `integration-v2-init-kmp` and `integration-v2-capture-kmp` without `framework`, so the shipped default flow aborts for KMP today. This widens the impact; it does not remove it.
- **Found:** the new parity test compares `${v.id}:${v.framework}` pairs (`scripts/lib/tests/error-tracking-step-parity.test.js`), so it matches `kmp:undefined` on both sides and lets the gap through. It also fails if you add `framework: kmp` to this file alone, which confirms that both configs must change together.
- **Impact:** a Kotlin Multiplatform project that runs `wizard error-tracking` stops at preflight with a misleading download error, before any task runs. The variant is unreachable although this file gives it a KMP error-tracking docs page. The other two framework-less variants, `react-vite` and `php`, are safe because the wizard has no matching detected framework for them.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Add `framework: kmp` here and to the canonical `integration` KMP variant. Add a resolver test that covers KMP across all composed step skills.
</potential_solution>
| The SDK is declared in the manifest at a real version — whether it already | ||
| was or you just installed it — or your handoff plainly says why the | ||
| environment stopped you. Your handoff names the manifest and the package, so | ||
| later steps import it under the name they will actually get. |
There was a problem hiding this comment.
An install failure still unblocks tasks that import the SDK
Issue description
These lines accept an environment failure as successful completion. The install skill can finish without declaring the SDK. The next task then assumes the SDK exists and adds imports for it. A registry or lockfile failure can therefore leave the project unable to build.
Why we think it's a valid issue
- Checked:
context/agents/error-tracking/install.mdin full, the skill it delegates to (context/skills/integration-v2/install/description.md), the sibling promptcontext/agents/integration-v2/install.md, the downstream promptscapture-exceptions.mdandinit.md, andreport.md. - Found: The prompt hands the failure path to its skill.
context/agents/error-tracking/install.md:19says the skill owns "what counts as an environment failure, and the fallback". The skill defines no fallback.context/skills/integration-v2/install/description.md:20ends that path with "do not spiral: report the exact failure and command in your handoff and finish". No step declares the package. - Found: The manifest fallback exists in exactly one place, and this flow does not reach it.
context/agents/integration-v2/install.md:53tells the agent to "add the PostHog package to the manifest by hand at a known-good version, so the dependency is still declared for the later steps and for the user's own install". That text sits inline in the default flow's own prompt, not in the shared skill, so the new prompt points at a fallback that never existed. - Found: The next task edits code on the assumption that the SDK resolves.
context/agents/error-tracking/capture-exceptions.md:29states "The SDK is installed and initialized ... build on that, do not re-check it", and that task holdsWriteandEdit.init.mdalso writes an init that constructs the client. - Impact: On the environment failures that
context/agents/integration-v2/install.md:38itself lists — a peer conflict the project already had, a broken lockfile, a package-manager provisioning failure — the run finishes with PostHog imports and an init in the source and no dependency in the manifest. The project stops building, and it built before the run.install.md:9already grantsEdit, so the repair is one sentence of prompt text, not new tooling. - Impact: The report does not reliably surface the failure as a follow-up either. The "What you still need to do" section that carries "Any other manual follow-up the handoffs carry" (
report.md:38) is written only "When the run wired source-map upload" (report.md:30). On a readable-stack platform the upload subgraph is skipped, so that section is absent and the failed install has no guaranteed place in the user-facing output. - Priority: Lowered to
should_fix. The path needs an environment failure, so it does not hit every run, and the same wording already ships incontext/agents/replay-vision/install.mdonmain— this PR copies an established pattern rather than creating a regression. Note also that only part of the suggestion applies: restoring the known-good manifest fallback fits the design, while finishing the task as failed fights the deliberate "do not spiral" rule in both the skill andcontext/agents/integration-v2/install.md:32.
Suggested fix
Finish the install task as failed unless the package is declared. Alternatively, add a known-good manifest fallback before completion. Make capture-exceptions skip edits when the install or init handoff reports failure.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/install.md#L28-31
<issue_description>
These lines accept an environment failure as successful completion. The install skill can finish without declaring the SDK. The next task then assumes the SDK exists and adds imports for it. A registry or lockfile failure can therefore leave the project unable to build.
</issue_description>
<issue_validation>
- **Checked:** `context/agents/error-tracking/install.md` in full, the skill it delegates to (`context/skills/integration-v2/install/description.md`), the sibling prompt `context/agents/integration-v2/install.md`, the downstream prompts `capture-exceptions.md` and `init.md`, and `report.md`.
- **Found:** The prompt hands the failure path to its skill. `context/agents/error-tracking/install.md:19` says the skill owns "what counts as an environment failure, and the fallback". The skill defines no fallback. `context/skills/integration-v2/install/description.md:20` ends that path with "do not spiral: report the exact failure and command in your handoff and finish". No step declares the package.
- **Found:** The manifest fallback exists in exactly one place, and this flow does not reach it. `context/agents/integration-v2/install.md:53` tells the agent to "add the PostHog package to the manifest by hand at a known-good version, so the dependency is still declared for the later steps and for the user's own install". That text sits inline in the default flow's own prompt, not in the shared skill, so the new prompt points at a fallback that never existed.
- **Found:** The next task edits code on the assumption that the SDK resolves. `context/agents/error-tracking/capture-exceptions.md:29` states "The SDK is installed and initialized ... build on that, do not re-check it", and that task holds `Write` and `Edit`. `init.md` also writes an init that constructs the client.
- **Impact:** On the environment failures that `context/agents/integration-v2/install.md:38` itself lists — a peer conflict the project already had, a broken lockfile, a package-manager provisioning failure — the run finishes with PostHog imports and an init in the source and no dependency in the manifest. The project stops building, and it built before the run. `install.md:9` already grants `Edit`, so the repair is one sentence of prompt text, not new tooling.
- **Impact:** The report does not reliably surface the failure as a follow-up either. The "What you still need to do" section that carries "Any other manual follow-up the handoffs carry" (`report.md:38`) is written only "When the run wired source-map upload" (`report.md:30`). On a readable-stack platform the upload subgraph is skipped, so that section is absent and the failed install has no guaranteed place in the user-facing output.
- **Priority:** Lowered to `should_fix`. The path needs an environment failure, so it does not hit every run, and the same wording already ships in `context/agents/replay-vision/install.md` on `main` — this PR copies an established pattern rather than creating a regression. Note also that only part of the suggestion applies: restoring the known-good manifest fallback fits the design, while finishing the task as failed fights the deliberate "do not spiral" rule in both the skill and `context/agents/integration-v2/install.md:32`.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Finish the install task as failed unless the package is declared. Alternatively, add a known-good manifest fallback before completion. Make `capture-exceptions` skip edits when the install or init handoff reports failure.
</potential_solution>
| of it. A platform with no autocapture at all, such as Go, is the other case: | ||
| there the capture boundary at the entry point is the mechanism, not a copy of | ||
| one. |
There was a problem hiding this comment.
One Go entry-point boundary cannot cover the process
Issue description
Go has no process-wide exception boundary. recover only catches a panic in the same goroutine. An entry-point boundary misses HTTP handlers, worker goroutines, and returned errors. The Go installation reference instead captures errors with NewDefaultException.
Why we think it's a valid issue
- Checked:
context/agents/error-tracking/capture-exceptions.mdin full, the skill it loads (context/skills/integration-v2/error-tracking-step/— itsdescription.mdand thegovariant atconfig.yaml:306-312), the repo's own Go rules incontext/commandments.yaml, the Go example incontext/skills/error-tracking-upload-source-maps/description.md:362-366, and the docs page thegovariant hands the agent (https://posthog.com/docs/error-tracking/installation/go.md). - Found: The half of the sentence about missing autocapture is correct — the Go docs page describes no panic hook, no process-wide handler, and never mentions
recover(),defer-based recovery, or HTTP middleware. The named mechanism is the part that fails: the page offers exactly two ways to capture,posthog.NewDefaultException(...)plusclient.Enqueue(...), orposthog.NewSlogCaptureHandler(baseHandler, client, ...)wrapped inslog.New(...)for automatic capture atslog.LevelWarnand above. Neither one is "the capture boundary at the entry point". - Found: The repo's own Go guidance agrees with the docs and not with the prompt.
context/commandments.yaml:91lists onlyposthog.NewDefaultException(...)for direct captures andposthog.NewSlogCaptureHandler(...)for automatic capture. The only Go capture snippet elsewhere in the repo,context/skills/error-tracking-upload-source-maps/description.md:362-366, is an explicitclient.Enqueue(posthog.NewDefaultException(...))on a route, not a recover boundary. - Found: The prompt contradicts itself on this platform.
capture-exceptions.md:20-21tells the agent to "Follow the docs and the reference example for this one", andcapture-exceptions.md:63-64sets the success bar as capture "through the mechanism this SDK gives you rather than one you invented". Lines 57-59 name a Go mechanism the SDK does not give, so the agent that obeys those lines fails the file's own bar. - Found: The path is reachable. A
go.modproject is a supported target of this flow —setup-error-tracking.mdmapsgo.modto thegouploader variant, and the capture task's skill carries agovariant with the Go error-tracking page as its docs. - Impact: A
recover()at the process entry recovers only panics on its own goroutine.net/httpruns each connection on its own goroutine and recovers handler panics itself, so a boundary inmain()never sees them. On the most common Go target, a web service, the agent can finish, report a wired capture mechanism, and leave a build where no uncaught error reaches PostHog — a silent no-op, which is the one outcome the flow exists to prevent. - Impact: The mitigating factor is that the correct docs travel with the task, so a careful agent may follow
NewSlogCaptureHandleror explicitNewDefaultExceptioncaptures instead of the sentence. That keeps the defect belowmust_fix, but the sentence still steers against the docs and costs one clause to correct. - Note on the fix: Only part of the suggestion fits the file. Central capture through
NewSlogCaptureHandleror an existing shared error path matches both the docs and the prompt's own "set it up in one place" rule (capture-exceptions.md:21). Adding recovery wrappers per worker goroutine pulls toward the hand-wrapping the same file forbids atcapture-exceptions.md:66-67.
Suggested fix
Direct the task to use existing central boundaries. Add recovery middleware around HTTP requests. Recover inside worker wrappers. Capture returned errors at central handlers with NewDefaultException. Report any paths that have no shared boundary.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/capture-exceptions.md#L57-59
<issue_description>
Go has no process-wide exception boundary. `recover` only catches a panic in the same goroutine. An entry-point boundary misses HTTP handlers, worker goroutines, and returned errors. The Go installation reference instead captures errors with `NewDefaultException`.
</issue_description>
<issue_validation>
- **Checked:** `context/agents/error-tracking/capture-exceptions.md` in full, the skill it loads (`context/skills/integration-v2/error-tracking-step/` — its `description.md` and the `go` variant at `config.yaml:306-312`), the repo's own Go rules in `context/commandments.yaml`, the Go example in `context/skills/error-tracking-upload-source-maps/description.md:362-366`, and the docs page the `go` variant hands the agent (`https://posthog.com/docs/error-tracking/installation/go.md`).
- **Found:** The half of the sentence about missing autocapture is correct — the Go docs page describes no panic hook, no process-wide handler, and never mentions `recover()`, `defer`-based recovery, or HTTP middleware. The named mechanism is the part that fails: the page offers exactly two ways to capture, `posthog.NewDefaultException(...)` plus `client.Enqueue(...)`, or `posthog.NewSlogCaptureHandler(baseHandler, client, ...)` wrapped in `slog.New(...)` for automatic capture at `slog.LevelWarn` and above. Neither one is "the capture boundary at the entry point".
- **Found:** The repo's own Go guidance agrees with the docs and not with the prompt. `context/commandments.yaml:91` lists only `posthog.NewDefaultException(...)` for direct captures and `posthog.NewSlogCaptureHandler(...)` for automatic capture. The only Go capture snippet elsewhere in the repo, `context/skills/error-tracking-upload-source-maps/description.md:362-366`, is an explicit `client.Enqueue(posthog.NewDefaultException(...))` on a route, not a recover boundary.
- **Found:** The prompt contradicts itself on this platform. `capture-exceptions.md:20-21` tells the agent to "Follow the docs and the reference example for this one", and `capture-exceptions.md:63-64` sets the success bar as capture "through the mechanism this SDK gives you rather than one you invented". Lines 57-59 name a Go mechanism the SDK does not give, so the agent that obeys those lines fails the file's own bar.
- **Found:** The path is reachable. A `go.mod` project is a supported target of this flow — `setup-error-tracking.md` maps `go.mod` to the `go` uploader variant, and the capture task's skill carries a `go` variant with the Go error-tracking page as its docs.
- **Impact:** A `recover()` at the process entry recovers only panics on its own goroutine. `net/http` runs each connection on its own goroutine and recovers handler panics itself, so a boundary in `main()` never sees them. On the most common Go target, a web service, the agent can finish, report a wired capture mechanism, and leave a build where no uncaught error reaches PostHog — a silent no-op, which is the one outcome the flow exists to prevent.
- **Impact:** The mitigating factor is that the correct docs travel with the task, so a careful agent may follow `NewSlogCaptureHandler` or explicit `NewDefaultException` captures instead of the sentence. That keeps the defect below `must_fix`, but the sentence still steers against the docs and costs one clause to correct.
- **Note on the fix:** Only part of the suggestion fits the file. Central capture through `NewSlogCaptureHandler` or an existing shared error path matches both the docs and the prompt's own "set it up in one place" rule (`capture-exceptions.md:21`). Adding recovery wrappers per worker goroutine pulls toward the hand-wrapping the same file forbids at `capture-exceptions.md:66-67`.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Direct the task to use existing central boundaries. Add recovery middleware around HTTP requests. Recover inside worker wrappers. Capture returned errors at central handlers with `NewDefaultException`. Report any paths that have no shared boundary.
</potential_solution>
| ## The runtime boundary has names too | ||
|
|
||
| Upload credentials are a build-time concern. The app also reads its own | ||
| variables at **run** time, and the step that starts it is another place a name | ||
| has to match — a `docker run -e`, a compose file, a systemd unit, a platform's | ||
| environment settings. That step was written before PostHog existed here, so the | ||
| name it passes is whatever the project used back then. If `init` settled on a | ||
| different one, the deployed process starts with an undefined key. | ||
|
|
||
| Nothing fails loudly. In production the init guard returns quietly instead of | ||
| throwing, so the process boots, serves traffic, and reports nothing — the same | ||
| silent shape as a mismatched upload variable, one boundary later. | ||
|
|
||
| So read the names the app's own source reads, and make the start step pass | ||
| exactly those. Rename the deploy step's variable, not the code: the code is the | ||
| half that already exists. Name any secret the user must create under its new | ||
| name in your handoff. |
There was a problem hiding this comment.
Client SDK variables are wired at the wrong lifecycle stage
Issue description
The task treats every SDK token and host as runtime values. Vite and NEXT_PUBLIC_* values are replaced during the production build. Runtime environment settings cannot update these values after the bundle exists. CI can upload maps but ship an SDK with an undefined token. Error capture then fails without an error.
Why we think it's a valid issue
- Checked: every prompt in
context/agents/error-tracking/for who wires the app's own PostHog token into a container or CI build. I grepped the flow fordocker,runtime, andbuild time. Only wire-ci.md names a container build;configure.md:47-55covers a local run script, andinit.md:52-62covers.envloading on the developer machine. - Found: wire-ci.md classifies the app's own variables as runtime only. Line 65 says the app reads them "at run time". Lines 67-68 list the places to fix as "a
docker run -e, a compose file, a systemd unit, a platform's environment settings". Line 77-78 says "make the start step pass exactly those". The success criterion at lines 86-87 repeats it: "the step that starts the app passes the variable names the app's own source reads". - Found: the build-time half is never named. wire-ci.md:56-57 scopes the
ARG-plus-ENVshape to "the git variables and the non-secret settings only", and in context those settings are the uploader's project id and host, not the SDK's public token. No line in the file tells the agent to pass the SDK token into the build stage. - Found: most variants this flow reaches are client bundles. The seed's uploader table (context/agents/error-tracking/setup-error-tracking.md) selects
nextjs,nuxt,angular,vite,webpack,rollup,react, andweb. For these the bundler replaces the token expression during the production build, so a value supplied at start time cannot reach the shipped bundle. - Found: one platform escapes the gap by accident. init.md:80-88 writes the real public token as a literal into the committed
src/environments/*files for Angular on the stock builder, so that build needs no environment. - Impact: for a Vite or Next project with a Dockerfile or CI build, the agent follows this section, wires the token into the start step, and the client bundle still builds with an undefined token. The env file is gitignored, so the build stage has no value to inline. The result is the exact silent failure the section warns about at lines 73-75: the app boots, serves traffic, and reports nothing. A static SPA served by a web server has no start step at all, so the guidance is a no-op there. The gap sits inside this task's lane, because wire-ci already edits the build stage and the CI job where the token belongs.
- Priority: lowered to
should_fix. The task's primary job, carrying the upload credentials and the release identity into the build, still works. The failure needs a specific shape: a client-bundled project with a container or CI build whose existing config does not already carry the token. Angular is covered by the literal token in init.md. The flow's local outcome is unaffected, so this is an incomplete new section rather than a break.
Suggested fix
Classify each SDK variable as build-time or runtime from its source expression. Pass client variables into the production build. Pass server variables into the start step. For Docker builds, add the public token and host to the build stage.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/wire-ci.md#L64-80
<issue_description>
The task treats every SDK token and host as runtime values. Vite and `NEXT_PUBLIC_*` values are replaced during the production build. Runtime environment settings cannot update these values after the bundle exists. CI can upload maps but ship an SDK with an undefined token. Error capture then fails without an error.
</issue_description>
<issue_validation>
- **Checked:** every prompt in `context/agents/error-tracking/` for who wires the app's own PostHog token into a container or CI build. I grepped the flow for `docker`, `runtime`, and `build time`. Only wire-ci.md names a container build; `configure.md:47-55` covers a local run script, and `init.md:52-62` covers `.env` loading on the developer machine.
- **Found:** wire-ci.md classifies the app's own variables as runtime only. Line 65 says the app reads them "at **run** time". Lines 67-68 list the places to fix as "a `docker run -e`, a compose file, a systemd unit, a platform's environment settings". Line 77-78 says "make the start step pass exactly those". The success criterion at lines 86-87 repeats it: "the step that starts the app passes the variable names the app's own source reads".
- **Found:** the build-time half is never named. wire-ci.md:56-57 scopes the `ARG`-plus-`ENV` shape to "the git variables and the non-secret settings only", and in context those settings are the uploader's project id and host, not the SDK's public token. No line in the file tells the agent to pass the SDK token into the build stage.
- **Found:** most variants this flow reaches are client bundles. The seed's uploader table (context/agents/error-tracking/setup-error-tracking.md) selects `nextjs`, `nuxt`, `angular`, `vite`, `webpack`, `rollup`, `react`, and `web`. For these the bundler replaces the token expression during the production build, so a value supplied at start time cannot reach the shipped bundle.
- **Found:** one platform escapes the gap by accident. init.md:80-88 writes the real public token as a literal into the committed `src/environments/*` files for Angular on the stock builder, so that build needs no environment.
- **Impact:** for a Vite or Next project with a Dockerfile or CI build, the agent follows this section, wires the token into the start step, and the client bundle still builds with an undefined token. The env file is gitignored, so the build stage has no value to inline. The result is the exact silent failure the section warns about at lines 73-75: the app boots, serves traffic, and reports nothing. A static SPA served by a web server has no start step at all, so the guidance is a no-op there. The gap sits inside this task's lane, because wire-ci already edits the build stage and the CI job where the token belongs.
- **Priority:** lowered to `should_fix`. The task's primary job, carrying the upload credentials and the release identity into the build, still works. The failure needs a specific shape: a client-bundled project with a container or CI build whose existing config does not already carry the token. Angular is covered by the literal token in init.md. The flow's local outcome is unaffected, so this is an incomplete new section rather than a break.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Classify each SDK variable as build-time or runtime from its source expression. Pass client variables into the production build. Pass server variables into the start step. For Docker builds, add the public token and host to the build stage.
</potential_solution>
| So forward the provider's git variables into the build the same way you forward | ||
| credentials, and declare each one as `ARG` **and** `ENV` — `ARG` alone is not | ||
| visible to the uploader's environment lookup. Your skill's "Associate the | ||
| release with a git commit" step lists the variables per provider. Read that | ||
| step even though it is not the CI step: this boundary is where it applies. | ||
|
|
||
| Forward them only where a provider actually sets them. A project with no | ||
| pipeline — an image built and run by hand — has nothing to inherit from, and | ||
| variables declared but never filled resolve to no release at all: it reads as | ||
| wired and still fails the build. There the identity has to be supplied outright, | ||
| both halves, from something the build itself holds — a build argument the | ||
| operator passes, or the manifest's own version — so name and version are always | ||
| present. Decide which case you are in by reading the repo, not by assuming a |
There was a problem hiding this comment.
Release forwarding does not cover the stated CI providers
Issue description
The prompt says the skill lists git variables for each provider. The skill only lists GitHub variables. The CLI reads GitHub and Vercel variables. Docker builds on other CI providers can forward variables that the CLI ignores. Error Tracking then loses the release association.
Why we think it's a valid issue
- Checked: the step wire-ci.md points the agent at, and what that step actually contains.
- Found: the statement is wrong about the reference. wire-ci.md:43-44 says "Your skill's "Associate the release with a git commit" step lists the variables per provider." That step runs from description.md:309 to description.md:323. It holds one tip, "Forward GitHub's git env vars into the Docker build", and two examples: "GitHub Actions → docker build", listing
GITHUB_ACTIONS,GITHUB_SHA,GITHUB_REF_NAME,GITHUB_REPOSITORYandGITHUB_SERVER_URL, and "Inline CI build (no Docker)", which says GitHub Actions already sets them. One provider, not several. - Found: the skill's escape hatch for other providers does not transfer to this step. description.md:186 tells the agent the worked examples are "exemplars, not an allowlist" and to apply the same principle with its own knowledge of the provider. That works for credentials, because the variable names there are the project's own and any name works once both sides agree. Release detection is the opposite: the CLI decides which names it reads, so an invented name is not equivalent.
- Found: non-GitHub providers are an anticipated shape in this flow. The skill's CI step carries a dedicated GitLab example at description.md:288-305, and wire-ci.md:22 tells the agent to wire credentials "whatever the CI provider".
- Found: the real consequence is the one the skill states, not the one the prompt states. description.md:311 says that without the variables "the release ends up linked to nothing". wire-ci.md:38-39 instead claims the uploader "stops the build when it has only one" half. Nothing in the skill supports a build failure.
- Found: part of the suggested remedy conflicts with the skill. description.md:311 says "you never touch the CLI invocation itself (it's usually baked into
npm run buildor a bundler plugin)", so adding explicit--release-nameand--release-versionflags is not a change this skill permits. - Impact: on a GitLab, CircleCI, Jenkins, or Bitbucket pipeline that builds a container, the agent is sent to a reference that answers only for GitHub. It forwards that provider's own git variables as
ARGandENV, the uploader does not read them, and the release association is lost. The Dockerfile keeps inert lines that read as wired. Source maps still upload and stack traces still resolve, so nothing announces the gap. - Priority: lowered to
consider. The loss is confined to the release-to-commit association, which is metadata on an issue rather than symbolication. The claim that the CLI reads only GitHub and Vercel variables comes from outside this repo and I could not confirm it here, so the exact set of affected providers is unverified.
Suggested fix
Document exact mappings for each supported CI provider. For unsupported provider variables, pass explicit --release-name and --release-version values or matching plugin options. Add a fallback that verifies both release fields are available.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/wire-ci.md#L41-53
<issue_description>
The prompt says the skill lists git variables for each provider. The skill only lists GitHub variables. The CLI reads GitHub and Vercel variables. Docker builds on other CI providers can forward variables that the CLI ignores. Error Tracking then loses the release association.
</issue_description>
<issue_validation>
- **Checked:** the step wire-ci.md points the agent at, and what that step actually contains.
- **Found:** the statement is wrong about the reference. wire-ci.md:43-44 says "Your skill's "Associate the release with a git commit" step lists the variables per provider." That step runs from description.md:309 to description.md:323. It holds one tip, "Forward GitHub's git env vars into the Docker build", and two examples: "GitHub Actions → docker build", listing `GITHUB_ACTIONS`, `GITHUB_SHA`, `GITHUB_REF_NAME`, `GITHUB_REPOSITORY` and `GITHUB_SERVER_URL`, and "Inline CI build (no Docker)", which says GitHub Actions already sets them. One provider, not several.
- **Found:** the skill's escape hatch for other providers does not transfer to this step. description.md:186 tells the agent the worked examples are "exemplars, not an allowlist" and to apply the same principle with its own knowledge of the provider. That works for credentials, because the variable names there are the project's own and any name works once both sides agree. Release detection is the opposite: the CLI decides which names it reads, so an invented name is not equivalent.
- **Found:** non-GitHub providers are an anticipated shape in this flow. The skill's CI step carries a dedicated GitLab example at description.md:288-305, and wire-ci.md:22 tells the agent to wire credentials "whatever the CI provider".
- **Found:** the real consequence is the one the skill states, not the one the prompt states. description.md:311 says that without the variables "the release ends up linked to nothing". wire-ci.md:38-39 instead claims the uploader "stops the build when it has only one" half. Nothing in the skill supports a build failure.
- **Found:** part of the suggested remedy conflicts with the skill. description.md:311 says "you never touch the CLI invocation itself (it's usually baked into `npm run build` or a bundler plugin)", so adding explicit `--release-name` and `--release-version` flags is not a change this skill permits.
- **Impact:** on a GitLab, CircleCI, Jenkins, or Bitbucket pipeline that builds a container, the agent is sent to a reference that answers only for GitHub. It forwards that provider's own git variables as `ARG` and `ENV`, the uploader does not read them, and the release association is lost. The Dockerfile keeps inert lines that read as wired. Source maps still upload and stack traces still resolve, so nothing announces the gap.
- **Priority:** lowered to `consider`. The loss is confined to the release-to-commit association, which is metadata on an issue rather than symbolication. The claim that the CLI reads only GitHub and Vercel variables comes from outside this repo and I could not confirm it here, so the exact set of affected providers is unverified.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Document exact mappings for each supported CI provider. For unsupported provider variables, pass explicit `--release-name` and `--release-version` values or matching plugin options. Add a fallback that verifies both release fields are available.
</potential_solution>
| **1. Is PostHog already integrated?** Look for `posthog-js` or a server SDK in | ||
| the dependency manifests, or a `posthog.init(...)` / snippet in the source. | ||
| Check the project state for existing events if the repo is ambiguous. | ||
|
|
||
| Integrated means the init runs, not that the package is listed. An init point | ||
| is a pair: the call, and the key it is constructed from. A repo can carry the | ||
| call while the key it names is defined nowhere — then the client is built from | ||
| an empty string and captures nothing, however complete the manifest looks. So | ||
| when you find an init that reads a variable, look for that same name in the | ||
| repo's committed env template or its build config. Found, or the project state | ||
| shows real events arriving: integrated. Named nowhere, or you cannot tell: | ||
| **queue `init`**. It re-checks the pair itself and leaves a complete init |
There was a problem hiding this comment.
Check integration for every runtime surface
Issue description
Fact 1 marks a project as integrated after it finds one valid init pair. This is incomplete for full-stack and multi-platform applications. A project can have a working browser init but no server SDK or server init. The seed then queues neither install nor init. capture-exceptions cannot add dependencies, so uncaught server errors never reach PostHog.
Why we think it's a valid issue
- Checked: how fact 1 is phrased, what the two queueing rules test, whether the downstream tasks can cover a second runtime surface, and whether anything reports a surface the run left out.
- Found: fact 1 is a single-pair test.
setup-error-tracking.md:24-25asks the planner to "Look forposthog-jsor a server SDK in the dependency manifests, or aposthog.init(...)/ snippet in the source", and:28-29defines the result as one pair — "An init point is a pair: the call, and the key it is constructed from." The prompt names both SDK kinds but treats them as alternatives for one yes-or-no answer, never as a checklist. - Found: both queueing rules inherit that.
setup-error-tracking.md:82queuesinstall"only when the SDK is missing from the manifest" (singular), and:83-84queuesinitonly "whenever fact 1 did not show a complete pair". A full-stack repo that declaresposthog-jsand has a working browser init satisfies both gates and gets neither task. - Found: the machinery to cover the server half exists and is simply never reached.
install.md:22-23already says to "Install the SDK the errors will report through (the server library too, if the app runs server-side code)", and the init skill already requires breadth: "An app that builds for several platforms needs an init point per platform it targets, not one shared init — a single SDK call often covers only some of them, and the rest stay uninstrumented while the build still succeeds" (context/skills/integration-v2/init/description.md:48-52). - Found:
capture-exceptionscannot close the gap itself. Its grant is[Read, Write, Edit, Glob, Grep]with noBash(capture-exceptions.md:10), its body says "Do not install dependencies" (:33), and its success criterion requires "You did not install anything" (:65). It is also told the SDK is ready and to "build on that, do not re-check it" (:29-31). - Found: one soft safety net exists. The runner injects a tool inventory into every task that ends: "Later tasks in this run hold tools you do not: when your task needs one, hand that work off in your handoff for the task that can do it, or note it for the final report" (
renderToolInventoryin PostHog/wizardsrc/lib/agent/agent-prompt-loader.ts). An agent that recognises the missing server library can carry it to the report as a follow-up. - Impact: a full-stack app whose PostHog was added client-side only — a common shape for Next.js, Nuxt, SvelteKit and Remix projects that adopted
posthog-jsfor analytics — finishes the run with browser exceptions captured and server-side uncaught errors captured nowhere. No task installs the server library, and the seed passes no surface information tocapture-exceptions(setup-error-tracking.md:85-86). - Priority: lowered from
must_fix. The client half does work, nothing that existed breaks, and the run has a defined way to surface what it could not do. The gap is a coarse gate in front of tasks that already know how to handle several surfaces, which puts it level with the other queueing-gate findings rather than above them.
Suggested fix
Map every runtime surface before you set fact 1. Require an installed SDK and a valid init pair for each target. Queue install or init when any required surface is incomplete. Pass the missing surfaces in task inputs.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/setup-error-tracking.md#L24-35
@context/agents/error-tracking/setup-error-tracking.md#L82-86
<issue_description>
Fact 1 marks a project as integrated after it finds one valid init pair. This is incomplete for full-stack and multi-platform applications. A project can have a working browser init but no server SDK or server init. The seed then queues neither `install` nor `init`. `capture-exceptions` cannot add dependencies, so uncaught server errors never reach PostHog.
</issue_description>
<issue_validation>
- **Checked:** how fact 1 is phrased, what the two queueing rules test, whether the downstream tasks can cover a second runtime surface, and whether anything reports a surface the run left out.
- **Found:** fact 1 is a single-pair test. `setup-error-tracking.md:24-25` asks the planner to "Look for `posthog-js` **or** a server SDK in the dependency manifests, or a `posthog.init(...)` / snippet in the source", and `:28-29` defines the result as one pair — "An init point is a pair: the call, and the key it is constructed from." The prompt names both SDK kinds but treats them as alternatives for one yes-or-no answer, never as a checklist.
- **Found:** both queueing rules inherit that. `setup-error-tracking.md:82` queues `install` "only when the SDK is missing from the manifest" (singular), and `:83-84` queues `init` only "whenever fact 1 did not show a complete pair". A full-stack repo that declares `posthog-js` and has a working browser init satisfies both gates and gets neither task.
- **Found:** the machinery to cover the server half exists and is simply never reached. `install.md:22-23` already says to "Install the SDK the errors will report through (the server library too, if the app runs server-side code)", and the init skill already requires breadth: "An app that builds for several platforms needs an init point per platform it targets, not one shared init — a single SDK call often covers only some of them, and the rest stay uninstrumented while the build still succeeds" (`context/skills/integration-v2/init/description.md:48-52`).
- **Found:** `capture-exceptions` cannot close the gap itself. Its grant is `[Read, Write, Edit, Glob, Grep]` with no `Bash` (`capture-exceptions.md:10`), its body says "Do not install dependencies" (`:33`), and its success criterion requires "You did not install anything" (`:65`). It is also told the SDK is ready and to "build on that, do not re-check it" (`:29-31`).
- **Found:** one soft safety net exists. The runner injects a tool inventory into every task that ends: "Later tasks in this run hold tools you do not: when your task needs one, hand that work off in your handoff for the task that can do it, or note it for the final report" (`renderToolInventory` in PostHog/wizard `src/lib/agent/agent-prompt-loader.ts`). An agent that recognises the missing server library can carry it to the report as a follow-up.
- **Impact:** a full-stack app whose PostHog was added client-side only — a common shape for Next.js, Nuxt, SvelteKit and Remix projects that adopted `posthog-js` for analytics — finishes the run with browser exceptions captured and server-side uncaught errors captured nowhere. No task installs the server library, and the seed passes no surface information to `capture-exceptions` (`setup-error-tracking.md:85-86`).
- **Priority:** lowered from `must_fix`. The client half does work, nothing that existed breaks, and the run has a defined way to surface what it could not do. The gap is a coarse gate in front of tasks that already know how to handle several surfaces, which puts it level with the other queueing-gate findings rather than above them.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Map every runtime surface before you set fact 1. Require an installed SDK and a valid init pair for each target. Queue `install` or `init` when any required surface is incomplete. Pass the missing surfaces in task inputs.
</potential_solution>
| ## Make the environment actually reachable | ||
|
|
||
| Writing the keys to `.env` is only half the job — something has to load that | ||
| file at runtime, or the app throws on boot and captures nothing. | ||
|
|
||
| Most frameworks do it for you: Next, Nuxt, Astro and SvelteKit auto-load `.env`, | ||
| and Vite auto-loads it for client code. Nothing to do there. | ||
|
|
||
| A plain Node backend does not — Express, Fastify, Hono, Koa, a raw `node:http` | ||
| server — and neither does a bare Rollup or webpack config. When your init point | ||
| reads `process.env` on one of those, wire the loading too, either way: | ||
|
|
||
| - install `dotenv` with the project's own package manager (detect it from the | ||
| lockfile) and import it above the PostHog init — `require('dotenv').config()`, | ||
| or `import 'dotenv/config'` for ESM; or | ||
| - add `--env-file=.env` to the `start` and `dev` scripts, when the project is on | ||
| Node 20.6+ and would rather not take a new dependency. |
There was a problem hiding this comment.
Non-JavaScript runtimes never load the env file
Issue description
These instructions explain .env loading only for JavaScript runtimes. Java and Elixir read process variables, but neither runtime loads .env by default. These variants skip configure, so the flow can report success with an empty token.
Why we think it's a valid issue
- Checked: the whole "Make the environment actually reachable" section (
init.md:49-89) and the success criterion it feeds (init.md:106-114), the variant matrix the init skill inherits (context/skills/integration/config.yaml), the seed's platform routing (setup-error-tracking.md:62), and the per-language rules that reach this task throughposthog-best-practices(init.md:9,context/commandments.yaml). - Found: The section states a runtime-agnostic rule and then covers one ecosystem.
init.md:51-52says "something has to load that file at runtime, or the app throws on boot and captures nothing", after which every named platform is JavaScript: Next, Nuxt, Astro, SvelteKit and Vite at:54-55, Express/Fastify/Hono/Koa/node:httpplus bare Rollup and webpack at:57-65, and Angular at:72-89. Python, Ruby, Java, Elixir and .NET get no rule. - Found: Those runtimes are first-class targets of this task.
context/skills/integration/config.yamlshipsdjango,flask,fastapi,python,ruby,ruby-on-rails,java(:250, "Java (Spring Boot)"),elixir,phpandlaravel, andsetup-error-tracking.md:62routes them away only from the upload subgraph, never frominit. - Found: The two halves the agent is given do not meet on those runtimes.
init.md:108-110requires "keys in the env file and confirmed there withcheck_env_keys, never hardcoded", while the language rules tell the code to read the process environment —commandments.yaml:144"Initialize PostHog in AppConfig.ready() with api_key and host from environment variables",:213for Rails,:124for Elixir "read secrets from environment or runtime config". Nothing populates that environment from the.envthe wizard just wrote, andcommandments.yamlhas nojavasection at all. - Found: No later task closes it, though not for the reason given.
configureowns build-config edits for source-map upload and is queued only when an uploader variant matched, so it would not wire runtime env loading even when present — the gap is that no task in any branch owns this for non-JS runtimes. - Found: The claim does not hold uniformly, which bounds the impact. Laravel loads
.envnatively andcommandments.yaml:179-180has it read throughenv(), so that chain works. Flask's CLI loads.envwhen python-dotenv is installed, and many Python and Ruby projects already carrypython-dotenv,django-environordotenv-rails, in which case the written keys resolve unchanged. - Found: The file also carries a correct fallback that generalises, at
init.md:85-86: "do not wire a lookup unless you have opened the thing it reads from and seen your key defined there." Applied to a Django or Phoenix project with no loader, that rule stops the broken lookup — it is simply never connected to these runtimes. - Impact: On a Django, Spring Boot or Phoenix repo with no existing loader, the run writes the token to
.env, writes code reading the process environment, and finishes. The client is then constructed from an empty or missing value and captures nothing, which is the quiet failureinit.md:45-47exists to prevent — or the lookup raises and the app fails to boot. - Priority: Lowered to
should_fix. The prompt is silent for these runtimes rather than wrong about them, several of the platforms named in the finding do load.envor commonly already have a loader, andinit.md:85-86plus the framework docs give a correct path when applied — so the failure is likely on a subset of these projects, not all of them.
Suggested fix
Add loader rules for every supported server runtime. Update the existing launch path or framework configuration to load the selected env file.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/init.md#L49-65
@context/agents/error-tracking/init.md#L106-112
<issue_description>
These instructions explain `.env` loading only for JavaScript runtimes. Java and Elixir read process variables, but neither runtime loads `.env` by default. These variants skip `configure`, so the flow can report success with an empty token.
</issue_description>
<issue_validation>
- **Checked:** the whole "Make the environment actually reachable" section (`init.md:49-89`) and the success criterion it feeds (`init.md:106-114`), the variant matrix the init skill inherits (`context/skills/integration/config.yaml`), the seed's platform routing (`setup-error-tracking.md:62`), and the per-language rules that reach this task through `posthog-best-practices` (`init.md:9`, `context/commandments.yaml`).
- **Found:** The section states a runtime-agnostic rule and then covers one ecosystem. `init.md:51-52` says "something has to load that file at runtime, or the app throws on boot and captures nothing", after which every named platform is JavaScript: Next, Nuxt, Astro, SvelteKit and Vite at `:54-55`, Express/Fastify/Hono/Koa/`node:http` plus bare Rollup and webpack at `:57-65`, and Angular at `:72-89`. Python, Ruby, Java, Elixir and .NET get no rule.
- **Found:** Those runtimes are first-class targets of this task. `context/skills/integration/config.yaml` ships `django`, `flask`, `fastapi`, `python`, `ruby`, `ruby-on-rails`, `java` (`:250`, "Java (Spring Boot)"), `elixir`, `php` and `laravel`, and `setup-error-tracking.md:62` routes them away only from the *upload* subgraph, never from `init`.
- **Found:** The two halves the agent is given do not meet on those runtimes. `init.md:108-110` requires "keys in the env file and confirmed there with `check_env_keys`, never hardcoded", while the language rules tell the code to read the process environment — `commandments.yaml:144` "Initialize PostHog in AppConfig.ready() with api_key and host from environment variables", `:213` for Rails, `:124` for Elixir "read secrets from environment or runtime config". Nothing populates that environment from the `.env` the wizard just wrote, and `commandments.yaml` has no `java` section at all.
- **Found:** No later task closes it, though not for the reason given. `configure` owns build-config edits for source-map upload and is queued only when an uploader variant matched, so it would not wire runtime env loading even when present — the gap is that no task in any branch owns this for non-JS runtimes.
- **Found:** The claim does not hold uniformly, which bounds the impact. Laravel loads `.env` natively and `commandments.yaml:179-180` has it read through `env()`, so that chain works. Flask's CLI loads `.env` when python-dotenv is installed, and many Python and Ruby projects already carry `python-dotenv`, `django-environ` or `dotenv-rails`, in which case the written keys resolve unchanged.
- **Found:** The file also carries a correct fallback that generalises, at `init.md:85-86`: "do not wire a lookup unless you have opened the thing it reads from and seen your key defined there." Applied to a Django or Phoenix project with no loader, that rule stops the broken lookup — it is simply never connected to these runtimes.
- **Impact:** On a Django, Spring Boot or Phoenix repo with no existing loader, the run writes the token to `.env`, writes code reading the process environment, and finishes. The client is then constructed from an empty or missing value and captures nothing, which is the quiet failure `init.md:45-47` exists to prevent — or the lookup raises and the app fails to boot.
- **Priority:** Lowered to `should_fix`. The prompt is silent for these runtimes rather than wrong about them, several of the platforms named in the finding do load `.env` or commonly already have a loader, and `init.md:85-86` plus the framework docs give a correct path when applied — so the failure is likely on a subset of these projects, not all of them.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Add loader rules for every supported server runtime. Update the existing launch path or framework configuration to load the selected env file.
</potential_solution>
| So do not wire a lookup unless you have opened the thing it reads from and seen | ||
| your key defined there. When nothing populates it, write the real public project | ||
| token as a literal in the committed `src/environments/*` files. This is the | ||
| skill's "no valid environment to read from" case, and the public token is | ||
| publishable — it ships inside the browser bundle either way. |
There was a problem hiding this comment.
Angular fallback omits the project host
Issue description
The fallback writes only the project token. Angular initialization also passes environment.posthogHost to api_host. An undefined or US-default host sends EU and self-hosted events to the wrong deployment.
Why we think it's a valid issue
- Checked: every mention of configuration values in
context/agents/error-tracking/init.md(the env-var path at:49-70, the Angular fallback at:72-89, the type-check rule at:91-104, the success criterion at:106-114), the skill this task loads (context/skills/integration-v2/init/description.md), the Angular rules that reach it viaposthog-best-practices(context/commandments.yaml:310-314), and the Angular page the variant supplies (https://posthog.com/docs/error-tracking/installation/angular.md). - Found: The host is absent from this file end to end.
init.md:86-87says to "write the real public project token as a literal in the committedsrc/environments/*files",init.md:111-112checks only that "the token is a literal rather than a lookup into something that never defines it", and no line in the file names the API host, the region, or a<UI_HOST>substitution — unlikecredentials.mdandreport.md, which tell the agent to replace<UI_HOST>from project context. - Found: The Angular init genuinely needs two values. The docs page initializes with
posthog.init(environment.posthogKey, { api_host: environment.posthogHost, defaults: '2026-05-30' }), reading both properties fromsrc/environments/environment.ts— the same file the fallback tells the agent to write.commandments.yaml:314says only "Configure PostHog credentials in src/environments/environment.ts files" and names neither value. - Found: The requirement reaches the agent only through a path that does not cover this case. The init skill's §Environment variables lists both "the public project token" and "the PostHog host", but scopes them to values set "through the wizard tools (
set_env_values)". Its own fallback sentence — "embed the real public token in the config the build ships" — omits the host exactly asinit.mddoes, so on the no-environment branch nothing in context asks for it. - Found: Angular's type checking bounds the failure. Angular projects are TypeScript, and
init.md:93-96requires the init to compile by construction, so anenvironment.posthogHostreference with no matching property fails the build rather than shipping. The reachable failure is therefore a wrong host value — an empty string, or a hardcoded US address on a non-US project — not a missing property. - Impact:
posthog-jsfalls back to the US ingestion host whenapi_hostis empty or absent, so an EU-cloud or self-hosted project initialised this way posts events with a token that region does not know. Nothing throws, the app runs, and Error Tracking stays empty — the same quiet failureinit.md:45-47warns about, on the stock@angular/buildbuilder this file itself calls "the common one" (init.md:73). A US-cloud project is unaffected because the default matches. - Priority: Lowered to
should_fix. The docs snippet and the compile-by-construction rule force both properties to exist, so this needs the agent to source a wrong host value rather than skip it, and the consequence lands only on non-US-cloud projects. The repair is still small — name the host beside the token ininit.md:86-87and require both at:111-112.
Suggested fix
Write both the project token and client API host into each active Angular environment file. Require both values in the success criteria.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/init.md#L85-89
@context/agents/error-tracking/init.md#L108-112
<issue_description>
The fallback writes only the project token. Angular initialization also passes `environment.posthogHost` to `api_host`. An undefined or US-default host sends EU and self-hosted events to the wrong deployment.
</issue_description>
<issue_validation>
- **Checked:** every mention of configuration values in `context/agents/error-tracking/init.md` (the env-var path at `:49-70`, the Angular fallback at `:72-89`, the type-check rule at `:91-104`, the success criterion at `:106-114`), the skill this task loads (`context/skills/integration-v2/init/description.md`), the Angular rules that reach it via `posthog-best-practices` (`context/commandments.yaml:310-314`), and the Angular page the variant supplies (`https://posthog.com/docs/error-tracking/installation/angular.md`).
- **Found:** The host is absent from this file end to end. `init.md:86-87` says to "write the real public project token as a literal in the committed `src/environments/*` files", `init.md:111-112` checks only that "the token is a literal rather than a lookup into something that never defines it", and no line in the file names the API host, the region, or a `<UI_HOST>` substitution — unlike `credentials.md` and `report.md`, which tell the agent to replace `<UI_HOST>` from project context.
- **Found:** The Angular init genuinely needs two values. The docs page initializes with `posthog.init(environment.posthogKey, { api_host: environment.posthogHost, defaults: '2026-05-30' })`, reading both properties from `src/environments/environment.ts` — the same file the fallback tells the agent to write. `commandments.yaml:314` says only "Configure PostHog credentials in src/environments/environment.ts files" and names neither value.
- **Found:** The requirement reaches the agent only through a path that does not cover this case. The init skill's §Environment variables lists both "the public project token" and "the PostHog host", but scopes them to values set "through the wizard tools (`set_env_values`)". Its own fallback sentence — "embed the real public token in the config the build ships" — omits the host exactly as `init.md` does, so on the no-environment branch nothing in context asks for it.
- **Found:** Angular's type checking bounds the failure. Angular projects are TypeScript, and `init.md:93-96` requires the init to compile by construction, so an `environment.posthogHost` reference with no matching property fails the build rather than shipping. The reachable failure is therefore a *wrong* host value — an empty string, or a hardcoded US address on a non-US project — not a missing property.
- **Impact:** `posthog-js` falls back to the US ingestion host when `api_host` is empty or absent, so an EU-cloud or self-hosted project initialised this way posts events with a token that region does not know. Nothing throws, the app runs, and Error Tracking stays empty — the same quiet failure `init.md:45-47` warns about, on the stock `@angular/build` builder this file itself calls "the common one" (`init.md:73`). A US-cloud project is unaffected because the default matches.
- **Priority:** Lowered to `should_fix`. The docs snippet and the compile-by-construction rule force both properties to exist, so this needs the agent to source a wrong host value rather than skip it, and the consequence lands only on non-US-cloud projects. The repair is still small — name the host beside the token in `init.md:86-87` and require both at `:111-112`.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Write both the project token and client API host into each active Angular environment file. Require both values in the success criteria.
</potential_solution>
| 1. Ask with `wizard_ask`, exactly: | ||
| `{ id: "api-key", prompt: "Paste your PostHog personal API key below.\n\nDon't have one yet? Create one here:\n<SETTINGS_URL>\n\nWhen creating the key, choose the 'Source map upload' preset, then come back and paste it here.", kind: "text", sensitive: true }` | ||
| You receive `{ secretRef: "secret:..." }` — a vaulted reference, never the raw | ||
| value. If `wizard_ask` is unavailable (non-interactive run), report this task | ||
| with status `not needed` and say in your handoff that the user must create | ||
| the key and set the variables themselves; do not block. |
There was a problem hiding this comment.
Handle a canceled API-key question
Issue description
A canceled or timed-out wizard_ask field returns "__cancelled__". These instructions handle only an unavailable tool. Cancellation can reach set_env_values as an invalid reference or literal value. The task then fails or stores an unusable API key.
Why we think it's a valid issue
- Checked: how this repo's other
wizard_askconsumers treat a cancelled prompt, what branches credentials.md provides, and what reads the resulting handoff. - Found: cancellation is a documented, first-class outcome in this repo, not an edge case. context/skills/data-warehouse-source/description.md:34 states "A cancelled or timed-out
wizard_askdoes not count against the per-run cap. Treat a cancelled ask as 'the user declined' for that source", and context/skills/integration-v2/warehouse/description.md:74 repeats it. A test pins that sentence (scripts/lib/tests/data-warehouse-source-skill.test.js:60). - Found: the same repo already defines the correct response. context/skills/data-warehouse-source/description.md:123 says to report
not neededand "Use this when the user cancelled or declined the prompts". - Found: credentials.md has no such branch. credentials.md:29-31 covers one case only: "If
wizard_askis unavailable (non-interactive run), report this task with statusnot needed". A user who closes the modal, or who never returns, is a different case, and step 3 at credentials.md:36-41 then still says to callset_env_valueswith the secretRef. - Found: the trigger is ordinary, not rare. The prompt text at credentials.md:27 sends the user out of the terminal to create a personal API key in PostHog settings and come back. Abandoning or dismissing that prompt is a normal user action.
- Found: a downstream task reads this handoff and branches on it. report.md:34 tells the user to create a personal API key and says to "skip when the credentials handoff says the key is already written". An ambiguous handoff after a cancelled prompt therefore drops the instruction, so the user is never told to create the key, uploads never run, and the report claims the setup is complete.
- Found: two specifics in the suggested fix are not verifiable here. The string
__cancelled__appears nowhere in this repo, and no file mentions anotNeededReasonfield. The wizard's tool contract lives in another repo that I cannot read from here, so the sentinel and that field are unconfirmed. - Impact: with no branch, the agent improvises. The benign outcome is a task reported incomplete with a vague handoff, which already misleads the report. The harmful outcome is a call to
set_env_valueswith a reference that resolves to nothing, which writes an unusable value under the API-key name. A later run cannot detect that, because credentials.md:32-35 usescheck_env_keys, which reports presence only and never a value.
Suggested fix
Check the api-key answer for "__cancelled__". Report not needed with notNeededReason: "user_declined", and do not call set_env_values.
Prompt to fix with AI (copy-paste)
## Context
@context/agents/error-tracking/credentials.md#L26-31
<issue_description>
A canceled or timed-out `wizard_ask` field returns `"__cancelled__"`. These instructions handle only an unavailable tool. Cancellation can reach `set_env_values` as an invalid reference or literal value. The task then fails or stores an unusable API key.
</issue_description>
<issue_validation>
- **Checked:** how this repo's other `wizard_ask` consumers treat a cancelled prompt, what branches credentials.md provides, and what reads the resulting handoff.
- **Found:** cancellation is a documented, first-class outcome in this repo, not an edge case. context/skills/data-warehouse-source/description.md:34 states "A cancelled or timed-out `wizard_ask` does **not** count against the per-run cap. Treat a cancelled ask as 'the user declined' for that source", and context/skills/integration-v2/warehouse/description.md:74 repeats it. A test pins that sentence (scripts/lib/tests/data-warehouse-source-skill.test.js:60).
- **Found:** the same repo already defines the correct response. context/skills/data-warehouse-source/description.md:123 says to report `not needed` and "Use this when the user cancelled or declined the prompts".
- **Found:** credentials.md has no such branch. credentials.md:29-31 covers one case only: "If `wizard_ask` is unavailable (non-interactive run), report this task with status `not needed`". A user who closes the modal, or who never returns, is a different case, and step 3 at credentials.md:36-41 then still says to call `set_env_values` with the secretRef.
- **Found:** the trigger is ordinary, not rare. The prompt text at credentials.md:27 sends the user out of the terminal to create a personal API key in PostHog settings and come back. Abandoning or dismissing that prompt is a normal user action.
- **Found:** a downstream task reads this handoff and branches on it. report.md:34 tells the user to create a personal API key and says to "skip when the credentials handoff says the key is already written". An ambiguous handoff after a cancelled prompt therefore drops the instruction, so the user is never told to create the key, uploads never run, and the report claims the setup is complete.
- **Found:** two specifics in the suggested fix are not verifiable here. The string `__cancelled__` appears nowhere in this repo, and no file mentions a `notNeededReason` field. The wizard's tool contract lives in another repo that I cannot read from here, so the sentinel and that field are unconfirmed.
- **Impact:** with no branch, the agent improvises. The benign outcome is a task reported incomplete with a vague handoff, which already misleads the report. The harmful outcome is a call to `set_env_values` with a reference that resolves to nothing, which writes an unusable value under the API-key name. A later run cannot detect that, because credentials.md:32-35 uses `check_env_keys`, which reports presence only and never a value.
</issue_validation>
## Task
Investigate the issue and solve it
<potential_solution>
Check the `api-key` answer for `"__cancelled__"`. Report `not needed` with `notNeededReason: "user_declined"`, and do not call `set_env_values`.
</potential_solution>
Part of a 3-PR set
Problem
wizard error-tracking(PostHog/wizard#1185) needs an orchestrator flow. The flow installs PostHog first when the repo has no PostHog.Changes
New flow
context/agents/error-tracking/, a seed and 8 tasks:setup-error-tracking(seed): checks the PostHog init and its key, and picks the source-map uploader. React Native and Expo usereact-native. Readable-stack platforms and Astro use none.install,init: add and initialize the SDK.initkeeps the env names that the code reads, loads.envwhere the platform does not, and writes init code that type-checks.capture-exceptions: turns on SDK autocapture and hooks the framework error handler. It adds no second global listener.credentials,configure,wire-ci,test-setup: wire the source-map upload into the build and CI. The API key stays a secret.report: writesposthog-error-tracking-report.md.Shared skill changes:
integration-v2/error-tracking-step: 40 variants with the error-tracking docs. The default integration-v2 flow uses them too. A new test keeps the variants in sync withintegration.error-tracking-upload-source-maps: install with the project's package manager. Do not set a webpackdevtoolbeside the PostHog plugin.Test plan
node scripts/build.jspasses.pnpm testpasses (174 tests).🤖 Generated with Claude Code
https://claude.ai/code/session_01668rN31F4aotxmenNksz4Z