Skip to content

feat: model invokable workflow skills - #93

Open
tkstang wants to merge 90 commits into
tkstang:mainfrom
voxmedia:feat/model-invokable-workflow-skills
Open

feat: model invokable workflow skills#93
tkstang wants to merge 90 commits into
tkstang:mainfrom
voxmedia:feat/model-invokable-workflow-skills

Conversation

@tkstang

@tkstang tkstang commented Jun 4, 2026

Copy link
Copy Markdown
Owner

Summary: skill-automation-and-review

Overview

This project reduced OAT lifecycle friction by making generated artifacts harder to consume unchecked, making review artifacts easier to resolve from natural-language workflows, and expanding model-invokable lifecycle skills with explicit offer-and-confirm gates.

What Was Implemented

  • Default-on artifact quality gates. Added workflow.autoArtifactReview.plan and workflow.autoArtifactReview.analysis, both defaulting to true, so generated plans and generated docs/agent-instructions analysis artifacts run bounded review loops before implementation or apply workflows consume them.
  • Reviewer subject expansion. Extended oat-reviewer with artifact: plan and analysis: docs|agent-instructions structured review modes. The reviewer stays stateless; authoring skills own the retry loop and artifact rewrites.
  • Shared loop contract and wiring. oat-project-plan-writing owns the canonical auto artifact-review loop contract. oat-project-plan, oat-project-quick-start, and oat-project-import-plan invoke it for plans; oat-docs-analyze and oat-agent-instructions-analyze invoke it for analysis artifacts.
  • Latest review CLI. Added oat review latest to scan project and ad-hoc review locations and return the newest artifact by frontmatter timestamp. A final-review fix added lifecycle recency for same-day ties so final outranks phase reviews and higher phase/task scopes outrank lower ones.
  • Model-invocation guardrails. Made oat-project-review-provide, oat-project-review-receive, oat-project-discover, and oat-project-progress model-invokable for explicit asks only. Each skill documents its gate and prevents silent workflow jumps.
  • Quick-start state fix. oat-project-quick-start now completes discovery in the lightweight-design path before plan generation, preventing quick-mode projects from reaching a complete plan with discovery.md still in_progress.
  • Docs and release. Updated the docs app for the new review/config surfaces and bumped all five lockstep public packages to 0.1.18.

Key Decisions

  • Use the existing oat-reviewer agent rather than creating a separate artifact-review agent.
  • Keep retry behavior bounded by the existing oat_orchestration_retry_limit instead of introducing another limit key.
  • Gate generated-artifact review through config defaults rather than user prompts, because these artifacts are immediately consumed by downstream lifecycle steps.
  • Make review/discovery/progress skills model-invokable only for explicit user asks and confirmation flows.

Design Deltas

  • p01-t01 expanded beyond its initial file list to update config resolver and config command support required by oat config get workflow.autoArtifactReview.*.
  • p03 updated the quick-start skill version contract tests alongside planned quick-start skill version bumps.
  • p05 needed a review-fix commit to align oat-project-review-provide Step 0 with the advertised active-project or explicit-target gate.
  • p06 and final review both found config-precedence wording drift. Docs and config catalog now state that workflow.autoArtifactReview.* has no environment aliases and resolves through local > shared > user > default.
  • Final review added same-day lifecycle ordering for oat review latest.

Verification

  • Phase reviews p01 through p06 passed.
  • Final code review passed after one fix iteration.
  • Focused vitest for review/config surfaces passed.
  • Full p06 definition-of-done gate passed in the phase worktree: pnpm build, pnpm lint, pnpm format, pnpm type-check, pnpm test, pnpm release:validate, and git diff --check.
  • Final focused smokes passed: oat review latest --project ... --json, config describe workflow.autoArtifactReview.plan, config describe workflow.autoArtifactReview.analysis, git diff --check, and pnpm release:validate.

tkstang added 30 commits April 24, 2026 14:27
Removing `disable-model-invocation: true` lets orchestrator skills
(notably `oat-project-document`) chain this skill under Claude's
harness, matching the behavior Codex already exhibits by ignoring
the flag. Tightens the description so Claude does not fire it
speculatively outside the doc-sync flow.

Bumps skill version 1.0.0 -> 1.1.0 and lockstep-bumps the five
public packages 0.0.50 -> 0.0.51 per AGENTS.md release policy.
* feat(skills): make end-of-lifecycle skills model-invokable with explicit-ask gating

Removes `disable-model-invocation` from `oat-project-document`,
`oat-project-pr-final`, `oat-project-summary`, and
`oat-pjm-add-backlog-item` so the model can run them when the user
confirms a previously offered step. Each description now leads with
the explicit-ask trigger phrasings and a "do NOT auto-invoke" clause
naming the false-positive trigger to suppress, so discovery still
filters out unsolicited invocations. Bumps the lockstep public
package versions per the AGENTS.md guardrail.

* refactor(skills): tighten gating descriptions and broaden trigger to user confirmation

Replaces "explicitly asks" with "requests or confirms" so the
description matches the dominant real-world path: bot offers to run
the skill at end of implementation, user replies "yes". Each example
list now includes "or confirms a previously offered <step>" so the
model recognizes a bare confirmation as a valid trigger. Drops the
redundant "wait for an explicit user signal" tail clause; the lead
phrasing and example list already cover that ground.

* chore(cli): untrack bundled CLI assets

These files were tracked by accident before `packages/cli/assets/`
was added to .gitignore — they are derived artifacts regenerated by
`packages/cli/scripts/bundle-assets.sh` during `pnpm build`, and the
release pipeline (`tools/release/validate-public-packages.ts`) builds
each public package before packing, so the published tarball always
ships fresh content. Tracking them created two problems: silent
drift between checked-in versions and what `pnpm build` produced,
and `git add` rejecting routine updates because the path is ignored.
Untracking removes both.
…ing to oat-project-spec (#77)

* fix(skills): rewrite PR description after archive in oat-project-complete

After oat-project-complete archives a project locally (per pre-merge
archive flow), the open PR's description still pointed at the active
artifact paths (plan.md, implementation.md, etc.) — those files were
moved to a gitignored archive location and the Step 10 commit removed
them from the branch, so every blob link in the PR body 404'd while
the PR was still open.

oat-project-complete now captures WAS_PR_OPEN_AT_START before any state
mutation, regenerates the PR artifact with archive-aware References when
SHOULD_ARCHIVE is true (drops bullets for artifacts about to be local-only,
adds a project-record link to the summaryExportPath copy on the current
head branch), and a new Step 11.5 syncs that body to the open PR via
`gh pr edit`. The project-record link targets {BRANCH} rather than
{BASE_BRANCH} because the summary export is committed on the feature
branch in Step 10 and only reaches the base branch after merge.

Contract test in review-skill-contracts.test.ts pins the new behavior
and guards against regressing the project-record link to {BASE_BRANCH}.

* fix: stop recommending oat-project-spec as a default next step

Requirements confirmation was folded into oat-project-design, but the
state-dashboard router, the new-project scaffold, and several router
skills still treated `spec` as a discrete phase and pushed users at
oat-project-spec right after discovery. Newly scaffolded projects
opened with `Phase: spec (in_progress)` and `Recommended next:
oat-project-spec`, undoing the consolidation.

Source-of-truth changes:

- Scaffold spec-driven projects with HiLL checkpoints
  ['discovery', 'design'], reword the Spec artifact bullet to indicate
  oat-project-design authors it inline, and update the next-milestone
  text to reference the design phase.
- Route spec-driven discovery:complete to oat-project-design (not
  oat-project-spec). Legacy projects whose state.md still reads
  spec:in_progress also route to design as graceful migration.
- Map the HiLL phase->skill table so a pending `spec` checkpoint on a
  legacy project routes to oat-project-design.

Skill updates:

- oat-project-progress (1.2.3 -> 1.2.4): HiLL recommendations,
  spec-driven routing matrix, and the help-text inventory now point at
  oat-project-design and frame oat-project-spec as an opt-in standalone.
- oat-project-next (1.0.3 -> 1.0.4): HiLL phase->skill map and the
  spec-driven boundary-tier routing table all route spec entries to
  oat-project-design, with a clarifying note about legacy projects.

oat-project-spec itself is preserved as a deliberate opt-in path.
oat-project-design already reuses any existing spec.md, so the
standalone skill is still wired into the consolidated flow.

Tests:

- Added two generate.test.ts cases pinning the new routing
  (spec-driven discovery:complete -> design; legacy spec:in_progress
  -> design).
- Updated the scaffold.test.ts spec-artifact assertion to match the
  new wording.

Lockstep public package bump 0.0.67 -> 0.0.68 (third content stage on
this branch).

* chore: gitignore .oat/repo/reviews and drop the in-PR review artifact

Treat repo-level review artifacts under .oat/repo/reviews as local-only
scratch, the same way archived review artifacts and brainstorm/idea
artifacts are already handled. Add the directory to .gitignore and to
the localPaths list in .oat/config.json so the oat-project-pr-final
References policy stops linking to paths that will not exist on the
remote.

Also drop the ad-hoc review artifact that landed in commit
1108e06 (it documents the review of that commit and was never
meant to ship as repo content).

Pre-existing review files committed before this rule remain tracked
as historical content. The new gitignore rule only affects future
adds — already-tracked files keep tracking as-is.
tkstang added 29 commits June 4, 2026 18:44
@tkstang
tkstang force-pushed the feat/model-invokable-workflow-skills branch from 81e4929 to 6e3e2f0 Compare June 4, 2026 23:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant