diff --git a/.changeset/lazy-radios-obey.md b/.changeset/lazy-radios-obey.md new file mode 100644 index 0000000..f10f966 --- /dev/null +++ b/.changeset/lazy-radios-obey.md @@ -0,0 +1,5 @@ +--- +"stack-effect": patch +--- + +Reject stale scaffolding plans before previewing or applying repository changes. diff --git a/.okf/architecture/index.md b/.okf/architecture/index.md index 0d80ce3..1af1729 100644 --- a/.okf/architecture/index.md +++ b/.okf/architecture/index.md @@ -1,5 +1,6 @@ # Architecture - [Scaffold lifecycle](scaffold-lifecycle.md) describes service ownership, planning, Apply, previews, and Finalize. +- [Repository state authority](plan-apply-repository-state.md) records the implemented Plan to Apply state contract. - [Catalog architecture](catalog.md) describes definitions, lookup, and contributions. - [Catalog ID naming](catalog-id-semantics.md) records human-facing naming conventions. diff --git a/.okf/architecture/plan-apply-repository-state.md b/.okf/architecture/plan-apply-repository-state.md new file mode 100644 index 0000000..8ab00df --- /dev/null +++ b/.okf/architecture/plan-apply-repository-state.md @@ -0,0 +1,73 @@ +--- +type: Decision +title: Repository state authority for Plan and Apply +description: Agreed state, freshness, and publication rules for the Plan to Apply handoff. +status: stable +sources: + - id: plan + resource: ../../packages/domain/src/Plan.ts + title: Plan contract + - id: apply + resource: ../../packages/domain/src/Apply.ts + title: Apply contract + - id: capture + resource: ../../packages/scaffold/src/service/plan/RepoSnapshotService.ts + title: Repository text capture + - id: baseline + resource: ../../packages/scaffold/src/service/plan/RepositoryStateService.ts + title: Repository baseline and freshness checks + - id: planning + resource: ../../packages/scaffold/src/service/plan/PlanService.ts + title: Plan construction + - id: execution + resource: ../../packages/scaffold/src/service/apply/ApplyService.ts + title: Apply behavior + - id: writing + resource: ../../packages/scaffold/src/service/apply/WriteEngine.ts + title: Host writes +generated: { by: codex, at: "2026-09-27T09:22:00+00:00" } +--- + +# Repository state authority for Plan and Apply + +Status: Implemented for Plan, preview, and Apply. + +## Problem + +Without a repository baseline, Apply could read newer contents when composing a file or overwrite a file that changed after planning. Preview could show a different host state from the one Plan classified. See the [scaffold lifecycle](scaffold-lifecycle.md "describes implemented behavior"). + +`Plan.baseline` now stores the canonical root and the state of inspected paths. `RepositoryStateService` fingerprints existing text files and compares their current state with the baseline. `PlanService` checks the capture again before returning. Apply and both preview paths check the baseline before reading or writing. Apply checks each target and its ancestors again before writing. A stale error reports changed paths and, after partial publication, the paths already written. File preview performs its publication step in a private memory filesystem after checking the source repository. + +## Decision + +### Bind Plan to its repository and captured state + +- A host Plan belongs to one canonical repository root. An alias resolving to that root is valid; another root is invalid even when its relevant files match. Planning must also work before the root exists, using the canonical existing parent and intended path to identify it. +- Plan records the type of every path planning inspected, including missing paths, ancestors, unchanged outcomes, and paths later skipped by an Apply decision. Existing text files use cryptographic content fingerprints. The serializable Plan does not contain copies of existing user files. +- Explicit planning paths remain in scope even when `.gitignore` matches them. Timestamps and permissions alone do not make a Plan stale. A file changed and restored before validation is acceptable when its checked type and contents match the recorded state. +- A symlink within the relevant paths, a non-text file, or a special entry makes planning fail clearly. A symlink used only as an alias to the repository root is resolved to the canonical root. + +### Reject drift before preview or publication + +- Planning rechecks its capture before returning a Plan. This detects a change observed during capture; it does not make the host filesystem transactional. +- Ordinary dry-run, file preview, and Apply reject a stale Plan and require replanning. They must not silently recompose from changed host contents. A Plan is stale after its successful Apply changes the repository. +- Before the first Apply write, compare every captured path with the current repository. Report every detected difference by path and change kind, without file contents. Detected preflight drift causes no Apply writes. +- Recheck each path before its write. If a later check detects drift, stop remaining writes, preserve the changed file, and return a typed stale error with the partial Apply result. The CLI should state what was already written and tell the user to replan. + +### Keep host publication guarded + +Apply's writer owns host publication and baseline checks. A private virtual workspace may hold the selected state during one workflow, while Plan keeps the serializable baseline. A broad tree export cannot replace the guarded writer. Shared workspace creation, seeding, and result capture are a separate design step described in the [staged workspace research](../research/staged-workspace.md "builds on this decision"). + +## Limits + +The final check and the actual write are separate operations. Another process can change a file between them. Writes also remain per file, so an execution failure can leave a partial result. This decision does not promise a repository-wide transaction. + +Configuration written before Plan and commands run after Apply have their own lifecycle contracts. The repository state rule here covers Plan, preview, and Apply, not those separate actions. + +## Evidence required for implementation + +- Modify, delete, or create a relevant path after planning. Preview and Apply must reject the stale Plan; preflight must leave the host unchanged. Cover composed and authoritative outcomes, missing paths, ancestors, unchanged paths, and skipped conflicts. +- Apply the same Plan through a canonical alias and try it against an unrelated root with identical relevant contents. Preserve greenfield planning for a root that does not exist yet. +- Preview an incremental add, edit a host file, then attempt Apply. The host edit must survive. A file restored to its captured state must be accepted. +- Change a later path after preflight but before its write. Apply must stop, preserve that path, report previous writes, and avoid subsequent writes. +- Reject relevant symlinks, non-text files, and special entries during planning. Verify that Plan and stale errors do not expose original file contents. diff --git a/.okf/architecture/scaffold-lifecycle.md b/.okf/architecture/scaffold-lifecycle.md index 7c399dd..a0b626b 100644 --- a/.okf/architecture/scaffold-lifecycle.md +++ b/.okf/architecture/scaffold-lifecycle.md @@ -26,24 +26,24 @@ sources: resource: ../../packages/scaffold/src/index.ts - id: finalize resource: ../../packages/scaffold/src/service/finalize/FinalizeService.ts -generated: { by: codex, at: "2026-09-22T17:40:50+00:00" } +generated: { by: codex, at: "2026-09-27T09:22:00+00:00" } --- # Scaffold lifecycle Selection records user intent. Blueprint resolves dependencies. Plan describes repository-aware outcomes and conflicts. Apply adds explicit decisions only for conflicted paths. VFS state does not replace these domain values. -Plan reads relevant paths and ancestors through `RepoSnapshotService`. This is a selective text view, not a complete VFS snapshot. Plan retains outcomes rather than the full captured baseline. +Plan reads relevant paths, ancestors, and the root through `RepoSnapshotService`. This is a selective text view, not a complete VFS snapshot. Plan retains outcomes and a serializable baseline with the canonical root, path types, and SHA-256 fingerprints of existing text files. It does not retain existing file contents. -Apply prepares composition before writing. For modified composed files, it re-reads current contents. Individual writes validate path state and use temporary-file rename; failures are collected while later writes continue. This does not provide a repository-wide transaction. +Apply checks every baseline path before preparing composition or writing. For modified composed files, it re-reads contents after that check. Each write checks its target and ancestors again, then uses a temporary-file rename. A later stale check stops remaining writes and reports the partial result. Other execution failures are collected while later writes continue. This does not provide a repository-wide transaction. -`ApplyPreviewService` copies changed, non-skipped paths into a fresh `MemoryFileSystem`, runs actual Apply, and returns successful changed files. `RecipePreviewService` plans in another memory filesystem and appends configuration to its result separately. Neither changed-file list claims to contain a complete repository. +`ApplyPreviewService` checks the host baseline, copies every baseline path into a fresh `MemoryFileSystem`, checks the host baseline again, and runs guarded Apply against that private filesystem. It returns successful changed files. `RecipePreviewService` plans in another memory filesystem and appends configuration to its result separately. Neither changed-file list claims to contain a complete repository. -Private previews use VFS `makeCrypto`, which provides reproducible identity generation without a platform Crypto dependency. These volume identities are not security credentials. The public VFS `make` and `layer` APIs require an explicitly supplied Crypto service. +Private previews use VFS `make` with an explicitly supplied Crypto service. Volume identities are not security credentials. -Ordinary dry-run prepares actions without the same virtual write execution. Finalize commands use a process spawner, and the CLI can proceed into Finalize handling after reporting failed Apply paths. +Ordinary dry-run checks the baseline and prepares actions without the same virtual write execution. Finalize commands use a process spawner, and the CLI can proceed into Finalize handling after reporting failed Apply paths. -The [staged workspace research](../research/staged-workspace.md "addresses consistency gaps") proposes a shared foundation. The [validation research](../research/virtual-validation.md "examines host execution") retains the external-tool boundary. +The [repository state decision](plan-apply-repository-state.md "defines the implemented handoff") records the freshness rule. The [staged workspace research](../research/staged-workspace.md "addresses consistency gaps") proposes a shared foundation. The [validation research](../research/virtual-validation.md "examines host execution") retains the external-tool boundary. ## Service ownership diff --git a/.okf/assets/effect-vfs-design-research.html b/.okf/assets/effect-vfs-design-research.html index 9da2f24..42a55d1 100644 --- a/.okf/assets/effect-vfs-design-research.html +++ b/.okf/assets/effect-vfs-design-research.html @@ -298,11 +298,11 @@

Effect VFS in Stack Effect

-

Research review · 22 September 2026 · Discussion input, not an accepted design

+

Research review · 22 September 2026 · Discussion input, not an accepted design

Start with a shared workspace, keep catalog intent explicit

Effect VFS fits Stack Effect best as a shared place to prepare, inspect, and repeat filesystem changes. Stack Effect already uses it for previews. The next useful step is to make the captured repository state and candidate result consistent across planning, decisions, and preview.

For the catalog, prefer immutable snapshots of selected generated recipes, with declarative modules remaining authoritative. An overlay is a private branch of one filesystem snapshot. It does not merge independently generated modules or resolve JSON and TypeScript conflicts. preview: ApplyPreviewService.ts L31–175 overlay: VirtualFileSystem.ts L1973–2035 domain: Catalog.ts L126–223

-

This report recommends exploring these directions, not replacing Selection, Blueprint, Plan, or Apply. No issues or OKF records have been created. The final sections identify questions and evidence that can support that later work.

+

This report recommends exploring these directions, not replacing Selection, Blueprint, Plan, or Apply. The repository state decision and focused research concepts record the current direction.

What exists today

Selection expresses user intent. Blueprint resolves dependencies. Plan assesses proposed changes against repository contents. Apply combines those outcomes with explicit conflict decisions and executes them. Finalize runs commands derived from the Blueprint and configuration. These boundaries remain useful with a virtual filesystem. plan: PlanService.ts L44–164 apply: ApplyService.ts L268–400 finalize: FinalizeService.ts L107–212

The current filesystem integration has two layers:

@@ -378,12 +378,7 @@

Effect VFS in Stack Effect

The diagram describes a proposal. VFS supplies the private candidate workspace. Stack Effect still owns the Plan, decisions, host checks, publication, and Finalize policy.

A bounded first version can capture only planned paths and the ancestors needed to reproduce obstructions. Its result must state that scope. A consumer needing a complete tree must seed and capture a complete supported tree rather than treat a changed-file list as one. Decide how to represent bytes, symlinks, modes, directories, and exclusions before broadening beyond the current text-file workflow. reads: RepoSnapshotService.ts L19–88 preview: ApplyPreviewService.ts L31–175

Conflict experiments then become repeatable. Start two candidates from the same baseline, apply skip in one and override in the other, and compare their resulting files. Replacing either candidate leaves the baseline intact. The JSON and TypeScript composers still decide what each accepted action means. overlay: VirtualFileSystem.ts L1973–2035 apply: ApplyService.ts L268–400

-

The public behavior decision is what to do when the host changes after review:

- -

In either policy, publish previously reviewed bytes only when the relevant baseline preconditions still hold. There is still a check-to-write race unless the publication design addresses it. A content hash check alone does not create a host transaction. Wrong-repository detection, newly created paths, deleted files, ancestor changes, and partial publication all need stated outcomes. Existing write behavior should change only through a deliberate contract decision. write: WriteEngine.ts L44–201 apply: ApplyService.ts L268–400

+

The repository state decision now requires a new Plan and review when relevant host state changes. It also defines wrong-root detection, newly created paths, deleted files, ancestor changes, and partial publication. A content fingerprint check alone does not create a host transaction. write: WriteEngine.ts L44–201

Direction 2: use snapshots for generated artifacts

Recommended independent discovery track. Store snapshots of concrete recipes or validated reference projects. Create live volumes only when opening or modifying those artifacts. This avoids keeping a live volume for every catalog variation.

A generated artifact can support browser previews, reproductions, example downloads, or a cache. The first consumer should decide the format's completeness and lifecycle stage. A pre-Finalize source tree has a much smaller scope than a project with installed dependencies, generated lockfiles, and formatter output. Current previews synthesize configuration separately; current catalog workspace generation also runs host Finalize commands after applying files. recipe: RecipePreviewService.ts L49–117 authoring: catalog.ts L609–645

@@ -455,41 +450,37 @@

Effect VFS in Stack Effect

Injected Effect FileSystem does not redirect node:fs, package managers, native extensions, or spawned commands. The VFS Vite demo uses an explicit plugin with a narrow supported profile. It does not demonstrate arbitrary builds, formatters, or type-checkers working in memory. guide: overlay-filesystems.mdx L75–136 virtual-build: VirtualBuild.ts L13–97

For generated-project validation, a practical first option is to export supported files to a temporary real directory, run declared commands, and clean it up. If command output becomes part of the artifact, import and capture it under a defined policy. Do not describe that as process isolation or rollback of external effects.

The CLI currently continues into Finalize handling after reporting failed Apply paths. Configuration writing is also separate. The research exposes decisions about whether failures should stop Finalize and which generated configuration belongs in staged output. They are lifecycle questions, not responsibilities to add to the low-level workspace facility. pipeline: ScaffoldPipeline.ts L184–348 config: init.ts L330–350

-

Existing issues already cover much of the foundation

-

The following issues were checked live on 22 September 2026 and were all open. Their state may change after this report.

-
+
Existing roadmap issues
- + - - - - - - + + - - + + - - + + - - + + - - + +
Related repository concepts
Existing issueConcept Relationship to this research
#175: repository state represented by PlanOwns drift, repository authority, and preview consistency
#250: reusable in-memory workspaceOwns setup, seeding, binding and capture; blocked by #175Repository state authorityAccepted drift, repository identity, and preview contract
#253: materialized catalog combination testsExercises combinations and incremental add after #250Staged workspace lifecycleProposed setup, seeding, binding, and capture after state authority
#251: in-memory Vite validationNarrow validation experiment after #250; keeps host validationVirtual validationCandidate structural tests and a bounded build experiment
#252: portable generated-workspace artifactIndependent discovery; needs a concrete consumer and owned versioned schemaGenerated workspace artifactsOpen completeness and compatibility requirements
#249: Community Catalog foundationTrusted additive declarative fragments; distribution and fragment Finalize excludedCompiled catalog profilesRelationship between concrete snapshots and declarative fragments
-

The next issue-writing round should refine and connect these rather than duplicate them. The less-developed question is how compiled artifacts relate to declarative fragments, parametrization, provenance, and upgrade identity.

+

The less-developed question is how compiled artifacts relate to declarative fragments, parametrization, provenance, and upgrade identity.

Experiments that would settle the design

  1. Workspace parity. Run one realistic incremental-add scenario against host and VFS. Include existing user fields, a skipped conflict, an ancestor obstruction, and two modules changing one shared file. Compare outcomes and bytes; prove preview leaves the host untouched.
  2. @@ -514,7 +505,7 @@

    Effect VFS in Stack Effect

    The evidence currently supports extending the existing workspace integration. It does not yet establish a performance case for a registry, arbitrary module overlay composition, or atomic host application.

Evidence and compatibility

-

Primary evidence is the local source, tests, and documentation in both maintained repositories, plus live read-only GitHub issue inspection. Historical memory was used to locate existing roadmap work, then issue state and relevant implementation were checked again.

+

Primary evidence is the local source, tests, and documentation in both maintained repositories.