Skip to content

test: add materialized-workspace contract tests for catalog combinations #253

Description

@lloydrichards

Context

Catalog validation proves that definitions and dependencies are valid, but some failures appear only after modules are combined and applied. Existing generated-workspace tests cover important journeys, but they are slower and depend on host directories, subprocesses, and external tools.

Benefits

An in-memory contract-test harness would let us verify the repository the catalog claims to generate while still exercising the production Blueprint, Plan, composition, and Apply behavior.

Failures could report the selection, starting fixture, changed paths, and relevant contents without leaving temporary repositories behind.

Expected changes

  • Add a reusable scaffold integration-test helper that starts from a repository fixture and materializes a selection in isolation.
  • Cover representative module combinations rather than attempting every possible combination.
  • Include incremental additions, preservation of user-owned content, and deterministic generation.
  • Test idempotency by rebuilding the Plan from the first resulting workspace before applying again.
  • Retain host-filesystem E2E tests for Git, package managers, subprocesses, and platform behavior.

This complements #181's real-filesystem lifecycle contract test. This issue owns broad, fast coverage of representative built-in catalog combinations and their final repository trees; it does not replace the host-backed vertical test or own general workflow orchestration.

The harness should live with scaffold integration testing. The catalog package should remain typed, deterministic data rather than gaining a filesystem dependency. Contributed catalog fragments can be added only if #249 establishes how they compose.

Done when

  • Tests materialize a multi-module create journey without host filesystem writes.
  • Tests cover at least one incremental-add journey.
  • Replanning and applying an unchanged generated workspace has an explicit expected result.
  • Unexpected final file changes produce a useful test failure.
  • The responsibilities retained by host-filesystem E2E tests are documented.

Depends on

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions