Skip to content

agentHost: centralize session list metadata in sessions_v2 - #332410

Draft
Sandeep Somavarapu (sandy081) wants to merge 4 commits into
mainfrom
sandy081/agents/session-db-data-migration-plan
Draft

agentHost: centralize session list metadata in sessions_v2#332410
Sandeep Somavarapu (sandy081) wants to merge 4 commits into
mainfrom
sandy081/agents/session-db-data-migration-plan

Conversation

@sandy081

Copy link
Copy Markdown
Member

Summary

This change introduces an independent, host-wide sessions_v2 registry/catalog for Agent Host session and chat list metadata.

The current session list path reconstructs each entry by enumerating providers and opening individual per-session databases. That makes startup/listing dependent on provider availability and causes work proportional to the number of sessions. It also spreads host-owned list metadata across provider state and many session.db files.

sessions_v2 stores one bounded, verified row per session so current builds can list eligible sessions centrally. Turns, messages, drafts, annotations, detailed changesets, and opaque provider backing remain in session.db and continue to load only when a session or chat is opened.

What changed

Central session catalog

  • Adds an independent sessions_v2 registry/catalog to agent-host.db.
  • Stores registry identity plus bounded list-visible metadata:
    • title/source, modified time, read/archive state;
    • project and ordered working directories;
    • Git, GitHub, source-control, changes, artifacts, and orchestration summaries;
    • adoptable/adopted legacy provenance and chat-backing eligibility;
    • ordered default/peer chat membership, title/source, and origin.
  • Uses projection-versioned canonical JSON and SHA-256 hashes for deterministic verification.
  • Keeps the predecessor sessions registry as an optional compatibility source rather than a prerequisite.

Direct migration

  • Unions existing sessions_v2, optional predecessor sessions rows, and provider/SDK discovery by stable session URI.
  • Migrates provider-only and legacy-only sessions directly to verified v2 rows.
  • Uses projection-versioned per-provider markers independent of predecessor migration markers.
  • Supports partial old/current migrations, provider failures, and resumable per-session progress.
  • Persists durable, revivable exclusions for backing, subagent, stale, and provider-absent sessions.

Cross-version compatibility

  • Current runtime mutations are authoritative in v2 and atomically mirror identity/provenance to sessions during the cooling window.
  • Direct migration remains v2-only and does not run the intermediate data migration first.
  • Returning from an intermediate build imports legacy-only additions and reconciles resolved identity changes.
  • Legacy-row absence is not treated as deletion; shared tombstones are the durable cross-version delete signal.
  • Explicit coverage includes Old → New → Intermediate → New and repeated cycles without duplicate rows or incarnation churn.

Crash recovery and reconciliation

  • Adds a compact per-session synchronization receipt.
  • Writes downgrade-compatible metadata and a pending canonical payload before the central upsert.
  • Clears the payload after exact acknowledgement, leaving only generation/revision/version/hash in steady state.
  • Serializes writes per session, replays interrupted updates, and repairs old-build metadata changes.
  • Runs bounded, resumable reconciliation with cancellation-safe scheduling.

Read rollout

  • Adds internal legacy, shadow, centralWithFallback, and central modes.
  • Shadow validation is non-blocking and reports privacy-safe aggregate mismatch categories.
  • Central-with-fallback chooses per session: verified current rows use the central catalog; stale/missing rows retain the legacy path and schedule repair.
  • Eligible central rows require no provider metadata call and no per-session database open.

Code organization

  • Extracts focused helpers for:
    • canonical catalog projection;
    • source/legacy metadata resolution;
    • dual-write synchronization;
    • reconciliation;
    • shadow validation;
    • central list conversion;
    • direct v2 migration;
    • downgrade-compatible peer-chat persistence.

This reduces the migration-specific growth in AgentService by roughly 482 lines compared with the initial implementation.

Why

  • Faster and more predictable session listing.
  • Session lists remain available when a provider is temporarily unavailable.
  • Avoids opening every session.db during normal verified listing.
  • Gives host-owned metadata one authoritative current representation.
  • Preserves downgrade compatibility without forcing direct upgrades through every intermediate data migration.
  • Makes crash recovery, partial migration, and cross-version reconciliation explicit and testable.

Compatibility and safety

  • Published database migrations remain forward-only; a new migration removes the previously introduced v2→legacy foreign key without rewriting historical migrations.
  • Tombstones, session incarnations, source revisions, and hashes reject stale or resurrecting writes.
  • Current and predecessor migration markers remain independent.
  • Pending payloads are retained only until acknowledgement.
  • Malformed legacy peer-chat metadata is tolerated and self-healed.
  • Runtime rollback can select the legacy read path without schema rollback.

Validation

  • npm run compile
  • repository pre-commit hygiene
  • npm run typecheck-client
  • npm run valid-layers-check
  • targeted ESLint for changed Agent Host files
  • 575 focused Agent Host tests passing, 16 expected pending
  • full AgentService suite included in the focused matrix
  • direct old → new → intermediate → new regression coverage

Reviewer notes

The production default remains conservative; central read modes are internal rollout controls. The larger diff is primarily the database migration/compatibility contract and focused failure/recovery tests rather than UI behavior.

Add a backward-compatible sessions_v2 catalog, legacy-first synchronization receipts, reconciliation, shadow validation, central fallback reads, and durable chat metadata while retaining open-only content in per-session databases.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adapt the sessions_v2 catalog and reconciliation work to the latest Agent Host composition, adoption, metadata, and test infrastructure changes.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Make sessions_v2 an independent current registry, import directly from current, legacy, and provider sources, mirror runtime identities for downgrade compatibility, and reconcile cross-version changes with durable exclusions and versioned markers.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move catalog source resolution and downgrade-compatible peer chat persistence out of AgentService into focused helpers without changing migration or runtime behavior.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 24, 2026 19:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a centralized, verified sessions_v2 catalog for efficient Agent Host session listing, migration, downgrade compatibility, and recovery.

Changes:

  • Adds catalog schemas, canonical projections, synchronization receipts, migration, reconciliation, and shadow validation.
  • Integrates central listing with session mutations, peer chats, Git state, titles, and legacy adoption.
  • Adds extensive persistence, compatibility, recovery, and rollout tests.
Show a summary per file
File Description
src/vs/platform/agentHost/common/agent.ts Extends adoption results with recovered list metadata.
src/vs/platform/agentHost/common/sessionDataService.ts Defines catalog synchronization receipt APIs.
src/vs/platform/agentHost/node/agentHostBootstrap.ts Wires catalog persistence into bootstrap.
src/vs/platform/agentHost/node/agentHostCatalogListReader.ts Converts verified catalog rows into list metadata.
src/vs/platform/agentHost/node/agentHostCatalogProjection.ts Implements canonical bounded projections and hashing.
src/vs/platform/agentHost/node/agentHostCatalogReconciliationService.ts Repairs interrupted or stale synchronization.
src/vs/platform/agentHost/node/agentHostCatalogShadowValidator.ts Compares central and legacy listing results.
src/vs/platform/agentHost/node/agentHostCatalogSourceResolver.ts Resolves canonical catalog source metadata.
src/vs/platform/agentHost/node/agentHostCatalogSyncService.ts Coordinates local-first catalog synchronization.
src/vs/platform/agentHost/node/agentHostDatabase.ts Adds sessions_v2, exclusions, and compatibility migrations.
src/vs/platform/agentHost/node/agentHostGitStateService.ts Persists Git summaries through the catalog path.
src/vs/platform/agentHost/node/agentHostPeerChatStore.ts Persists downgrade-compatible peer-chat membership.
src/vs/platform/agentHost/node/agentHostServices.ts Registers catalog-aware persistence dependencies.
src/vs/platform/agentHost/node/agentHostSessionTitleController.ts Routes title metadata through catalog synchronization.
src/vs/platform/agentHost/node/agentHostSessionsV2MigrationService.ts Implements direct, resumable v2 migration.
src/vs/platform/agentHost/node/agentService.ts Integrates migration, listing, synchronization, and peer chats.
src/vs/platform/agentHost/node/agentServiceComposition.ts Supplies catalog persistence callbacks.
src/vs/platform/agentHost/node/agentSessionRegistry.ts Adds dual-registry compatibility behavior.
src/vs/platform/agentHost/node/agentSideEffects.ts Persists list-visible state changes centrally.
src/vs/platform/agentHost/node/copilot/copilotAgent.ts Updates legacy adoption and metadata recovery.
src/vs/platform/agentHost/node/localCommands/localChatCommand.ts Extends coordinated metadata persistence.
src/vs/platform/agentHost/node/localCommands/renameLocalCommand.ts Persists title and title-source metadata together.
src/vs/platform/agentHost/node/sessionCoordination.ts Integrates catalog-aware session coordination.
src/vs/platform/agentHost/node/sessionDatabase.ts Adds synchronization snapshot storage and transactions.
src/vs/platform/agentHost/test/common/sessionTestHelpers.ts Extends test database helpers for receipts.
src/vs/platform/agentHost/test/node/agentHostCatalogListReader.test.ts Tests central row conversion and eligibility.
src/vs/platform/agentHost/test/node/agentHostCatalogProjection.test.ts Tests canonicalization, limits, and verification.
src/vs/platform/agentHost/test/node/agentHostCatalogReconciliationService.test.ts Tests repair and interrupted-write recovery.
src/vs/platform/agentHost/test/node/agentHostCatalogShadowValidator.test.ts Tests shadow mismatch classification.
src/vs/platform/agentHost/test/node/agentHostCatalogSourceResolver.test.ts Tests source and legacy metadata resolution.
src/vs/platform/agentHost/test/node/agentHostCatalogSyncService.test.ts Tests synchronization ordering and recovery.
src/vs/platform/agentHost/test/node/agentHostDatabase.test.ts Tests v2 schema and compatibility behavior.
src/vs/platform/agentHost/test/node/agentHostGitStateService.test.ts Tests catalog-backed Git persistence.
src/vs/platform/agentHost/test/node/agentHostPeerChatStore.test.ts Tests peer-chat storage and malformed data handling.
src/vs/platform/agentHost/test/node/agentHostServices.test.ts Tests service dependency registration.
src/vs/platform/agentHost/test/node/agentHostSessionTitleController.test.ts Tests coordinated title persistence.
src/vs/platform/agentHost/test/node/agentHostTurnTelemetry.test.ts Updates telemetry fixtures for persistence wiring.
src/vs/platform/agentHost/test/node/agentService.test.ts Tests rollout modes, migration, compatibility, and adoption.
src/vs/platform/agentHost/test/node/agentServiceTestUtils.ts Extends AgentService database test doubles.
src/vs/platform/agentHost/test/node/agentSessionRegistry.test.ts Tests dual-registry session lifecycle behavior.
src/vs/platform/agentHost/test/node/agentSideEffects.test.ts Tests catalog persistence from state changes.
src/vs/platform/agentHost/test/node/copilotAgent.test.ts Tests legacy adoption metadata behavior.
src/vs/platform/agentHost/test/node/sessionCoordination.test.ts Tests catalog-aware coordination behavior.
src/vs/platform/agentHost/test/node/sessionDatabase.test.ts Tests receipt migration and atomic persistence.
src/vs/sessions/contrib/providers/agentHost/AGENT_HOST_SESSIONS_PROVIDER.md Documents the centralized catalog architecture.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 44/45 changed files
  • Comments generated: 7
  • Review effort level: Balanced

legacyMetadata: { customTitle: 'one', [META_GIT_STATE]: '{"branch":"first"}' },
});

test('advances changed content beyond a newer local pending revision after central failure', async () => {
repairs: 1,
});

test('validates central-only rows against durable top-level eligibility', async () => {
...(peerChatOrigin !== undefined ? { origin: peerChatOrigin } : {}),
},
];
this._catalogSyncSuppressedSessions.add(sessionKey);
Comment on lines +3311 to +3316
// The host owns list-visible state (title / read): report it back so it
// lands in the session catalog and the per-session database together,
// instead of writing a half of it here.
const listVisible = customTitle !== undefined
? { title: customTitle, titleSource: 'user' as const, isRead: true }
: { isRead: true };
return { status: 'incomplete' };
}

const newlyRegistered = !effectiveCandidate.current;
Comment on lines +3189 to +3193
const catalogChats = [
...existingCatalogChats,
{
uri: chat.toString(),
kind: 'peer' as const,
Comment on lines +3336 to +3339
[customChatTitleSourceMetadataKey(chatKey)]: '',
},
this._catalogChatsFromState(state).filter(candidate => candidate.uri !== chatKey),
);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants