Skip to content

Make Slack E2E assertions follow the intended behavior - #982

Open
ReganBell wants to merge 2 commits into
mainfrom
codex/live-e2e-test-intent
Open

Make Slack E2E assertions follow the intended behavior#982
ReganBell wants to merge 2 commits into
mainfrom
codex/live-e2e-test-intent

Conversation

@ReganBell

@ReganBell ReganBell commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

The live Slack catalog should fail when the requested behavior fails, rather than because an otherwise valid response omits a synthetic marker, uses the channel top level, or follows an interim progress message. The file fixture uploader also wrapped a 1,336-byte PNG in a 1,556-byte multipart payload that a raw upload endpoint could store as image data.

  • Judge the requested channel answers without incidental marker echoes; retain strict checks when threading is explicitly requested. Exclude stale, human, status, and unrelated-thread messages while accepting top-level thread parents.
  • Make the three-person prompt explicit about its channel destination and use a stable factual audience question. Accept a real acceptance draft even when commentary acknowledges the earlier conflicting request; declines, missing drafts, and claimed sends still fail.
  • Seed the history scenario with ordinary conversation and an unpredictable launch name. Wait through progress messages for that fact; approval refusals and missing facts do not pass.
  • Upload exact file bytes through the supported external-upload protocol, preserving share metadata.
  • Retry transient Arga status GET 5xx responses within one timeout budget. Never replay provisioning POSTs, and still require confirmed cleanup.

Validation: 14 affected tests pass; typecheck, ESLint, and upstream CI pass. Independent review and its follow-up are complete. Regression coverage includes exact PNG bytes, thread metadata, channel/thread selection, delayed history answers, transient and persistent cleanup failures, and no POST replay. Live Arga replays have passed all four originally affected cases; the final history replay passed in 61 seconds. The full final catalog is still queued/running. Alerts and release gates remain enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant