Skip to content

fix(envs): serve each environment's own State subclass on /schema and /state - #1198

Open
k21993 wants to merge 6 commits into
huggingface:mainfrom
k21993:feat/env-state-cls
Open

k21993 wants to merge 6 commits into
huggingface:mainfrom
k21993:feat/env-state-cls

Conversation

@k21993

@k21993 k21993 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

#1174 added state_cls to the app factories, but nothing passes it yet, so the fix is not reaching users. All 28 envs that declare a State subclass still publish a state schema of episode_id and step_count, and still strip every subclass field from the /state body, while their WebSocket state frame returns the full object. This wires each env to its own class and adds a static regression test so a new env cannot forget.

Follow-up to #1174, which closed #1155.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • New environment
  • Refactoring

Alignment Checklist

  • I have read .claude/docs/PRINCIPLES.md and this PR aligns with our principles
  • I have checked .claude/docs/INVARIANTS.md and no invariants are violated
  • I have run lint and tests and addressed all issues

uv is not on my shell PATH, so I ran the checks in a Python 3.12 virtualenv with the test dependencies rather than through .claude/hooks/lint.sh.

RFC Status

  • Not required (bug fix, docs, minor refactoring)

No API changes. Every edit is an argument at an existing call site.

Test Plan

New file tests/envs/test_env_state_cls_wiring.py: for every env that declares a State subclass, assert its app factory passes state_cls, that the value is a subclass declared in that env, and that the name is imported in server/app.py. It walks the source with ast rather than importing, so playwright, carla, dm_control and the rest of the optional-dependency tail stay out of it and the check runs everywhere. A guard case fails if the discovery ever matches nothing.

  • New test: 29 passed.
  • Reverting the envs/ changes and rerunning it fails 28 of the 29, one per env, which is the point of the test.
  • tests/core: 641 passed, 2 skipped. tests/envs/test_manifest_app_targets.py: 78 passed.
  • Every changed file passes ruff format --check and compiles.
  • usort reports the same 10 pre-existing envs/**/app.py files before and after this change, so nothing here adds to that drift. I left those alone rather than reformatting files this PR only touches by one line.

Notes for review

Three cases needed care rather than a blanket edit:

  • thinkingbox_env builds HTTPEnvServer directly and registers routes in production mode, where /state is not registered at all. It still gets state_cls because /schema is served in production and was publishing the base model.
  • Six envs (coding_tools_env, finqa_env, jupyter_env, opencode_env, pi_env, terminus_env) take CallToolObservation from openenv.core, so their State import comes from the env's own models module and follows each file's existing in-repo and standalone import pattern.
  • wildfire_env builds its state with WildfireState.model_construct, and opencode_env, pi_env and jupyter_env bind the class to an instance attribute first. All four still return their declared subclass, which is what the endpoints now serve.

Envs that do not declare a State subclass are untouched and keep the base model.


Note

Low Risk
Mechanical wiring at existing factory call sites with no API signature changes; risk is limited to incorrect state_cls mapping, which the new AST and Harbor tests guard against.

Overview
Fixes inconsistent state over HTTP vs WebSocket by passing each environment’s *State subclass into create_app (or HTTPEnvServer for ThinkingBox) via state_cls, so /schema and /state include the same fields WebSocket clients already get instead of only episode_id and step_count.

The change is applied across 28 env server/app.py files plus Harbor (build_app). repl_env and textarena_env only set state_cls when inspect.signature(create_app) supports it, leaving legacy fallback paths unchanged.

Regression coverage: new tests/envs/test_env_state_cls_wiring.py statically checks every env that declares a State subclass; Harbor gets an HTTP integration test asserting custom fields (e.g. llm_url) appear on /schema and /state.

Reviewed by Cursor Bugbot for commit e438c49. Bugbot is set up for automated code reviews on this repo. Configure here.

huggingface#1174 added state_cls to the app factories, but nothing passes it, so the 28
envs that declare a State subclass still publish a state schema of episode_id
and step_count and still strip every subclass field from the /state body. Their
WebSocket state frame returns the full object, so the two transports disagree.

Pass each env's State subclass at its app factory call site, and add a static
regression test that walks envs/ with ast, so a new env cannot quietly skip it.

Follow-up to huggingface#1174.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread envs/repl_env/server/app.py Outdated
Comment thread tests/envs/test_env_state_cls_wiring.py Outdated
repl_env and textarena_env pick between two create_app calls by probing
inspect.signature(create_app). The first branch is the one that runs against a
current openenv; the second exists for a release predating gradio_builder. The
previous commit wired only the second, so /schema and /state still served the
base State model for these two envs.

Pass state_cls in the live branch, guarded by the same signature probe the file
already uses for its other newer kwargs, and drop it from the legacy branch,
where an openenv old enough to take that path would reject the argument.

The test inspected only the last factory call in a module, which is why it did
not catch this. It now checks every call, resolves values passed through a
splatted kwargs dict, and exempts calls inside a signature-guarded fallback.
@k21993

k21993 commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Both findings were right, thanks. repl_env and textarena_env select between two create_app calls with inspect.signature, and I had wired only the legacy fallback, so the branch that actually runs still served the base State model.

Fixed in 1a84a1b0: the live branch passes state_cls behind the same signature guard the file already uses for its other newer kwargs, and the legacy branch no longer passes it at all. Both envs pin openenv>=0.2.2, so an installation old enough to take that path would reject the argument outright.

The test had the same blind spot, which is why it stayed green through the first commit. It now checks every factory call in a module rather than the last one, resolves values passed through a splatted kwargs dict, and exempts only calls inside a signature-guarded fallback. Removing the guard from repl_env's live branch now fails that env's case.

@cursor cursor Bot mentioned this pull request Sep 19, 2026
21 tasks
cursor[bot]
cursor Bot approved these changes Sep 21, 2026
@burtenshaw

Copy link
Copy Markdown
Collaborator

cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REQUEST_CHANGES at 3b085d82 (Ben ask: cursor review)

The 28 env app.py wirings + repl/textarena branch logic + AST test are correct — 29/29 wiring tests pass. One completeness gap remains:

Tier 1 — Harbor still unwired

src/openenv/harbor/serving.py build_app() still calls create_app(HarborEnvironment, …) without state_cls=HarborState. HarborState lives under src/openenv/harbor/models.py, so the test’s envs/-only scanner finds no State subclass and silently skips Harbor. Live Harbor /schema and /state still drop the custom fields.

This PR does not touch Harbor, but the title/claim is “each environment’s own State subclass.” Please either:

  1. Wire state_cls=HarborState in build_app() and extend the scanner to src/openenv/<name>/ (or an allowlist), or
  2. Explicitly carve Harbor out of the claim with a tracked follow-up.

Not 0.6.0 wheel cargo (envs/** + test only; Harbor fix would be package cargo if included). Branch was refreshed onto main 228bd147.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@burtenshaw

Copy link
Copy Markdown
Collaborator

Addressed the Harbor review in da82e56. build_app() now passes state_cls=HarborState. Added a focused real-HTTP regression for both /schema and /state in the existing Harbor serving tests; the generic scanner is unchanged.

The new test failed before the fix because all six Harbor-specific fields were missing. After the fix, the Harbor serving, environment wiring, and core state-schema suites passed: 46 tests. Touched-file usort, Ruff format/check, and git diff --check passed. The fix includes src/openenv/harbor/serving.py, so it is package code as well as environment wiring.

Fork workflows still require approval to run; no CI approval or rerun was performed.

@burtenshaw

Copy link
Copy Markdown
Collaborator

cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit da82e56. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at da82e564

Prior Harbor blocker is fixed: build_app() now passes state_cls=HarborState, and test_harbor_app_serves_its_own_state_class asserts /schema + /state retain Harbor fields over HTTP (llm_url observed).

Validation:

  • tests/envs/test_harbor_hosted_serving.py + test_env_state_cls_wiring.py: 38/38 pass
  • lint/format clean on the touched files

Because this changes src/openenv/harbor/serving.py, treat as 0.6.0 package cargo if merged (refresh draft #1211 after green exact-head CI). Still needs fork Approve-and-run / repository checks before merge.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at 7a967005 (merge-main only)

Rebased onto main after #1178. Feature tip unchanged from prior APPROVE at da82e564: Harbor build_app() still passes state_cls=HarborState (serving.py). 0.6.0 package cargo if merged — refresh draft #1211 after green exact-head CI.

Still needs fork Approve-and-run / repository CI.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE at e438c498 (merge-main only)

Rebased onto main after #1219. Feature unchanged: Harbor state_cls=HarborState wiring remains.

Package cargo if merged (would require #1211 refresh/TestPyPI). Fork may still need Approve-and-run.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Release

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

/schema and /state always use the base State model, dropping every field an environment's State subclass declares

2 participants