Skip to content

Load only requested demos during fallback conversion - #64

Open
eust-w wants to merge 1 commit into
NeuracoreAI:masterfrom
eust-w:upstream/lazy-demo-loading
Open

Load only requested demos during fallback conversion#64
eust-w wants to merge 1 commit into
NeuracoreAI:masterfrom
eust-w:upstream/lazy-demo-loading

Conversation

@eust-w

@eust-w eust-w commented Aug 6, 2026

Copy link
Copy Markdown

Summary

  • select and limit fallback demo paths before deserializing safetensors
  • preserve the existing random sampling behavior
  • deserialize each selected demo immediately before conversion
  • add regression coverage proving that a two-demo request loads only two files

Why

When the requested frequency or observation mode is not cached, DemoStore
currently loads every source demo before applying amount. Long-horizon pixel
demonstrations can therefore exhaust memory even when the caller requests only
one or two episodes.

Applying the existing limit to paths first bounds deserialization and conversion
work by the requested amount without changing the public API or output type.

Compatibility

Negative amount values still load all demos, zero still returns immediately,
requests larger than the available archive still raise
TooManyDemosRequestedError, and positive requests retain random selection.

Validation

  • pre-commit run --files demonstrations/demo_store.py tests/test_demo_store.py CHANGELOG.md
  • pytest -q tests/test_demo_store.py (13 passed)
  • git diff --check

Supersedes #59 after the source branch was renamed to remove the agent/ prefix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant