+
Abstract
+
+ An agentic benchmark returns one number per episode, and that number contains at least three
+ separable things: what the agent can do, how the world was presented to it, and the harness it
+ ran inside. In a fixed world, none of the three can be varied independently of the others. So
+ a model that never worked out what an object does, a model that had the right plan and drifted
+ mid-execution, and a model that took the distractor bait all arrive as the same failed episode
+ — even though they are different problems with different fixes.
+
+
+ MultiNet v2.0 is built so those things stay separable. The task is an agent in a
+ 2D grid with walls, keys, doors, switches, gates and distractors, and a goal tile.
+ There are six actions and the mechanics are never explained. Nothing is explained
+ because prior domain knowledge is the confound the design removes: a failure here cannot be
+ attributed to an unfamiliar API, library or interface, because there is none.
+
+
+ This release is the first measurement on that substrate, in its most basic
+ rendering. We evaluate three frontier models — Claude Opus 4.8, Kimi k2.6 and Qwen3.6-27B
+ — on 50 generated mazes, 150 episodes in total, under a single evaluation protocol chosen by
+ a controlled 540-episode sweep rather than by intuition. Frontier performance is near zero:
+ 6 solves in 150 episodes, and 45 of the 50 mazes were never solved by any
+ model. Models operate the mechanisms they already understand — 27 successful pickups, 15
+ doors opened, and 4 of the 6 solves on key-door mazes — and never work out the one they would
+ have to learn during the episode: 132 TOGGLE actions produced 2 switch flips, and agents stood on a
+ live switch in 38 episodes without pressing it.
+
+
+ This release is a substrate and design thesis plus a first measurement. Because the representation
+ is projectable, it is also the first step toward A/B testing modality, complexity and other
+ downstream variables against the same underlying puzzle.
+
+
MultiNet
- 
MultiNet
-
-
-