Proposal: EvalPort adapter to export ExpResult data as portable eval results

[adhabnr-ux]

EvalPort (https://github.com/adhabnr-ux/evalport) is a small open interchange spec + SDK (evalport-sdk on PyPI) for portable LLM/agent eval datasets and results, so results produced by one framework can be validated and read by tooling that doesn't know that framework's internals.

I read through browsergym/experiments/src/browsergym/experiments/loop.py and think BrowserGym's ExpResult is a genuinely good fit, not just eval-adjacent:

• get_exp_result(exp_dir) / yield_all_exp_results(savedir_base) already give you a per-task-run iterator over a benchmark run.
• ExpResult.summary_info is a real structured per-episode record: n_steps, cum_reward, cum_raw_reward, terminated, truncated, err_msg, stack_trace, plus flattened stats.* (aggregated StepInfo.stats, itself built from AbstractBrowserTask.validate()'s (reward, done, message, info) return).
• ExpResult.exp_args (ExpArgs → agent_args, env_args.task_name, env_args.task_seed) gives a stable per-run identifier.

That maps cleanly onto EvalPort's per-test-case result model: one ExpResult -> one result record (id from task_name/task_seed, pass/fail from terminated + absence of err_msg, score from cum_reward, everything else as metadata).

Sketch of what a zero-footprint browsergym-openeval-adapter package would let you do:

# today
from browsergym.experiments import yield_all_exp_results

for exp_result in yield_all_exp_results("./results/my_run"):
    summary = exp_result.summary_info   # n_steps, cum_reward, terminated, truncated, err_msg, stats.*
    args = exp_result.exp_args          # env_args.task_name, env_args.task_seed
    print(args.env_args.task_name, summary["cum_reward"], summary.get("terminated"))

# with an adapter
from browsergym_openeval_adapter import results_to_openeval
from openeval.validate import validate_result_set

result_set = results_to_openeval(yield_all_exp_results("./results/my_run"))
validate_result_set(result_set)
# each ExpResult becomes one OpenEval result:
#   id       = f"{args.env_args.task_name}_{args.env_args.task_seed}"
#   passed   = bool(summary.get("terminated")) and not summary.get("err_msg")
#   score    = summary.get("cum_reward")
#   metadata = {"n_steps": ..., "cum_raw_reward": ..., "stack_trace": ..., **stats}

This would let anyone who runs BrowserGym benchmarks (MiniWoB, WebArena, WorkArena, etc.) drop their savedir_base results into any OpenEval-consuming tool without writing custom parsing against the pickle/json layout.

Happy to build this myself and open it as a PR (adapters/browsergym-openeval-adapter/, standalone installable package, depends on evalport-sdk, includes tests + README) if that's of interest — just wanted to check first given BrowserGym has no CONTRIBUTING.md spelling out a preferred process for this kind of thing.

— Sahi, independent contributor, maintainer of EvalPort