Conversation
- Add LLMConfig and task_mode configuration - Implement scenario loader for prompt configuration - Create LLMScenarioRunner for Gateway HTTP requests - Extend LLMMetrics and TestSnapshot for LLM mode - Update stats_collector for LLM mode reports - Add scenarios.yaml template and usage documentation - Add comprehensive unit tests (28 tests) Key features: - Health check for MockLLM service - Gateway HTTP requests to sandbox internal endpoint - Scenario-based benchmark with configurable timeouts - Compatible with existing browser task mode Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add requests>=2.28.0 to pyproject.toml dependencies - Revert urllib changes, use requests for health check - Update unit tests to mock requests.get Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add OpenClawConfig class for dynamic provider configuration - Configure MockLLM endpoint before benchmark starts - Use 'openclaw config set' and 'openclaw models set' commands - Gateway hot-reloads after configuration change - Update documentation with dynamic config flow Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Preserved both LLM benchmark (PR #49) and round-robin/refactored ReportFormatter (main) features: - bench.py: Keep all imports (llm_runner, round_robin, SandboxState) - stats_collector.py: - _take_snapshot: LLM mode collects llm_metrics, browser mode keeps round comparison (_round_start_totals) - generate_report: LLM mode uses new format_llm_stats_section(), browser mode uses refactored ReportFormatter methods - Added format_llm_stats_section() to ReportFormatter for LLM stats - llm_runner.py: ruff-format fixes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Key features: