diff --git a/ScratchV-topic06-deliverable/.github/workflows/benchmark.yml b/ScratchV-topic06-deliverable/.github/workflows/benchmark.yml
new file mode 100644
index 0000000..6e4bffd
--- /dev/null
+++ b/ScratchV-topic06-deliverable/.github/workflows/benchmark.yml
@@ -0,0 +1,41 @@
+name: ScratchV Course Benchmark Suite
+
+on:
+ push:
+ pull_request:
+
+jobs:
+ benchmark:
+ runs-on: ubuntu-latest
+
+ steps:
+ - name: Check out repository
+ uses: actions/checkout@v4
+
+ - name: Set up Python
+ uses: actions/setup-python@v5
+ with:
+ python-version: "3.11"
+
+ - name: Install package and course benchmark dependencies
+ run: |
+ python -m pip install --upgrade pip
+ python -m pip install -e .
+ python -m pip install tinyfive pytest jinja2 matplotlib
+
+ - name: Run unit tests
+ run: python -m pytest -q
+
+ - name: Run course benchmark suite
+ run: |
+ python run_tests.py --benchmark 3
+
+ - name: Upload course benchmark reports
+ uses: actions/upload-artifact@v4
+ with:
+ name: scratchv-course-benchmark-reports
+ path: |
+ reports/report.md
+ reports/report.html
+ reports/course_report_instructions.png
+ reports/benchmark_baseline.json
diff --git a/ScratchV-topic06-deliverable/LICENSE b/ScratchV-topic06-deliverable/LICENSE
new file mode 100644
index 0000000..87324d7
--- /dev/null
+++ b/ScratchV-topic06-deliverable/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 ScratchV
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/ScratchV-topic06-deliverable/README.md b/ScratchV-topic06-deliverable/README.md
new file mode 100644
index 0000000..10c5879
--- /dev/null
+++ b/ScratchV-topic06-deliverable/README.md
@@ -0,0 +1,123 @@
+# ScratchV 课题 06 交付说明
+本测试套件不是独立项目,需要放在 ScratchV 仓库根目录下运行,并依赖 ScratchV 原项目环境。
+本目录是 ScratchV 课题 06 的性能基准与验证测试套件,包含 23 个 DSL测试用例、自动化测试脚本、依赖说明以及可选的测试报告。
+其中3个branch中的三个if分支调用tinyfive时目前还不通过,是因为当前 ScratchV 的分支汇编和 tinyfive 适配层没有对齐。
+
+## 运行依赖
+
+该交付目录需要放在 ScratchV 项目环境中运行。`run_tests.py` 会导入
+`scratchv` 包,并通过 ScratchV 编译器把每个 DSL 用例编译为 RISC-V 汇编。
+若调用真实的tinyfive而非stub需要安装两个源码补丁:
+https://github.com/ScratchV-Compiler/ScratchV/pull/15
+https://github.com/ScratchV-Compiler/ScratchV/pull/17
+安装额外依赖:
+
+```powershell
+pip install -r requirements-topic06.txt
+```
+
+至少需要以下依赖:
+
+- `tinyfive`:用于模拟执行生成的 RISC-V 汇编
+- `pytest`:用于测试支持
+- `jinja2`:用于生成 HTML 报告
+- `matplotlib`:用于生成性能图表
+
+## 目录结构
+
+```text
+run_tests.py
+requirements-topic06.txt
+tests_main/
+ activation/
+ branch/
+ elementwise/
+ loop/
+ reduction/
+ tensor/
+reports/ # 测试报告,可重新生成
+build/ # 编译输出的汇编文件,可重新生成
+```
+
+`tests_main/` 下共有 23 个 DSL 用例。每个用例由 `.dsl` 文件和对应的
+`.meta.json` 文件组成,`.meta.json` 中定义输入、期望返回值和用例说明。
+
+## 运行测试
+
+在本目录下执行:
+
+```powershell
+python run_tests.py
+```
+
+测试脚本会自动完成以下步骤:
+
+- 遍历 `tests_main/` 下的所有 `.dsl` 文件
+- 调用 ScratchV 编译器生成 RISC-V 汇编
+- 通过 ScratchV 的 TinyFive 适配层验证生成的汇编
+- 根据 `.meta.json` 中的期望值进行结果对比
+- 增加时间测量(time.perf_counter),输出到报告
+- 统计 PASS/FAIL 和指令数
+- 在 `reports/` 下生成 Markdown 和 HTML 报告
+
+## Benchmark 模式
+
+重复运行每个用例并统计平均指令数:
+
+```powershell
+python run_tests.py --benchmark 3
+```
+
+更新性能基线:
+
+```powershell
+python run_tests.py --benchmark 3 --update-baseline
+```
+
+性能基线文件位于:
+
+```text
+reports/benchmark_baseline.json
+```
+
+## 测试报告
+
+运行后会生成:
+
+```text
+reports/report.md
+reports/report.html
+reports/course_report_instructions.png
+```
+
+`report.md` 适合提交或归档,`report.html` 适合在浏览器中查看测试结果。
+
+## 添加测试用例
+
+新增用例时添加两个文件:
+
+```text
+tests_main/{category}/{name}.dsl
+tests_main/{category}/{name}.meta.json
+```
+
+DSL 示例:
+
+```text
+result = add(a, b)
+return result
+```
+
+元数据示例:
+
+```json
+{
+ "description": "Simple scalar add.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 2,
+ "b": 3
+ },
+ "expected_return": 5
+}
+```
\ No newline at end of file
diff --git a/ScratchV-topic06-deliverable/reports/benchmark_baseline.json b/ScratchV-topic06-deliverable/reports/benchmark_baseline.json
new file mode 100644
index 0000000..6c11a38
--- /dev/null
+++ b/ScratchV-topic06-deliverable/reports/benchmark_baseline.json
@@ -0,0 +1,117 @@
+{
+ "add_relu_relu": {
+ "category": "activation",
+ "avg_instr_count": 7.0,
+ "runs": 3
+ },
+ "relu_add": {
+ "category": "activation",
+ "avg_instr_count": 5.0,
+ "runs": 3
+ },
+ "relu_only": {
+ "category": "activation",
+ "avg_instr_count": 4.0,
+ "runs": 3
+ },
+ "relu_twice": {
+ "category": "activation",
+ "avg_instr_count": 6.0,
+ "runs": 3
+ },
+ "if_else": {
+ "category": "branch",
+ "avg_instr_count": 0.0,
+ "runs": 3
+ },
+ "if_relu": {
+ "category": "branch",
+ "avg_instr_count": 0.0,
+ "runs": 3
+ },
+ "if_then": {
+ "category": "branch",
+ "avg_instr_count": 0.0,
+ "runs": 3
+ },
+ "add_chain": {
+ "category": "elementwise",
+ "avg_instr_count": 4.0,
+ "runs": 3
+ },
+ "add_chain_3": {
+ "category": "elementwise",
+ "avg_instr_count": 5.0,
+ "runs": 3
+ },
+ "add_fan_in_4": {
+ "category": "elementwise",
+ "avg_instr_count": 5.0,
+ "runs": 3
+ },
+ "add_reuse": {
+ "category": "elementwise",
+ "avg_instr_count": 4.0,
+ "runs": 3
+ },
+ "vector_add": {
+ "category": "elementwise",
+ "avg_instr_count": 3.0,
+ "runs": 3
+ },
+ "loop_add_4": {
+ "category": "loop",
+ "avg_instr_count": 22.0,
+ "runs": 3
+ },
+ "loop_add_chain_4": {
+ "category": "loop",
+ "avg_instr_count": 26.0,
+ "runs": 3
+ },
+ "loop_relu_add_4": {
+ "category": "loop",
+ "avg_instr_count": 30.0,
+ "runs": 3
+ },
+ "dot_4": {
+ "category": "reduction",
+ "avg_instr_count": 3.0,
+ "runs": 3
+ },
+ "dot_8": {
+ "category": "reduction",
+ "avg_instr_count": 3.0,
+ "runs": 3
+ },
+ "dot_relu_4": {
+ "category": "reduction",
+ "avg_instr_count": 5.0,
+ "runs": 3
+ },
+ "dot_relu_8": {
+ "category": "reduction",
+ "avg_instr_count": 5.0,
+ "runs": 3
+ },
+ "matmul_2x2": {
+ "category": "tensor",
+ "avg_instr_count": 3.0,
+ "runs": 3
+ },
+ "matmul_4x4": {
+ "category": "tensor",
+ "avg_instr_count": 3.0,
+ "runs": 3
+ },
+ "matmul_add_2x2": {
+ "category": "tensor",
+ "avg_instr_count": 4.0,
+ "runs": 3
+ },
+ "matmul_relu_2x2": {
+ "category": "tensor",
+ "avg_instr_count": 5.0,
+ "runs": 3
+ }
+}
\ No newline at end of file
diff --git a/ScratchV-topic06-deliverable/reports/course_report_instructions.png b/ScratchV-topic06-deliverable/reports/course_report_instructions.png
new file mode 100644
index 0000000..842bcde
Binary files /dev/null and b/ScratchV-topic06-deliverable/reports/course_report_instructions.png differ
diff --git a/ScratchV-topic06-deliverable/reports/report.html b/ScratchV-topic06-deliverable/reports/report.html
new file mode 100644
index 0000000..5ab96da
--- /dev/null
+++ b/ScratchV-topic06-deliverable/reports/report.html
@@ -0,0 +1,383 @@
+
+
+
+
+ ScratchV 测试报告
+
+
+
+ ScratchV DSL 编译器性能测试报告
+
+
用例总数:23,通过:20,失败:3,通过率:87.0%
+
测试目录:D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\tests_main,性能退化阈值:5.0%,单次模拟超时:5.0s
+
+
+
+
+
+ | 用例 | 类别 | 状态 | 模拟后端 |
+ 平均指令数 | 95% 置信区间 |
+ 编译耗时(s) | 模拟耗时(s) | 总耗时(s) |
+ 变化率(%) | 是否退化 | 描述 |
+
+
+
+
+
+ | add_relu_relu |
+ activation |
+ PASS |
+ none |
+ 7.00 |
+ ±0.00 |
+ 0.0532 |
+ 0.1323 |
+ 0.4806 |
+ 0.00 |
+ False |
+ Add input and bias, then apply ReLU twice. |
+
+
+
+ | relu_add |
+ activation |
+ PASS |
+ none |
+ 5.00 |
+ ±0.00 |
+ 0.0564 |
+ 0.0981 |
+ 0.4456 |
+ 0.00 |
+ False |
+ Add input and bias, then apply one ReLU. |
+
+
+
+ | relu_only |
+ activation |
+ PASS |
+ none |
+ 4.00 |
+ ±0.00 |
+ 0.0516 |
+ 0.1000 |
+ 0.4359 |
+ 0.00 |
+ False |
+ Apply ReLU directly to a single input value. |
+
+
+
+ | relu_twice |
+ activation |
+ PASS |
+ none |
+ 6.00 |
+ ±0.00 |
+ 0.0533 |
+ 0.0928 |
+ 0.4265 |
+ 0.00 |
+ False |
+ Apply ReLU twice to the same activation path. |
+
+
+
+ | if_else |
+ branch |
+ FAIL |
+ timeout |
+ 0.00 |
+ ±0.00 |
+ 0.0512 |
+ 5.0168 |
+ 20.0935 |
+ 0.00 |
+ False |
+ if/else branch returns subtraction result when flag is zero. |
+
+
+
+ | if_relu |
+ branch |
+ FAIL |
+ timeout |
+ 0.00 |
+ ±0.00 |
+ 0.0538 |
+ 5.0090 |
+ 20.1086 |
+ 0.00 |
+ False |
+ if/else branch combined with add and relu. |
+
+
+
+ | if_then |
+ branch |
+ FAIL |
+ timeout |
+ 0.00 |
+ ±0.00 |
+ 0.0549 |
+ 5.0095 |
+ 20.1093 |
+ 0.00 |
+ False |
+ if/else branch returns add result when flag is non-zero. |
+
+
+
+ | add_chain |
+ elementwise |
+ PASS |
+ none |
+ 4.00 |
+ ±0.00 |
+ 0.0548 |
+ 0.0969 |
+ 0.4364 |
+ 0.00 |
+ False |
+ Add a and b, then add c to the intermediate result. |
+
+
+
+ | add_chain_3 |
+ elementwise |
+ PASS |
+ none |
+ 5.00 |
+ ±0.00 |
+ 0.0525 |
+ 0.0971 |
+ 0.4357 |
+ 0.00 |
+ False |
+ Chain three add operations across four symbolic inputs. |
+
+
+
+ | add_fan_in_4 |
+ elementwise |
+ PASS |
+ none |
+ 5.00 |
+ ±0.00 |
+ 0.0523 |
+ 0.1005 |
+ 0.4393 |
+ 0.00 |
+ False |
+ Compute two independent adds and then merge them with a final add. |
+
+
+
+ | add_reuse |
+ elementwise |
+ PASS |
+ none |
+ 4.00 |
+ ±0.00 |
+ 0.0523 |
+ 0.0935 |
+ 0.4320 |
+ 0.00 |
+ False |
+ Reuse the same intermediate add result on both operands of a second add. |
+
+
+
+ | vector_add |
+ elementwise |
+ PASS |
+ none |
+ 3.00 |
+ ±0.00 |
+ 0.0535 |
+ 0.0945 |
+ 0.4363 |
+ 0.00 |
+ False |
+ Single add over two symbolic vector inputs. |
+
+
+
+ | loop_add_4 |
+ loop |
+ PASS |
+ none |
+ 22.00 |
+ ±0.00 |
+ 0.0520 |
+ 0.0941 |
+ 0.4312 |
+ 0.00 |
+ False |
+ Run a four-iteration loop whose body computes one add; final returned value is the last loop-body result. |
+
+
+
+ | loop_add_chain_4 |
+ loop |
+ PASS |
+ none |
+ 26.00 |
+ ±0.00 |
+ 0.0534 |
+ 0.0950 |
+ 0.4375 |
+ 0.00 |
+ False |
+ Run a four-iteration loop whose body computes two chained adds; final returned value is the last loop-body result. |
+
+
+
+ | loop_relu_add_4 |
+ loop |
+ PASS |
+ none |
+ 30.00 |
+ ±0.00 |
+ 0.0511 |
+ 0.0950 |
+ 0.4302 |
+ 0.00 |
+ False |
+ Run a four-iteration loop whose body computes add followed by ReLU; final returned value is the last loop-body result. |
+
+
+
+ | dot_4 |
+ reduction |
+ PASS |
+ none |
+ 3.00 |
+ ±0.00 |
+ 0.0544 |
+ 0.0939 |
+ 0.4362 |
+ 0.00 |
+ False |
+ Compute the dot product of two symbolic vectors of length 4. |
+
+
+
+ | dot_8 |
+ reduction |
+ PASS |
+ none |
+ 3.00 |
+ ±0.00 |
+ 0.0537 |
+ 0.1021 |
+ 0.4474 |
+ 0.00 |
+ False |
+ Compute the dot product of two symbolic vectors of length 8. |
+
+
+
+ | dot_relu_4 |
+ reduction |
+ PASS |
+ none |
+ 5.00 |
+ ±0.00 |
+ 0.0523 |
+ 0.0954 |
+ 0.4342 |
+ 0.00 |
+ False |
+ Compute a length-4 dot product and pass it through ReLU. |
+
+
+
+ | dot_relu_8 |
+ reduction |
+ PASS |
+ none |
+ 5.00 |
+ ±0.00 |
+ 0.0524 |
+ 0.0945 |
+ 0.4350 |
+ 0.00 |
+ False |
+ Compute a length-8 dot product and pass it through ReLU. |
+
+
+
+ | matmul_2x2 |
+ tensor |
+ PASS |
+ none |
+ 3.00 |
+ ±0.00 |
+ 0.0518 |
+ 0.0952 |
+ 0.4328 |
+ 0.00 |
+ False |
+ Compute a symbolic 2x2 by 2x2 matrix multiplication. |
+
+
+
+ | matmul_4x4 |
+ tensor |
+ PASS |
+ none |
+ 3.00 |
+ ±0.00 |
+ 0.0529 |
+ 0.0942 |
+ 0.4400 |
+ 0.00 |
+ False |
+ Compute a symbolic 4x4 by 4x4 matrix multiplication. |
+
+
+
+ | matmul_add_2x2 |
+ tensor |
+ PASS |
+ none |
+ 4.00 |
+ ±0.00 |
+ 0.0522 |
+ 0.0940 |
+ 0.4316 |
+ 0.00 |
+ False |
+ Compute a 2x2 matmul and then add a symbolic bias term. |
+
+
+
+ | matmul_relu_2x2 |
+ tensor |
+ PASS |
+ none |
+ 5.00 |
+ ±0.00 |
+ 0.0514 |
+ 0.0966 |
+ 0.4342 |
+ 0.00 |
+ False |
+ Compute a 2x2 matmul and then apply ReLU to its result. |
+
+
+
+
+
+
\ No newline at end of file
diff --git a/ScratchV-topic06-deliverable/reports/report.md b/ScratchV-topic06-deliverable/reports/report.md
new file mode 100644
index 0000000..60cc532
--- /dev/null
+++ b/ScratchV-topic06-deliverable/reports/report.md
@@ -0,0 +1,588 @@
+# ScratchV DSL 编译器性能测试报告
+
+## 测试概览
+
+- 生成时间: 2026-07-07 18:48:01
+- 用例总数: 23
+- 通过数量: 20
+- 失败数量: 3
+- 通过率: 87.0%
+- 测试目录: `D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\tests_main`
+- 汇编输出目录: `D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build`
+- 性能基线文件: `D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\reports\benchmark_baseline.json`
+- 性能退化阈值: 5.0%
+- 单次模拟超时: 5s
+
+## 测试结果
+
+| 用例 | 类别 | 状态 | 模拟后端 | 平均指令数 | 95% 置信区间 | 最小 | 最大 | 编译耗时(s) | 模拟耗时(s) | 总耗时(s) | 基线 | 变化率(%) | 是否退化 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |
+|---|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|---|---|---|---|
+| add_relu_relu | activation | PASS | none | 7.00 | ±0.00 | 7 | 7 | 0.0532 | 0.1323 | 0.4806 | 7.00 | 0.00 | False | 7 | 7 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_relu_relu.s |
+| relu_add | activation | PASS | none | 5.00 | ±0.00 | 5 | 5 | 0.0564 | 0.0981 | 0.4456 | 5.00 | 0.00 | False | 3 | 3 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\relu_add.s |
+| relu_only | activation | PASS | none | 4.00 | ±0.00 | 4 | 4 | 0.0516 | 0.1000 | 0.4359 | 4.00 | 0.00 | False | 0 | 0 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\relu_only.s |
+| relu_twice | activation | PASS | none | 6.00 | ±0.00 | 6 | 6 | 0.0533 | 0.0928 | 0.4265 | 6.00 | 0.00 | False | 4 | 4 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\relu_twice.s |
+| if_else | branch | FAIL | timeout | 0.00 | ±0.00 | 0 | 0 | 0.0512 | 5.0168 | 20.0935 | 0.00 | 0.00 | False | 5 | 5 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\if_else.s |
+| if_relu | branch | FAIL | timeout | 0.00 | ±0.00 | 0 | 0 | 0.0538 | 5.0090 | 20.1086 | 0.00 | 0.00 | False | 0 | 0 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\if_relu.s |
+| if_then | branch | FAIL | timeout | 0.00 | ±0.00 | 0 | 0 | 0.0549 | 5.0095 | 20.1093 | 0.00 | 0.00 | False | 13 | 13 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\if_then.s |
+| add_chain | elementwise | PASS | none | 4.00 | ±0.00 | 4 | 4 | 0.0548 | 0.0969 | 0.4364 | 4.00 | 0.00 | False | 9 | 9 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_chain.s |
+| add_chain_3 | elementwise | PASS | none | 5.00 | ±0.00 | 5 | 5 | 0.0525 | 0.0971 | 0.4357 | 5.00 | 0.00 | False | 14 | 14 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_chain_3.s |
+| add_fan_in_4 | elementwise | PASS | none | 5.00 | ±0.00 | 5 | 5 | 0.0523 | 0.1005 | 0.4393 | 5.00 | 0.00 | False | 10 | 10 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_fan_in_4.s |
+| add_reuse | elementwise | PASS | none | 4.00 | ±0.00 | 4 | 4 | 0.0523 | 0.0935 | 0.4320 | 4.00 | 0.00 | False | 10 | 10 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_reuse.s |
+| vector_add | elementwise | PASS | none | 3.00 | ±0.00 | 3 | 3 | 0.0535 | 0.0945 | 0.4363 | 3.00 | 0.00 | False | 5 | 5 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\vector_add.s |
+| loop_add_4 | loop | PASS | none | 22.00 | ±0.00 | 22 | 22 | 0.0520 | 0.0941 | 0.4312 | 22.00 | 0.00 | False | 5 | 5 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\loop_add_4.s |
+| loop_add_chain_4 | loop | PASS | none | 26.00 | ±0.00 | 26 | 26 | 0.0534 | 0.0950 | 0.4375 | 26.00 | 0.00 | False | 9 | 9 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\loop_add_chain_4.s |
+| loop_relu_add_4 | loop | PASS | none | 30.00 | ±0.00 | 30 | 30 | 0.0511 | 0.0950 | 0.4302 | 30.00 | 0.00 | False | 2 | 2 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\loop_relu_add_4.s |
+| dot_4 | reduction | PASS | none | 3.00 | ±0.00 | 3 | 3 | 0.0544 | 0.0939 | 0.4362 | 3.00 | 0.00 | False | 70 | 70 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\dot_4.s |
+| dot_8 | reduction | PASS | none | 3.00 | ±0.00 | 3 | 3 | 0.0537 | 0.1021 | 0.4474 | 3.00 | 0.00 | False | 36 | 36 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\dot_8.s |
+| dot_relu_4 | reduction | PASS | none | 5.00 | ±0.00 | 5 | 5 | 0.0523 | 0.0954 | 0.4342 | 5.00 | 0.00 | False | 0 | 0 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\dot_relu_4.s |
+| dot_relu_8 | reduction | PASS | none | 5.00 | ±0.00 | 5 | 5 | 0.0524 | 0.0945 | 0.4350 | 5.00 | 0.00 | False | 8 | 8 | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\dot_relu_8.s |
+| matmul_2x2 | tensor | PASS | none | 3.00 | ±0.00 | 3 | 3 | 0.0518 | 0.0952 | 0.4328 | 3.00 | 0.00 | False | [[19, 22], [43, 50]] | [[19, 22], [43, 50]] | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\matmul_2x2.s |
+| matmul_4x4 | tensor | PASS | none | 3.00 | ±0.00 | 3 | 3 | 0.0529 | 0.0942 | 0.4400 | 3.00 | 0.00 | False | [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]] | [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]] | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\matmul_4x4.s |
+| matmul_add_2x2 | tensor | PASS | none | 4.00 | ±0.00 | 4 | 4 | 0.0522 | 0.0940 | 0.4316 | 4.00 | 0.00 | False | [[20, 23], [44, 51]] | [[20, 23], [44, 51]] | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\matmul_add_2x2.s |
+| matmul_relu_2x2 | tensor | PASS | none | 5.00 | ±0.00 | 5 | 5 | 0.0514 | 0.0966 | 0.4342 | 5.00 | 0.00 | False | [[0, 2], [0, 4]] | [[0, 2], [0, 4]] | True | D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\matmul_relu_2x2.s |
+
+## 性能图表
+
+
+
+### Mermaid 图表
+
+```mermaid
+xychart-beta
+ title "各测试用例指令数"
+ x-axis ["add_relu_relu", "relu_add", "relu_only", "relu_twice", "if_else", "if_relu", "if_then", "add_chain", "add_chain_3", "add_fan_in_4", "add_reuse", "vector_add", "loop_add_4", "loop_add_chain_4", "loop_relu_add_4", "dot_4", "dot_8", "dot_relu_4", "dot_relu_8", "matmul_2x2", "matmul_4x4", "matmul_add_2x2", "matmul_relu_2x2"]
+ y-axis "指令数" 0 --> 32.0
+ bar [7.0, 5.0, 4.0, 6.0, 0.0, 0.0, 0.0, 4.0, 5.0, 5.0, 4.0, 3.0, 22.0, 26.0, 30.0, 3.0, 3.0, 5.0, 5.0, 3.0, 3.0, 4.0, 5.0]
+```
+
+## 用例详情
+
+### add_relu_relu
+
+- 类别: activation
+- 描述: Add input and bias, then apply ReLU twice.
+- 预期输出 (return_value): 7
+- 实际输出: 7
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0532
+- 模拟耗时(s): 0.1323
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4806
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_relu_relu.s
+- Benchmark 重复次数: 3
+- 平均指令数: 7.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 7
+- 最大指令数: 7
+- 基线指令数: 7.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### relu_add
+
+- 类别: activation
+- 描述: Add input and bias, then apply one ReLU.
+- 预期输出 (return_value): 3
+- 实际输出: 3
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0564
+- 模拟耗时(s): 0.0981
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4456
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\relu_add.s
+- Benchmark 重复次数: 3
+- 平均指令数: 5.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 5
+- 最大指令数: 5
+- 基线指令数: 5.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### relu_only
+
+- 类别: activation
+- 描述: Apply ReLU directly to a single input value.
+- 预期输出 (return_value): 0
+- 实际输出: 0
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0516
+- 模拟耗时(s): 0.1000
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4359
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\relu_only.s
+- Benchmark 重复次数: 3
+- 平均指令数: 4.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 4
+- 最大指令数: 4
+- 基线指令数: 4.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### relu_twice
+
+- 类别: activation
+- 描述: Apply ReLU twice to the same activation path.
+- 预期输出 (return_value): 4
+- 实际输出: 4
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0533
+- 模拟耗时(s): 0.0928
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4265
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\relu_twice.s
+- Benchmark 重复次数: 3
+- 平均指令数: 6.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 6
+- 最大指令数: 6
+- 基线指令数: 6.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### if_else
+
+- 类别: branch
+- 描述: if/else branch returns subtraction result when flag is zero.
+- 预期输出 (return_value): 5
+- 实际输出: 5
+- 输出是否匹配: True
+- 模拟后端: timeout
+- 编译耗时(s): 0.0512
+- 模拟耗时(s): 5.0168
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 20.0935
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\if_else.s
+- Benchmark 重复次数: 3
+- 平均指令数: 0.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 0
+- 最大指令数: 0
+- 基线指令数: 0.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### if_relu
+
+- 类别: branch
+- 描述: if/else branch combined with add and relu.
+- 预期输出 (return_value): 0
+- 实际输出: 0
+- 输出是否匹配: True
+- 模拟后端: timeout
+- 编译耗时(s): 0.0538
+- 模拟耗时(s): 5.0090
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 20.1086
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\if_relu.s
+- Benchmark 重复次数: 3
+- 平均指令数: 0.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 0
+- 最大指令数: 0
+- 基线指令数: 0.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### if_then
+
+- 类别: branch
+- 描述: if/else branch returns add result when flag is non-zero.
+- 预期输出 (return_value): 13
+- 实际输出: 13
+- 输出是否匹配: True
+- 模拟后端: timeout
+- 编译耗时(s): 0.0549
+- 模拟耗时(s): 5.0095
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 20.1093
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\if_then.s
+- Benchmark 重复次数: 3
+- 平均指令数: 0.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 0
+- 最大指令数: 0
+- 基线指令数: 0.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### add_chain
+
+- 类别: elementwise
+- 描述: Add a and b, then add c to the intermediate result.
+- 预期输出 (return_value): 9
+- 实际输出: 9
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0548
+- 模拟耗时(s): 0.0969
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4364
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_chain.s
+- Benchmark 重复次数: 3
+- 平均指令数: 4.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 4
+- 最大指令数: 4
+- 基线指令数: 4.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### add_chain_3
+
+- 类别: elementwise
+- 描述: Chain three add operations across four symbolic inputs.
+- 预期输出 (return_value): 14
+- 实际输出: 14
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0525
+- 模拟耗时(s): 0.0971
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4357
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_chain_3.s
+- Benchmark 重复次数: 3
+- 平均指令数: 5.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 5
+- 最大指令数: 5
+- 基线指令数: 5.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### add_fan_in_4
+
+- 类别: elementwise
+- 描述: Compute two independent adds and then merge them with a final add.
+- 预期输出 (return_value): 10
+- 实际输出: 10
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0523
+- 模拟耗时(s): 0.1005
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4393
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_fan_in_4.s
+- Benchmark 重复次数: 3
+- 平均指令数: 5.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 5
+- 最大指令数: 5
+- 基线指令数: 5.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### add_reuse
+
+- 类别: elementwise
+- 描述: Reuse the same intermediate add result on both operands of a second add.
+- 预期输出 (return_value): 10
+- 实际输出: 10
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0523
+- 模拟耗时(s): 0.0935
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4320
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\add_reuse.s
+- Benchmark 重复次数: 3
+- 平均指令数: 4.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 4
+- 最大指令数: 4
+- 基线指令数: 4.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### vector_add
+
+- 类别: elementwise
+- 描述: Single add over two symbolic vector inputs.
+- 预期输出 (return_value): 5
+- 实际输出: 5
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0535
+- 模拟耗时(s): 0.0945
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4363
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\vector_add.s
+- Benchmark 重复次数: 3
+- 平均指令数: 3.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 3
+- 最大指令数: 3
+- 基线指令数: 3.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### loop_add_4
+
+- 类别: loop
+- 描述: Run a four-iteration loop whose body computes one add; final returned value is the last loop-body result.
+- 预期输出 (return_value): 5
+- 实际输出: 5
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0520
+- 模拟耗时(s): 0.0941
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4312
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\loop_add_4.s
+- Benchmark 重复次数: 3
+- 平均指令数: 22.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 22
+- 最大指令数: 22
+- 基线指令数: 22.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### loop_add_chain_4
+
+- 类别: loop
+- 描述: Run a four-iteration loop whose body computes two chained adds; final returned value is the last loop-body result.
+- 预期输出 (return_value): 9
+- 实际输出: 9
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0534
+- 模拟耗时(s): 0.0950
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4375
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\loop_add_chain_4.s
+- Benchmark 重复次数: 3
+- 平均指令数: 26.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 26
+- 最大指令数: 26
+- 基线指令数: 26.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### loop_relu_add_4
+
+- 类别: loop
+- 描述: Run a four-iteration loop whose body computes add followed by ReLU; final returned value is the last loop-body result.
+- 预期输出 (return_value): 2
+- 实际输出: 2
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0511
+- 模拟耗时(s): 0.0950
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4302
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\loop_relu_add_4.s
+- Benchmark 重复次数: 3
+- 平均指令数: 30.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 30
+- 最大指令数: 30
+- 基线指令数: 30.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### dot_4
+
+- 类别: reduction
+- 描述: Compute the dot product of two symbolic vectors of length 4.
+- 预期输出 (return_value): 70
+- 实际输出: 70
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0544
+- 模拟耗时(s): 0.0939
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4362
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\dot_4.s
+- Benchmark 重复次数: 3
+- 平均指令数: 3.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 3
+- 最大指令数: 3
+- 基线指令数: 3.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### dot_8
+
+- 类别: reduction
+- 描述: Compute the dot product of two symbolic vectors of length 8.
+- 预期输出 (return_value): 36
+- 实际输出: 36
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0537
+- 模拟耗时(s): 0.1021
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4474
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\dot_8.s
+- Benchmark 重复次数: 3
+- 平均指令数: 3.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 3
+- 最大指令数: 3
+- 基线指令数: 3.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### dot_relu_4
+
+- 类别: reduction
+- 描述: Compute a length-4 dot product and pass it through ReLU.
+- 预期输出 (return_value): 0
+- 实际输出: 0
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0523
+- 模拟耗时(s): 0.0954
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4342
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\dot_relu_4.s
+- Benchmark 重复次数: 3
+- 平均指令数: 5.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 5
+- 最大指令数: 5
+- 基线指令数: 5.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### dot_relu_8
+
+- 类别: reduction
+- 描述: Compute a length-8 dot product and pass it through ReLU.
+- 预期输出 (return_value): 8
+- 实际输出: 8
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0524
+- 模拟耗时(s): 0.0945
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4350
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\dot_relu_8.s
+- Benchmark 重复次数: 3
+- 平均指令数: 5.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 5
+- 最大指令数: 5
+- 基线指令数: 5.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### matmul_2x2
+
+- 类别: tensor
+- 描述: Compute a symbolic 2x2 by 2x2 matrix multiplication.
+- 预期输出 (return_value): [[19, 22], [43, 50]]
+- 实际输出: [[19, 22], [43, 50]]
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0518
+- 模拟耗时(s): 0.0952
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4328
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\matmul_2x2.s
+- Benchmark 重复次数: 3
+- 平均指令数: 3.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 3
+- 最大指令数: 3
+- 基线指令数: 3.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### matmul_4x4
+
+- 类别: tensor
+- 描述: Compute a symbolic 4x4 by 4x4 matrix multiplication.
+- 预期输出 (return_value): [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
+- 实际输出: [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0529
+- 模拟耗时(s): 0.0942
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4400
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\matmul_4x4.s
+- Benchmark 重复次数: 3
+- 平均指令数: 3.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 3
+- 最大指令数: 3
+- 基线指令数: 3.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### matmul_add_2x2
+
+- 类别: tensor
+- 描述: Compute a 2x2 matmul and then add a symbolic bias term.
+- 预期输出 (return_value): [[20, 23], [44, 51]]
+- 实际输出: [[20, 23], [44, 51]]
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0522
+- 模拟耗时(s): 0.0940
+- 参考解释器耗时(s): 0.0002
+- 总耗时(s): 0.4316
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\matmul_add_2x2.s
+- Benchmark 重复次数: 3
+- 平均指令数: 4.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 4
+- 最大指令数: 4
+- 基线指令数: 4.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
+### matmul_relu_2x2
+
+- 类别: tensor
+- 描述: Compute a 2x2 matmul and then apply ReLU to its result.
+- 预期输出 (return_value): [[0, 2], [0, 4]]
+- 实际输出: [[0, 2], [0, 4]]
+- 输出是否匹配: True
+- 模拟后端: none
+- 编译耗时(s): 0.0514
+- 模拟耗时(s): 0.0966
+- 参考解释器耗时(s): 0.0001
+- 总耗时(s): 0.4342
+- 汇编文件: D:\PycharmProjects\ScratchV\ScratchV\ScratchV-topic06-deliverable\build\matmul_relu_2x2.s
+- Benchmark 重复次数: 3
+- 平均指令数: 5.00
+- 95% 置信区间: ±0.00
+- 最小指令数: 5
+- 最大指令数: 5
+- 基线指令数: 5.00
+- 性能变化率: 0.00%
+- 性能退化阈值: 5.00%
+- 是否性能退化: False
+
diff --git a/ScratchV-topic06-deliverable/requirements-topic06.txt b/ScratchV-topic06-deliverable/requirements-topic06.txt
new file mode 100644
index 0000000..dc452fd
--- /dev/null
+++ b/ScratchV-topic06-deliverable/requirements-topic06.txt
@@ -0,0 +1,4 @@
+tinyfive
+pytest
+jinja2
+matplotlib
diff --git a/ScratchV-topic06-deliverable/run_tests.py b/ScratchV-topic06-deliverable/run_tests.py
new file mode 100644
index 0000000..fce002e
--- /dev/null
+++ b/ScratchV-topic06-deliverable/run_tests.py
@@ -0,0 +1,1481 @@
+import argparse
+import json
+import math
+import os
+import subprocess
+import sys
+import tempfile
+import time
+from datetime import datetime
+from pathlib import Path
+
+from scratchv.simulator.tinyfive import verify_assembly
+
+SUITE_DIR = Path(__file__).resolve().parent
+PROJECT_ROOT = SUITE_DIR.parent
+TEST_DIR = SUITE_DIR / "tests_main"
+BUILD_DIR = SUITE_DIR / "build"
+REPORT_DIR = SUITE_DIR / "reports"
+REPORT_FILE = REPORT_DIR / "report.md"
+HTML_REPORT_FILE = REPORT_DIR / "report.html"
+CHART_FILE = REPORT_DIR / "course_report_instructions.png"
+BASELINE_FILE = REPORT_DIR / "benchmark_baseline.json"
+REGRESSION_THRESHOLD_PCT = 5.0
+SIMULATION_TIMEOUT_SEC = 5.0
+
+
+def run_compile(dsl_file: Path):
+ output_file = BUILD_DIR / (dsl_file.stem + ".s")
+
+ cmd = [
+ sys.executable,
+ "-m",
+ "scratchv.main",
+ str(dsl_file),
+ "-o",
+ str(output_file),
+ "--optimize",
+ "all",
+ "--dump-ir",
+ ]
+
+ result = subprocess.run(
+ cmd,
+ capture_output=True,
+ text=True,
+ encoding="utf-8",
+ errors="ignore",
+ cwd=PROJECT_ROOT,
+ )
+
+ return result, output_file
+
+
+def load_metadata(dsl_file: Path):
+ meta_file = dsl_file.with_suffix(".meta.json")
+ if not meta_file.exists():
+ return {
+ "description": "",
+ "expected_output_type": "return_value",
+ "expected_return": "",
+ }
+ return json.loads(meta_file.read_text(encoding="utf-8"))
+
+
+def run_simulation(asm_file: Path, timeout: float = SIMULATION_TIMEOUT_SEC):
+ if not asm_file.exists():
+ return {
+ "success": False,
+ "instr_count": 0,
+ "return_value": None,
+ "backend": "none",
+ "error": "assembly file not found",
+ }
+
+ code = "\n".join([
+ "import json, sys",
+ "from pathlib import Path",
+ "from scratchv.simulator.tinyfive import verify_assembly",
+ "asm = Path(sys.argv[1]).read_text(encoding='utf-8')",
+ "try:",
+ " result = verify_assembly(asm)",
+ "except Exception as exc:",
+ " result = {'success': False, 'instr_count': 0, "
+ "'return_value': None, 'backend': 'tinyfive', 'error': str(exc)}",
+ "print(json.dumps(result, ensure_ascii=False))",
+ ])
+
+ try:
+ completed = subprocess.run(
+ [sys.executable, "-c", code, str(asm_file)],
+ capture_output=True,
+ text=True,
+ encoding="utf-8",
+ errors="ignore",
+ cwd=PROJECT_ROOT,
+ timeout=timeout,
+ )
+ except subprocess.TimeoutExpired:
+ return {
+ "success": False,
+ "instr_count": 0,
+ "return_value": None,
+ "backend": "timeout",
+ "error": f"simulation timeout after {timeout:.0f}s",
+ }
+
+ if completed.returncode != 0:
+ return {
+ "success": False,
+ "instr_count": 0,
+ "return_value": None,
+ "backend": "tinyfive",
+ "error": (completed.stderr or completed.stdout or "simulation failed").strip(),
+ }
+
+ try:
+ return json.loads(completed.stdout.strip().splitlines()[-1])
+ except (IndexError, json.JSONDecodeError) as exc:
+ return {
+ "success": False,
+ "instr_count": 0,
+ "return_value": None,
+ "backend": "tinyfive",
+ "error": f"invalid simulation output: {exc}",
+ }
+
+
+def apply_add(lhs, rhs):
+ if isinstance(lhs, list) and isinstance(rhs, list):
+ return [apply_add(a, b) for a, b in zip(lhs, rhs)]
+ if isinstance(lhs, list):
+ return [apply_add(a, rhs) for a in lhs]
+ if isinstance(rhs, list):
+ return [apply_add(lhs, b) for b in rhs]
+ return lhs + rhs
+
+
+def apply_binary(lhs, rhs, op):
+ if isinstance(lhs, list) and isinstance(rhs, list):
+ return [apply_binary(a, b, op) for a, b in zip(lhs, rhs)]
+ if isinstance(lhs, list):
+ return [apply_binary(a, rhs, op) for a in lhs]
+ if isinstance(rhs, list):
+ return [apply_binary(lhs, b, op) for b in rhs]
+ return op(lhs, rhs)
+
+
+def apply_relu(value):
+ if isinstance(value, list):
+ return [apply_relu(v) for v in value]
+ return value if value > 0 else 0
+
+
+def apply_gelu(value):
+ if isinstance(value, list):
+ return [apply_gelu(v) for v in value]
+ return 0.5 * value * (1.0 + math.erf(value / math.sqrt(2.0)))
+
+
+def apply_softmax(value):
+ if not isinstance(value, list):
+ return 1.0
+ max_value = max(value)
+ exp_values = [math.exp(v - max_value) for v in value]
+ total = sum(exp_values)
+ return [v / total for v in exp_values]
+
+
+def apply_maxpool(value, kernel, stride):
+ if not isinstance(value, list):
+ return value
+ return [max(value[i:i + kernel]) for i in range(0, len(value) - kernel + 1, stride)]
+
+
+def apply_dot(lhs, rhs, length):
+ return sum(lhs[i] * rhs[i] for i in range(length))
+
+
+def apply_matmul(lhs, rhs, m, n, k):
+ result = []
+ for i in range(m):
+ row = []
+ for j in range(n):
+ cell = 0
+ for kk in range(k):
+ cell += lhs[i][kk] * rhs[kk][j]
+ row.append(cell)
+ result.append(row)
+ return result
+
+
+def resolve_value(token, env):
+ token = token.strip()
+ try:
+ return int(token)
+ except ValueError:
+ pass
+ try:
+ return float(token)
+ except ValueError:
+ pass
+ return env[token]
+
+
+def resolve_condition(cond_text, env):
+ cond_text = cond_text.strip()
+ if cond_text.endswith(":"):
+ cond_text = cond_text[:-1].strip()
+ if cond_text.startswith("(") and cond_text.endswith(")"):
+ cond_text = cond_text[1:-1].strip()
+
+ for op in ("==", "!=", "<=", ">=", "<", ">"):
+ if op in cond_text:
+ lhs_text, rhs_text = [part.strip() for part in cond_text.split(op, 1)]
+ lhs = resolve_value(lhs_text, env)
+ rhs = resolve_value(rhs_text, env)
+ if op == "==":
+ return lhs == rhs
+ if op == "!=":
+ return lhs != rhs
+ if op == "<=":
+ return lhs <= rhs
+ if op == ">=":
+ return lhs >= rhs
+ if op == "<":
+ return lhs < rhs
+ if op == ">":
+ return lhs > rhs
+
+ return bool(resolve_value(cond_text, env))
+
+
+def execute_block(lines, env, start_idx=0, end_idx=None):
+ if end_idx is None:
+ end_idx = len(lines)
+
+ idx = start_idx
+ while idx < end_idx:
+ line = lines[idx]
+
+ if line.startswith("for "):
+ loop_var_text = line.replace("for ", "", 1)
+ loop_var, bounds = [p.strip() for p in loop_var_text.split("=", 1)]
+ loop_start_text, loop_end_text = [p.strip() for p in bounds.split(",", 1)]
+ loop_start = int(loop_start_text)
+ loop_end = int(loop_end_text)
+
+ depth = 1
+ body_start = idx + 1
+ body_end = body_start
+ while body_end < end_idx and depth > 0:
+ current = lines[body_end]
+ if current.startswith("for "):
+ depth += 1
+ elif current == "endfor":
+ depth -= 1
+ if depth == 0:
+ break
+ body_end += 1
+
+ for i in range(loop_start, loop_end):
+ env[loop_var] = i
+ returned, value = execute_block(lines, env, body_start, body_end)
+ if returned:
+ return True, value
+
+ idx = body_end + 1
+ continue
+
+ if line.startswith("if "):
+ cond_text = line.replace("if ", "", 1).strip()
+ cond_value = resolve_condition(cond_text, env)
+
+ depth = 1
+ body_start = idx + 1
+ scan_idx = body_start
+ else_idx = None
+ endif_idx = None
+ while scan_idx < end_idx:
+ current = lines[scan_idx]
+ if current.startswith("if "):
+ depth += 1
+ elif current == "endif":
+ depth -= 1
+ if depth == 0:
+ endif_idx = scan_idx
+ break
+ elif current == "else" and depth == 1:
+ else_idx = scan_idx
+ scan_idx += 1
+
+ if endif_idx is None:
+ raise ValueError("if without matching endif")
+
+ if cond_value:
+ branch_start = body_start
+ branch_end = else_idx if else_idx is not None else endif_idx
+ else:
+ branch_start = else_idx + 1 if else_idx is not None else endif_idx
+ branch_end = endif_idx
+
+ returned, value = execute_block(lines, env, branch_start, branch_end)
+ if returned:
+ return True, value
+
+ idx = endif_idx + 1
+ continue
+
+ if line in {"else", "endif"}:
+ return False, None
+
+ if line == "endfor":
+ return False, None
+
+ if line.startswith("return "):
+ return True, resolve_value(line.replace("return ", "", 1).strip(), env)
+
+ dest_name, expr = [p.strip() for p in line.split("=", 1)]
+ op_name = expr[:expr.index("(")]
+ arg_text = expr[expr.index("(") + 1: expr.rindex(")")]
+ args = [a.strip() for a in arg_text.split(",") if a.strip()]
+
+ plain_args = []
+ kwargs = {}
+ for arg in args:
+ if ":" in arg:
+ key, value = arg.split(":", 1)
+ kwargs[key.strip()] = int(value.strip())
+ else:
+ plain_args.append(resolve_value(arg, env))
+
+ if op_name == "add":
+ env[dest_name] = apply_add(plain_args[0], plain_args[1])
+ elif op_name == "sub":
+ env[dest_name] = apply_binary(plain_args[0], plain_args[1], lambda a, b: a - b)
+ elif op_name == "mul":
+ env[dest_name] = apply_binary(plain_args[0], plain_args[1], lambda a, b: a * b)
+ elif op_name == "div":
+ env[dest_name] = apply_binary(plain_args[0], plain_args[1], lambda a, b: a / b)
+ elif op_name == "relu":
+ env[dest_name] = apply_relu(plain_args[0])
+ elif op_name == "gelu":
+ env[dest_name] = apply_gelu(plain_args[0])
+ elif op_name == "softmax":
+ env[dest_name] = apply_softmax(plain_args[0])
+ elif op_name == "maxpool":
+ env[dest_name] = apply_maxpool(
+ plain_args[0],
+ kwargs.get("kernel", 2),
+ kwargs.get("stride", 2),
+ )
+ elif op_name == "dot":
+ env[dest_name] = apply_dot(plain_args[0], plain_args[1], kwargs["len"])
+ elif op_name == "matmul":
+ env[dest_name] = apply_matmul(
+ plain_args[0],
+ plain_args[1],
+ kwargs["m"],
+ kwargs["n"],
+ kwargs["k"],
+ )
+ else:
+ raise ValueError(f"Unsupported op in reference executor: {op_name}")
+
+ idx += 1
+
+ return False, None
+
+
+def execute_dsl_reference(dsl_file: Path, inputs):
+ raw_lines = dsl_file.read_text(encoding="utf-8").splitlines()
+ lines = []
+ for raw_line in raw_lines:
+ line = raw_line.strip()
+ if not line or line.startswith("#"):
+ continue
+ lines.append(line)
+
+ env = dict(inputs)
+ returned, value = execute_block(lines, env)
+ return value if returned else None
+
+
+def values_equal(lhs, rhs):
+ if isinstance(lhs, list) and isinstance(rhs, list):
+ if len(lhs) != len(rhs):
+ return False
+ return all(values_equal(a, b) for a, b in zip(lhs, rhs))
+ if isinstance(lhs, float) or isinstance(rhs, float):
+ return math.isclose(lhs, rhs, rel_tol=1e-7, abs_tol=1e-7)
+ return lhs == rhs
+
+
+def summarize_benchmark_runs(instr_counts):
+ if not instr_counts:
+ return {
+ "runs": 0,
+ "avg_instr_count": 0.0,
+ "min_instr_count": 0,
+ "max_instr_count": 0,
+ "ci95_instr_count": 0.0,
+ }
+ avg = sum(instr_counts) / len(instr_counts)
+ if len(instr_counts) > 1:
+ variance = sum((value - avg) ** 2 for value in instr_counts) / (len(instr_counts) - 1)
+ ci95 = 1.96 * math.sqrt(variance) / math.sqrt(len(instr_counts))
+ else:
+ ci95 = 0.0
+ return {
+ "runs": len(instr_counts),
+ "avg_instr_count": avg,
+ "min_instr_count": min(instr_counts),
+ "max_instr_count": max(instr_counts),
+ "ci95_instr_count": ci95,
+ }
+
+
+def detect_regression(avg_instr_count, baseline_instr_count):
+ delta = avg_instr_count - baseline_instr_count
+ delta_pct = 0.0 if baseline_instr_count == 0 else (delta / baseline_instr_count) * 100.0
+ return {
+ "baseline_instr_count": baseline_instr_count,
+ "delta": round(delta, 4),
+ "delta_pct": round(delta_pct, 4),
+ "threshold_pct": REGRESSION_THRESHOLD_PCT,
+ "regressed": delta_pct > REGRESSION_THRESHOLD_PCT,
+ }
+
+
+def load_baseline():
+ if not BASELINE_FILE.exists():
+ return {}
+ return json.loads(BASELINE_FILE.read_text(encoding="utf-8"))
+
+
+def save_baseline(results):
+ REPORT_DIR.mkdir(exist_ok=True)
+ payload = {}
+ for r in results:
+ payload[r["name"]] = {
+ "category": r["category"],
+ "avg_instr_count": r["avg_instr_count"],
+ "runs": r["benchmark_runs"],
+ }
+ BASELINE_FILE.write_text(json.dumps(payload, indent=2), encoding="utf-8")
+
+
+def generate_report_text(results, passed, failed):
+ lines = []
+ lines.append("# ScratchV DSL 编译器性能测试报告\n\n")
+
+ lines.append("## 测试概览\n\n")
+ lines.append(f"- 用例总数: {len(results)}\n")
+ lines.append(f"- 通过数量: {passed}\n")
+ lines.append(f"- 失败数量: {failed}\n\n")
+
+ benchmark_mode = any(r.get("benchmark_runs", 1) > 1 for r in results)
+ if benchmark_mode:
+ lines.append("## 性能基准概览\n\n")
+ lines.append("- 运行模式: benchmark\n")
+ lines.append(f"- 性能基线文件: `{BASELINE_FILE}`\n\n")
+
+ lines.append("## 测试结果\n\n")
+ if benchmark_mode:
+ lines.append("| 测试用例 | 类别 | 状态 | 模拟后端 | 平均指令数 | 最小值 | 最大值 | 基线 | 变化率 | 是否退化 | 预期输出 | 实际输出 | 是否匹配 | 汇编文件 |\n")
+ lines.append("|---|---|---|---|---:|---:|---:|---:|---:|---|---|---|---|---|\n")
+ else:
+ lines.append("| 测试用例 | 类别 | 状态 | 模拟后端 | 指令数 | 预期输出 | 实际输出 | 是否匹配 | 汇编文件 |\n")
+ lines.append("|---|---|---|---|---:|---|---|---|---|\n")
+
+ for r in results:
+ expected = str(r["expected"]).replace("\n", " ").replace("|", "\\|")
+ actual = str(r["actual"]).replace("\n", " ").replace("|", "\\|")
+ if benchmark_mode:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['avg_instr_count']:.2f} | {r['min_instr_count']} | {r['max_instr_count']} | "
+ f"{r['baseline_instr_count']:.2f} | {r['delta_pct']:.2f} | {r['regressed']} | "
+ f"{expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+ else:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['instr_count']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+
+ lines.append("\n## 性能图表\n\n")
+ chart_cases = [r["name"] for r in results]
+ chart_instr = [str(round(r.get("avg_instr_count", r["instr_count"]), 2)) for r in results]
+ lines.append("### 各测试用例指令数\n\n")
+ lines.append("```mermaid\n")
+ lines.append("xychart-beta\n")
+ lines.append(' title "各测试用例指令数"\n')
+ lines.append(" x-axis [" + ", ".join(f'"{name}"' for name in chart_cases) + "]\n")
+ max_instr = max((r.get("avg_instr_count", r["instr_count"]) for r in results), default=0)
+ lines.append(f' y-axis "指令数" 0 --> {max_instr + 2}\n')
+ lines.append(" bar [" + ", ".join(chart_instr) + "]\n")
+ lines.append("```\n\n")
+
+ category_totals = {}
+ for r in results:
+ category_totals[r["category"]] = category_totals.get(r["category"], 0) + r.get("avg_instr_count", r["instr_count"])
+ lines.append("### 各类别指令数占比\n\n")
+ lines.append("```mermaid\n")
+ lines.append("pie showData\n")
+ lines.append(' title 各类别指令数占比\n')
+ for category, total in sorted(category_totals.items()):
+ lines.append(f' "{category}" : {total}\n')
+ lines.append("```\n")
+
+ lines.append("\n## 用例详情\n\n")
+ for r in results:
+ lines.append(f"### {r['name']}\n\n")
+ lines.append(f"- 类别: {r['category']}\n")
+ lines.append(f"- 描述: {r['description']}\n")
+ lines.append(f"- 预期输出 ({r['expected_type']}): {r['expected']}\n")
+ lines.append(f"- 实际输出: {r['actual']}\n")
+ lines.append(f"- 是否匹配: {r['matched']}\n")
+ lines.append(f"- 模拟后端: {r['backend']}\n")
+ if benchmark_mode:
+ lines.append(f"- Benchmark 重复次数: {r['benchmark_runs']}\n")
+ lines.append(f"- 平均指令数: {r['avg_instr_count']:.2f}\n")
+ lines.append(f"- 最小指令数: {r['min_instr_count']}\n")
+ lines.append(f"- 最大指令数: {r['max_instr_count']}\n")
+ lines.append(f"- 基线指令数: {r['baseline_instr_count']:.2f}\n")
+ lines.append(f"- 性能变化率 (%): {r['delta_pct']:.2f}\n")
+ lines.append(f"- 是否性能退化: {r['regressed']}\n")
+ else:
+ lines.append(f"- 指令数: {r['instr_count']}\n")
+ lines.append(f"- 汇编文件: {r['asm']}\n\n")
+
+ return "".join(lines)
+
+
+def write_html_report(results, passed, failed):
+ try:
+ from jinja2 import Template
+ except ImportError:
+ return None
+
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ template = Template("""
+
+
+
+ ScratchV 课程版测试报告
+
+
+
+ ScratchV DSL 编译器性能测试报告
+
+
用例总数:{{ total }},通过:{{ passed }},失败:{{ failed }},通过率:{{ "%.1f"|format(pass_rate) }}%
+
测试目录:{{ test_dir }},性能退化阈值:{{ threshold }}%
+
+
+
+
+ | 用例 | 类别 | 状态 | 平均指令数 | 95%置信区间 | 变化率(%) | 是否退化 | 描述 |
+
+
+ {% for r in results %}
+
+ | {{ r.name }} |
+ {{ r.category }} |
+ {{ r.status }} |
+ {{ "%.2f"|format(r.avg_instr_count) }} |
+ ±{{ "%.2f"|format(r.ci95_instr_count) }} |
+ {{ "%.2f"|format(r.delta_pct) }} |
+ {{ r.regressed }} |
+ {{ r.description }} |
+
+ {% endfor %}
+
+
+
+
+""")
+ HTML_REPORT_FILE.write_text(
+ template.render(
+ total=len(results),
+ passed=passed,
+ failed=failed,
+ pass_rate=pass_rate,
+ test_dir=str(TEST_DIR),
+ threshold=REGRESSION_THRESHOLD_PCT,
+ chart_name=CHART_FILE.name,
+ results=results,
+ ),
+ encoding="utf-8",
+ )
+ return HTML_REPORT_FILE
+
+
+def generate_report_text(results, passed, failed):
+ benchmark_mode = any(r.get("benchmark_runs", 1) > 1 for r in results)
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ lines = [
+ "# ScratchV DSL 编译器性能测试报告\n\n",
+ "## 测试概览\n\n",
+ f"- 生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n",
+ f"- 用例总数: {len(results)}\n",
+ f"- 通过数量: {passed}\n",
+ f"- 失败数量: {failed}\n",
+ f"- 通过率: {pass_rate:.1f}%\n",
+ f"- 测试目录: `{TEST_DIR}`\n",
+ f"- 汇编输出目录: `{BUILD_DIR}`\n",
+ f"- 性能基线文件: `{BASELINE_FILE}`\n",
+ f"- 性能退化阈值: {REGRESSION_THRESHOLD_PCT:.1f}%\n\n",
+ "## 测试结果\n\n",
+ ]
+
+ if benchmark_mode:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 平均指令数 | 95%置信区间 | 最小 | 最大 | 基线 | 变化率(%) | 是否退化 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---:|---:|---:|---:|---:|---|---|---|---|---|\n",
+ ])
+ else:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 指令数 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---|---|---|---|\n",
+ ])
+
+ for r in results:
+ expected = _markdown_cell(r["expected"])
+ actual = _markdown_cell(r["actual"])
+ if benchmark_mode:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['avg_instr_count']:.2f} | ±{r['ci95_instr_count']:.2f} | "
+ f"{r['min_instr_count']} | {r['max_instr_count']} | "
+ f"{r['baseline_instr_count']:.2f} | {r['delta_pct']:.2f} | "
+ f"{r['regressed']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+ else:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['instr_count']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+
+ lines.extend([
+ "\n## 性能图表\n\n",
+ f"\n\n",
+ "### Mermaid 图表\n\n",
+ "```mermaid\n",
+ "xychart-beta\n",
+ ' title "各测试用例指令数"\n',
+ " x-axis [" + ", ".join(f'"{r["name"]}"' for r in results) + "]\n",
+ ])
+ max_instr = max((r.get("avg_instr_count", r["instr_count"]) for r in results), default=0)
+ chart_values = [str(round(r.get("avg_instr_count", r["instr_count"]), 2)) for r in results]
+ lines.extend([
+ f' y-axis "指令数" 0 --> {max_instr + 2}\n',
+ " bar [" + ", ".join(chart_values) + "]\n",
+ "```\n\n",
+ "## 用例详情\n\n",
+ ])
+
+ for r in results:
+ lines.extend([
+ f"### {r['name']}\n\n",
+ f"- 类别: {r['category']}\n",
+ f"- 描述: {r['description']}\n",
+ f"- 预期输出 ({r['expected_type']}): {r['expected']}\n",
+ f"- 实际输出: {r['actual']}\n",
+ f"- 输出是否匹配: {r['matched']}\n",
+ f"- 模拟后端: {r['backend']}\n",
+ f"- 汇编文件: {r['asm']}\n",
+ ])
+ if benchmark_mode:
+ lines.extend([
+ f"- Benchmark 重复次数: {r['benchmark_runs']}\n",
+ f"- 平均指令数: {r['avg_instr_count']:.2f}\n",
+ f"- 95% 置信区间: ±{r['ci95_instr_count']:.2f}\n",
+ f"- 最小指令数: {r['min_instr_count']}\n",
+ f"- 最大指令数: {r['max_instr_count']}\n",
+ f"- 基线指令数: {r['baseline_instr_count']:.2f}\n",
+ f"- 性能变化率: {r['delta_pct']:.2f}%\n",
+ f"- 性能退化阈值: {r['threshold_pct']:.2f}%\n",
+ f"- 是否性能退化: {r['regressed']}\n",
+ ])
+ else:
+ lines.append(f"- 指令数: {r['instr_count']}\n")
+ lines.append("\n")
+
+ return "".join(lines)
+
+
+def write_report(results, passed, failed):
+ REPORT_DIR.mkdir(exist_ok=True)
+
+ REPORT_FILE.write_text(generate_report_text(results, passed, failed), encoding="utf-8")
+ print(f"\nReport written to {REPORT_FILE}")
+
+
+def _markdown_cell(value):
+ return str(value).replace("\n", " ").replace("|", "\\|")
+
+
+def generate_report_text(results, passed, failed):
+ benchmark_mode = any(r.get("benchmark_runs", 1) > 1 for r in results)
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ lines = [
+ "# ScratchV DSL 编译器性能测试报告\n\n",
+ "## 测试概览\n\n",
+ f"- 生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n",
+ f"- 用例总数: {len(results)}\n",
+ f"- 通过数量: {passed}\n",
+ f"- 失败数量: {failed}\n",
+ f"- 通过率: {pass_rate:.1f}%\n",
+ f"- 测试目录: `{TEST_DIR}`\n",
+ f"- 汇编输出目录: `{BUILD_DIR}`\n",
+ f"- 性能基线文件: `{BASELINE_FILE}`\n",
+ f"- 性能退化阈值: {REGRESSION_THRESHOLD_PCT:.1f}%\n\n",
+ "## 测试结果\n\n",
+ ]
+
+ if benchmark_mode:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 平均指令数 | 95%置信区间 | 最小 | 最大 | 基线 | 变化率(%) | 是否退化 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---:|---:|---:|---:|---:|---|---|---|---|---|\n",
+ ])
+ else:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 指令数 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---|---|---|---|\n",
+ ])
+
+ for r in results:
+ expected = _markdown_cell(r["expected"])
+ actual = _markdown_cell(r["actual"])
+ if benchmark_mode:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['avg_instr_count']:.2f} | ±{r['ci95_instr_count']:.2f} | "
+ f"{r['min_instr_count']} | {r['max_instr_count']} | "
+ f"{r['baseline_instr_count']:.2f} | {r['delta_pct']:.2f} | "
+ f"{r['regressed']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+ else:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['instr_count']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+
+ lines.extend([
+ "\n## 性能图表\n\n",
+ f"\n\n",
+ "### Mermaid 图表\n\n",
+ "```mermaid\n",
+ "xychart-beta\n",
+ ' title "各测试用例指令数"\n',
+ " x-axis [" + ", ".join(f'"{r["name"]}"' for r in results) + "]\n",
+ ])
+ max_instr = max((r.get("avg_instr_count", r["instr_count"]) for r in results), default=0)
+ chart_values = [str(round(r.get("avg_instr_count", r["instr_count"]), 2)) for r in results]
+ lines.extend([
+ f' y-axis "指令数" 0 --> {max_instr + 2}\n',
+ " bar [" + ", ".join(chart_values) + "]\n",
+ "```\n\n",
+ "## 用例详情\n\n",
+ ])
+
+ for r in results:
+ lines.extend([
+ f"### {r['name']}\n\n",
+ f"- 类别: {r['category']}\n",
+ f"- 描述: {r['description']}\n",
+ f"- 预期输出 ({r['expected_type']}): {r['expected']}\n",
+ f"- 实际输出: {r['actual']}\n",
+ f"- 输出是否匹配: {r['matched']}\n",
+ f"- 模拟后端: {r['backend']}\n",
+ f"- 汇编文件: {r['asm']}\n",
+ ])
+ if benchmark_mode:
+ lines.extend([
+ f"- Benchmark 重复次数: {r['benchmark_runs']}\n",
+ f"- 平均指令数: {r['avg_instr_count']:.2f}\n",
+ f"- 95% 置信区间: ±{r['ci95_instr_count']:.2f}\n",
+ f"- 最小指令数: {r['min_instr_count']}\n",
+ f"- 最大指令数: {r['max_instr_count']}\n",
+ f"- 基线指令数: {r['baseline_instr_count']:.2f}\n",
+ f"- 性能变化率: {r['delta_pct']:.2f}%\n",
+ f"- 性能退化阈值: {r['threshold_pct']:.2f}%\n",
+ f"- 是否性能退化: {r['regressed']}\n",
+ ])
+ else:
+ lines.append(f"- 指令数: {r['instr_count']}\n")
+ lines.append("\n")
+
+ return "".join(lines)
+
+
+def generate_report_text(results, passed, failed):
+ benchmark_mode = any(r.get("benchmark_runs", 1) > 1 for r in results)
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ lines = [
+ "# ScratchV DSL 编译器性能测试报告\n\n",
+ "## 测试概览\n\n",
+ f"- 生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n",
+ f"- 用例总数: {len(results)}\n",
+ f"- 通过数量: {passed}\n",
+ f"- 失败数量: {failed}\n",
+ f"- 通过率: {pass_rate:.1f}%\n",
+ f"- 测试目录: `{TEST_DIR}`\n",
+ f"- 汇编输出目录: `{BUILD_DIR}`\n",
+ f"- 性能基线文件: `{BASELINE_FILE}`\n",
+ f"- 性能退化阈值: {REGRESSION_THRESHOLD_PCT:.1f}%\n\n",
+ "## 测试结果\n\n",
+ ]
+
+ if benchmark_mode:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 平均指令数 | 95%置信区间 | 最小 | 最大 | 基线 | 变化率(%) | 是否退化 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---:|---:|---:|---:|---:|---|---|---|---|---|\n",
+ ])
+ else:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 指令数 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---|---|---|---|\n",
+ ])
+
+ for r in results:
+ expected = _markdown_cell(r["expected"])
+ actual = _markdown_cell(r["actual"])
+ if benchmark_mode:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['avg_instr_count']:.2f} | ±{r['ci95_instr_count']:.2f} | "
+ f"{r['min_instr_count']} | {r['max_instr_count']} | "
+ f"{r['baseline_instr_count']:.2f} | {r['delta_pct']:.2f} | "
+ f"{r['regressed']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+ else:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['instr_count']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+
+ lines.extend([
+ "\n## 性能图表\n\n",
+ f"\n\n",
+ "### Mermaid 图表\n\n",
+ "```mermaid\n",
+ "xychart-beta\n",
+ ' title "各测试用例指令数"\n',
+ " x-axis [" + ", ".join(f'"{r["name"]}"' for r in results) + "]\n",
+ ])
+ max_instr = max((r.get("avg_instr_count", r["instr_count"]) for r in results), default=0)
+ chart_values = [str(round(r.get("avg_instr_count", r["instr_count"]), 2)) for r in results]
+ lines.extend([
+ f' y-axis "指令数" 0 --> {max_instr + 2}\n',
+ " bar [" + ", ".join(chart_values) + "]\n",
+ "```\n\n",
+ "## 用例详情\n\n",
+ ])
+
+ for r in results:
+ lines.extend([
+ f"### {r['name']}\n\n",
+ f"- 类别: {r['category']}\n",
+ f"- 描述: {r['description']}\n",
+ f"- 预期输出 ({r['expected_type']}): {r['expected']}\n",
+ f"- 实际输出: {r['actual']}\n",
+ f"- 输出是否匹配: {r['matched']}\n",
+ f"- 模拟后端: {r['backend']}\n",
+ f"- 汇编文件: {r['asm']}\n",
+ ])
+ if benchmark_mode:
+ lines.extend([
+ f"- Benchmark 重复次数: {r['benchmark_runs']}\n",
+ f"- 平均指令数: {r['avg_instr_count']:.2f}\n",
+ f"- 95% 置信区间: ±{r['ci95_instr_count']:.2f}\n",
+ f"- 最小指令数: {r['min_instr_count']}\n",
+ f"- 最大指令数: {r['max_instr_count']}\n",
+ f"- 基线指令数: {r['baseline_instr_count']:.2f}\n",
+ f"- 性能变化率: {r['delta_pct']:.2f}%\n",
+ f"- 性能退化阈值: {r['threshold_pct']:.2f}%\n",
+ f"- 是否性能退化: {r['regressed']}\n",
+ ])
+ else:
+ lines.append(f"- 指令数: {r['instr_count']}\n")
+ lines.append("\n")
+
+ return "".join(lines)
+
+
+def write_chart(results):
+ try:
+ mpl_config_dir = Path(tempfile.gettempdir()) / "scratchv-matplotlib"
+ mpl_config_dir.mkdir(parents=True, exist_ok=True)
+ os.environ.setdefault("MPLCONFIGDIR", str(mpl_config_dir))
+ import matplotlib
+ matplotlib.use("Agg")
+ import matplotlib.pyplot as plt
+ except ImportError:
+ return None
+
+ names = [r["name"] for r in results]
+ values = [r.get("avg_instr_count", r["instr_count"]) for r in results]
+ width = max(10, len(names) * 0.45)
+ fig, ax = plt.subplots(figsize=(width, 5))
+ ax.bar(range(len(names)), values, color="#2563eb")
+ ax.set_title("ScratchV Course Benchmark Instruction Counts")
+ ax.set_ylabel("Instructions")
+ ax.set_xticks(range(len(names)))
+ ax.set_xticklabels(names, rotation=60, ha="right", fontsize=8)
+ ax.grid(axis="y", linestyle="--", alpha=0.35)
+ fig.tight_layout()
+ fig.savefig(CHART_FILE, dpi=160)
+ plt.close(fig)
+ return CHART_FILE
+
+
+def write_html_report(results, passed, failed):
+ try:
+ from jinja2 import Template
+ except ImportError:
+ return None
+
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ template = Template("""
+
+
+
+ ScratchV 课程版测试报告
+
+
+
+ ScratchV DSL 编译器性能测试报告
+
+
用例总数:{{ total }},通过:{{ passed }},失败:{{ failed }},通过率:{{ "%.1f"|format(pass_rate) }}%
+
测试目录:{{ test_dir }},性能退化阈值:{{ threshold }}%
+
+
+
+
+ | 用例 | 类别 | 状态 | 平均指令数 | 95%置信区间 | 变化率(%) | 是否退化 | 描述 |
+
+
+ {% for r in results %}
+
+ | {{ r.name }} |
+ {{ r.category }} |
+ {{ r.status }} |
+ {{ "%.2f"|format(r.avg_instr_count) }} |
+ ±{{ "%.2f"|format(r.ci95_instr_count) }} |
+ {{ "%.2f"|format(r.delta_pct) }} |
+ {{ r.regressed }} |
+ {{ r.description }} |
+
+ {% endfor %}
+
+
+
+
+""")
+ HTML_REPORT_FILE.write_text(
+ template.render(
+ total=len(results),
+ passed=passed,
+ failed=failed,
+ pass_rate=pass_rate,
+ test_dir=str(TEST_DIR),
+ threshold=REGRESSION_THRESHOLD_PCT,
+ chart_name=CHART_FILE.name,
+ results=results,
+ ),
+ encoding="utf-8",
+ )
+ return HTML_REPORT_FILE
+
+
+def generate_report_text_cn(results, passed, failed):
+ benchmark_mode = any(r.get("benchmark_runs", 1) > 1 for r in results)
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ lines = [
+ "# ScratchV DSL 编译器性能测试报告\n\n",
+ "## 测试概览\n\n",
+ f"- 生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n",
+ f"- 用例总数: {len(results)}\n",
+ f"- 通过数量: {passed}\n",
+ f"- 失败数量: {failed}\n",
+ f"- 通过率: {pass_rate:.1f}%\n",
+ f"- 测试目录: `{TEST_DIR}`\n",
+ f"- 汇编输出目录: `{BUILD_DIR}`\n",
+ f"- 性能基线文件: `{BASELINE_FILE}`\n",
+ f"- 性能退化阈值: {REGRESSION_THRESHOLD_PCT:.1f}%\n\n",
+ "## 测试结果\n\n",
+ ]
+
+ if benchmark_mode:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 平均指令数 | 95%置信区间 | 最小 | 最大 | 基线 | 变化率(%) | 是否退化 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---:|---:|---:|---:|---:|---|---|---|---|---|\n",
+ ])
+ else:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 指令数 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---|---|---|---|\n",
+ ])
+
+ for r in results:
+ expected = _markdown_cell(r["expected"])
+ actual = _markdown_cell(r["actual"])
+ if benchmark_mode:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['avg_instr_count']:.2f} | ±{r['ci95_instr_count']:.2f} | "
+ f"{r['min_instr_count']} | {r['max_instr_count']} | "
+ f"{r['baseline_instr_count']:.2f} | {r['delta_pct']:.2f} | "
+ f"{r['regressed']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+ else:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['instr_count']} | {expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+
+ lines.extend([
+ "\n## 性能图表\n\n",
+ f"\n\n",
+ "### Mermaid 图表\n\n",
+ "```mermaid\n",
+ "xychart-beta\n",
+ ' title "各测试用例指令数"\n',
+ " x-axis [" + ", ".join(f'"{r["name"]}"' for r in results) + "]\n",
+ ])
+ max_instr = max((r.get("avg_instr_count", r["instr_count"]) for r in results), default=0)
+ chart_values = [str(round(r.get("avg_instr_count", r["instr_count"]), 2)) for r in results]
+ lines.extend([
+ f' y-axis "指令数" 0 --> {max_instr + 2}\n',
+ " bar [" + ", ".join(chart_values) + "]\n",
+ "```\n\n",
+ "## 用例详情\n\n",
+ ])
+
+ for r in results:
+ lines.extend([
+ f"### {r['name']}\n\n",
+ f"- 类别: {r['category']}\n",
+ f"- 描述: {r['description']}\n",
+ f"- 预期输出 ({r['expected_type']}): {r['expected']}\n",
+ f"- 实际输出: {r['actual']}\n",
+ f"- 输出是否匹配: {r['matched']}\n",
+ f"- 模拟后端: {r['backend']}\n",
+ f"- 汇编文件: {r['asm']}\n",
+ ])
+ if benchmark_mode:
+ lines.extend([
+ f"- Benchmark 重复次数: {r['benchmark_runs']}\n",
+ f"- 平均指令数: {r['avg_instr_count']:.2f}\n",
+ f"- 95% 置信区间: ±{r['ci95_instr_count']:.2f}\n",
+ f"- 最小指令数: {r['min_instr_count']}\n",
+ f"- 最大指令数: {r['max_instr_count']}\n",
+ f"- 基线指令数: {r['baseline_instr_count']:.2f}\n",
+ f"- 性能变化率: {r['delta_pct']:.2f}%\n",
+ f"- 性能退化阈值: {r['threshold_pct']:.2f}%\n",
+ f"- 是否性能退化: {r['regressed']}\n",
+ ])
+ else:
+ lines.append(f"- 指令数: {r['instr_count']}\n")
+ lines.append("\n")
+
+ return "".join(lines)
+
+
+def write_html_report_cn(results, passed, failed):
+ try:
+ from jinja2 import Template
+ except ImportError:
+ return None
+
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ template = Template("""
+
+
+
+ ScratchV 课程版测试报告
+
+
+
+ ScratchV DSL 编译器性能测试报告
+
+
用例总数:{{ total }},通过:{{ passed }},失败:{{ failed }},通过率:{{ "%.1f"|format(pass_rate) }}%
+
测试目录:{{ test_dir }},性能退化阈值:{{ threshold }}%
+
+
+
+
+ | 用例 | 类别 | 状态 | 平均指令数 | 95%置信区间 | 变化率(%) | 是否退化 | 描述 |
+
+
+ {% for r in results %}
+
+ | {{ r.name }} |
+ {{ r.category }} |
+ {{ r.status }} |
+ {{ "%.2f"|format(r.avg_instr_count) }} |
+ ±{{ "%.2f"|format(r.ci95_instr_count) }} |
+ {{ "%.2f"|format(r.delta_pct) }} |
+ {{ r.regressed }} |
+ {{ r.description }} |
+
+ {% endfor %}
+
+
+
+
+""")
+ HTML_REPORT_FILE.write_text(
+ template.render(
+ total=len(results),
+ passed=passed,
+ failed=failed,
+ pass_rate=pass_rate,
+ test_dir=str(TEST_DIR),
+ threshold=REGRESSION_THRESHOLD_PCT,
+ chart_name=CHART_FILE.name,
+ results=results,
+ ),
+ encoding="utf-8",
+ )
+ return HTML_REPORT_FILE
+
+
+def generate_report_text_cn(results, passed, failed):
+ benchmark_mode = any(r.get("benchmark_runs", 1) > 1 for r in results)
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ lines = [
+ "# ScratchV DSL 编译器性能测试报告\n\n",
+ "## 测试概览\n\n",
+ f"- 生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\n",
+ f"- 用例总数: {len(results)}\n",
+ f"- 通过数量: {passed}\n",
+ f"- 失败数量: {failed}\n",
+ f"- 通过率: {pass_rate:.1f}%\n",
+ f"- 测试目录: `{TEST_DIR}`\n",
+ f"- 汇编输出目录: `{BUILD_DIR}`\n",
+ f"- 性能基线文件: `{BASELINE_FILE}`\n",
+ f"- 性能退化阈值: {REGRESSION_THRESHOLD_PCT:.1f}%\n",
+ f"- 单次模拟超时: {SIMULATION_TIMEOUT_SEC:.0f}s\n\n",
+ "## 测试结果\n\n",
+ ]
+
+ if benchmark_mode:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 平均指令数 | 95% 置信区间 | 最小 | 最大 | 编译耗时(s) | 模拟耗时(s) | 总耗时(s) | 基线 | 变化率(%) | 是否退化 | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---|---|---|---|---|\n",
+ ])
+ else:
+ lines.extend([
+ "| 用例 | 类别 | 状态 | 模拟后端 | 指令数 | 编译耗时(s) | 模拟耗时(s) | 总耗时(s) | 预期输出 | 实际输出 | 输出匹配 | 汇编文件 |\n",
+ "|---|---|---|---|---:|---:|---:|---:|---|---|---|---|\n",
+ ])
+
+ for r in results:
+ expected = _markdown_cell(r["expected"])
+ actual = _markdown_cell(r["actual"])
+ if benchmark_mode:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['avg_instr_count']:.2f} | ±{r['ci95_instr_count']:.2f} | "
+ f"{r['min_instr_count']} | {r['max_instr_count']} | "
+ f"{r['compile_time_sec']:.4f} | {r['simulation_time_sec']:.4f} | "
+ f"{r['total_time_sec']:.4f} | {r['baseline_instr_count']:.2f} | "
+ f"{r['delta_pct']:.2f} | {r['regressed']} | {expected} | "
+ f"{actual} | {r['matched']} | {r['asm']} |\n"
+ )
+ else:
+ lines.append(
+ f"| {r['name']} | {r['category']} | {r['status']} | {r['backend']} | "
+ f"{r['instr_count']} | {r['compile_time_sec']:.4f} | "
+ f"{r['simulation_time_sec']:.4f} | {r['total_time_sec']:.4f} | "
+ f"{expected} | {actual} | {r['matched']} | {r['asm']} |\n"
+ )
+
+ lines.extend([
+ "\n## 性能图表\n\n",
+ f"\n\n",
+ "### Mermaid 图表\n\n",
+ "```mermaid\n",
+ "xychart-beta\n",
+ ' title "各测试用例指令数"\n',
+ " x-axis [" + ", ".join(f'"{r["name"]}"' for r in results) + "]\n",
+ ])
+ max_instr = max((r.get("avg_instr_count", r["instr_count"]) for r in results), default=0)
+ chart_values = [str(round(r.get("avg_instr_count", r["instr_count"]), 2)) for r in results]
+ lines.extend([
+ f' y-axis "指令数" 0 --> {max_instr + 2}\n',
+ " bar [" + ", ".join(chart_values) + "]\n",
+ "```\n\n",
+ "## 用例详情\n\n",
+ ])
+
+ for r in results:
+ lines.extend([
+ f"### {r['name']}\n\n",
+ f"- 类别: {r['category']}\n",
+ f"- 描述: {r['description']}\n",
+ f"- 预期输出 ({r['expected_type']}): {r['expected']}\n",
+ f"- 实际输出: {r['actual']}\n",
+ f"- 输出是否匹配: {r['matched']}\n",
+ f"- 模拟后端: {r['backend']}\n",
+ f"- 编译耗时(s): {r['compile_time_sec']:.4f}\n",
+ f"- 模拟耗时(s): {r['simulation_time_sec']:.4f}\n",
+ f"- 参考解释器耗时(s): {r['reference_time_sec']:.4f}\n",
+ f"- 总耗时(s): {r['total_time_sec']:.4f}\n",
+ f"- 汇编文件: {r['asm']}\n",
+ ])
+ if benchmark_mode:
+ lines.extend([
+ f"- Benchmark 重复次数: {r['benchmark_runs']}\n",
+ f"- 平均指令数: {r['avg_instr_count']:.2f}\n",
+ f"- 95% 置信区间: ±{r['ci95_instr_count']:.2f}\n",
+ f"- 最小指令数: {r['min_instr_count']}\n",
+ f"- 最大指令数: {r['max_instr_count']}\n",
+ f"- 基线指令数: {r['baseline_instr_count']:.2f}\n",
+ f"- 性能变化率: {r['delta_pct']:.2f}%\n",
+ f"- 性能退化阈值: {r['threshold_pct']:.2f}%\n",
+ f"- 是否性能退化: {r['regressed']}\n",
+ ])
+ else:
+ lines.append(f"- 指令数: {r['instr_count']}\n")
+ lines.append("\n")
+
+ return "".join(lines)
+
+
+def write_html_report_cn(results, passed, failed):
+ try:
+ from jinja2 import Template
+ except ImportError:
+ return None
+
+ pass_rate = 0.0 if not results else passed / len(results) * 100.0
+ template = Template("""
+
+
+
+ ScratchV 测试报告
+
+
+
+ ScratchV DSL 编译器性能测试报告
+
+
用例总数:{{ total }},通过:{{ passed }},失败:{{ failed }},通过率:{{ "%.1f"|format(pass_rate) }}%
+
测试目录:{{ test_dir }},性能退化阈值:{{ threshold }}%,单次模拟超时:{{ timeout }}s
+
+
+
+
+
+ | 用例 | 类别 | 状态 | 模拟后端 |
+ 平均指令数 | 95% 置信区间 |
+ 编译耗时(s) | 模拟耗时(s) | 总耗时(s) |
+ 变化率(%) | 是否退化 | 描述 |
+
+
+
+ {% for r in results %}
+
+ | {{ r.name }} |
+ {{ r.category }} |
+ {{ r.status }} |
+ {{ r.backend }} |
+ {{ "%.2f"|format(r.avg_instr_count) }} |
+ ±{{ "%.2f"|format(r.ci95_instr_count) }} |
+ {{ "%.4f"|format(r.compile_time_sec) }} |
+ {{ "%.4f"|format(r.simulation_time_sec) }} |
+ {{ "%.4f"|format(r.total_time_sec) }} |
+ {{ "%.2f"|format(r.delta_pct) }} |
+ {{ r.regressed }} |
+ {{ r.description }} |
+
+ {% endfor %}
+
+
+
+
+""")
+ HTML_REPORT_FILE.write_text(
+ template.render(
+ total=len(results),
+ passed=passed,
+ failed=failed,
+ pass_rate=pass_rate,
+ test_dir=str(TEST_DIR),
+ threshold=REGRESSION_THRESHOLD_PCT,
+ timeout=SIMULATION_TIMEOUT_SEC,
+ chart_name=CHART_FILE.name,
+ results=results,
+ ),
+ encoding="utf-8",
+ )
+ return HTML_REPORT_FILE
+
+
+def write_report(results, passed, failed):
+ REPORT_DIR.mkdir(exist_ok=True)
+ write_chart(results)
+ REPORT_FILE.write_text(generate_report_text_cn(results, passed, failed), encoding="utf-8")
+ html_path = write_html_report_cn(results, passed, failed)
+ print(f"\nMarkdown report written to {REPORT_FILE}")
+ if html_path:
+ print(f"HTML report written to {html_path}")
+
+
+def parse_args(argv=None):
+ parser = argparse.ArgumentParser(description="Run ScratchV DSL benchmark suite.")
+ parser.add_argument("--benchmark", type=int, default=0, metavar="N",
+ help="Run each case N times and report average instruction count.")
+ parser.add_argument("--update-baseline", action="store_true",
+ help="Write current benchmark averages to the baseline file.")
+ return parser.parse_args(argv)
+
+
+def main(argv=None):
+ args = parse_args(argv)
+ BUILD_DIR.mkdir(exist_ok=True)
+ baseline = load_baseline() if args.benchmark else {}
+
+ dsl_files = list(TEST_DIR.rglob("*.dsl"))
+
+ if not dsl_files:
+ print("No DSL test files found.")
+ return
+
+ passed = 0
+ failed = 0
+ results = []
+
+ print("Running DSL compiler tests...")
+ print("=" * 50)
+
+ for dsl_file in dsl_files:
+ print(f"\n[TEST] {dsl_file}")
+
+ case_start = time.perf_counter()
+ meta = load_metadata(dsl_file)
+ compile_start = time.perf_counter()
+ result, output_file = run_compile(dsl_file)
+ compile_time_sec = time.perf_counter() - compile_start
+ simulation_start = time.perf_counter()
+ sim_result = run_simulation(output_file) if result.returncode == 0 else {
+ "success": False,
+ "instr_count": 0,
+ "return_value": None,
+ "backend": "none",
+ "error": "compile failed",
+ }
+ simulation_time_sec = time.perf_counter() - simulation_start
+ expected_value = meta.get("expected_return")
+ reference_start = time.perf_counter()
+ actual_value = execute_dsl_reference(dsl_file, meta.get("inputs", {}))
+ reference_time_sec = time.perf_counter() - reference_start
+ matched = values_equal(actual_value, expected_value)
+ benchmark_counts = []
+ benchmark_summary = {
+ "runs": 1,
+ "avg_instr_count": sim_result.get("instr_count", 0),
+ "min_instr_count": sim_result.get("instr_count", 0),
+ "max_instr_count": sim_result.get("instr_count", 0),
+ "ci95_instr_count": 0.0,
+ }
+ regression = {
+ "baseline_instr_count": 0.0,
+ "delta": 0.0,
+ "delta_pct": 0.0,
+ "threshold_pct": REGRESSION_THRESHOLD_PCT,
+ "regressed": False,
+ }
+
+ if args.benchmark > 0 and result.returncode == 0 and output_file.exists():
+ for _ in range(args.benchmark):
+ benchmark_counts.append(run_simulation(output_file).get("instr_count", 0))
+ benchmark_summary = summarize_benchmark_runs(benchmark_counts)
+ baseline_entry = baseline.get(dsl_file.stem)
+ if baseline_entry:
+ regression = detect_regression(
+ avg_instr_count=benchmark_summary["avg_instr_count"],
+ baseline_instr_count=baseline_entry.get("avg_instr_count", 0.0),
+ )
+ total_time_sec = time.perf_counter() - case_start
+
+ ok = (
+ result.returncode == 0
+ and output_file.exists()
+ and sim_result["success"]
+ and matched
+ and not regression["regressed"]
+ )
+
+ if ok:
+ print("PASS")
+ passed += 1
+ status = "PASS"
+ else:
+ print("FAIL")
+ failed += 1
+ status = "FAIL"
+ print(sim_result.get("error") or result.stderr or result.stdout)
+
+ results.append({
+ "name": dsl_file.stem,
+ "category": dsl_file.parent.name,
+ "path": str(dsl_file),
+ "status": status,
+ "description": meta.get("description", ""),
+ "expected_type": meta.get("expected_output_type", "return_value"),
+ "expected": expected_value,
+ "actual": actual_value,
+ "matched": matched,
+ "backend": sim_result.get("backend", "none"),
+ "instr_count": sim_result.get("instr_count", 0),
+ "compile_time_sec": compile_time_sec,
+ "simulation_time_sec": simulation_time_sec,
+ "reference_time_sec": reference_time_sec,
+ "total_time_sec": total_time_sec,
+ "benchmark_runs": benchmark_summary["runs"],
+ "avg_instr_count": benchmark_summary["avg_instr_count"],
+ "min_instr_count": benchmark_summary["min_instr_count"],
+ "max_instr_count": benchmark_summary["max_instr_count"],
+ "ci95_instr_count": benchmark_summary["ci95_instr_count"],
+ "baseline_instr_count": regression["baseline_instr_count"],
+ "delta": regression["delta"],
+ "delta_pct": regression["delta_pct"],
+ "threshold_pct": regression["threshold_pct"],
+ "regressed": regression["regressed"],
+ "asm": str(output_file),
+ })
+
+ print("\n" + "=" * 50)
+ print(f"Total: {len(dsl_files)}")
+ print(f"Passed: {passed}")
+ print(f"Failed: {failed}")
+
+ if args.benchmark and args.update_baseline:
+ save_baseline(results)
+ print(f"Baseline written to {BASELINE_FILE}")
+
+ write_report(results, passed, failed)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ScratchV-topic06-deliverable/setup.py b/ScratchV-topic06-deliverable/setup.py
new file mode 100644
index 0000000..6068493
--- /dev/null
+++ b/ScratchV-topic06-deliverable/setup.py
@@ -0,0 +1,3 @@
+from setuptools import setup
+
+setup()
diff --git a/ScratchV-topic06-deliverable/tests_main/activation/add_relu_relu.dsl b/ScratchV-topic06-deliverable/tests_main/activation/add_relu_relu.dsl
new file mode 100644
index 0000000..52c1325
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/activation/add_relu_relu.dsl
@@ -0,0 +1,5 @@
+# Add followed by two ReLU stages
+x = add(input, bias)
+y = relu(x)
+result = relu(y)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/activation/add_relu_relu.meta.json b/ScratchV-topic06-deliverable/tests_main/activation/add_relu_relu.meta.json
new file mode 100644
index 0000000..e587d8a
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/activation/add_relu_relu.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Add input and bias, then apply ReLU twice.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "input": -3,
+ "bias": 10
+ },
+ "expected_return": 7
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/activation/relu_add.dsl b/ScratchV-topic06-deliverable/tests_main/activation/relu_add.dsl
new file mode 100644
index 0000000..7190772
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/activation/relu_add.dsl
@@ -0,0 +1,4 @@
+# Add + ReLU activation
+x = add(input, bias)
+y = relu(x)
+return y
\ No newline at end of file
diff --git a/ScratchV-topic06-deliverable/tests_main/activation/relu_add.meta.json b/ScratchV-topic06-deliverable/tests_main/activation/relu_add.meta.json
new file mode 100644
index 0000000..f1b7f84
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/activation/relu_add.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Add input and bias, then apply one ReLU.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "input": -2,
+ "bias": 5
+ },
+ "expected_return": 3
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/activation/relu_only.dsl b/ScratchV-topic06-deliverable/tests_main/activation/relu_only.dsl
new file mode 100644
index 0000000..404a911
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/activation/relu_only.dsl
@@ -0,0 +1,3 @@
+# Single ReLU activation
+result = relu(x)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/activation/relu_only.meta.json b/ScratchV-topic06-deliverable/tests_main/activation/relu_only.meta.json
new file mode 100644
index 0000000..b08af80
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/activation/relu_only.meta.json
@@ -0,0 +1,8 @@
+{
+ "description": "Apply ReLU directly to a single input value.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "x": -5
+ },
+ "expected_return": 0
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/activation/relu_twice.dsl b/ScratchV-topic06-deliverable/tests_main/activation/relu_twice.dsl
new file mode 100644
index 0000000..7d2defe
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/activation/relu_twice.dsl
@@ -0,0 +1,4 @@
+# Two-stage ReLU activation
+x = relu(input)
+result = relu(x)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/activation/relu_twice.meta.json b/ScratchV-topic06-deliverable/tests_main/activation/relu_twice.meta.json
new file mode 100644
index 0000000..fabbb38
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/activation/relu_twice.meta.json
@@ -0,0 +1,8 @@
+{
+ "description": "Apply ReLU twice to the same activation path.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "input": 4
+ },
+ "expected_return": 4
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/branch/if_else.dsl b/ScratchV-topic06-deliverable/tests_main/branch/if_else.dsl
new file mode 100644
index 0000000..6541768
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/branch/if_else.dsl
@@ -0,0 +1,8 @@
+# Branch takes the else path when flag is zero.
+if (flag != 0):
+result = add(a, b)
+return result
+else
+result = sub(a, b)
+return result
+endif
diff --git a/ScratchV-topic06-deliverable/tests_main/branch/if_else.meta.json b/ScratchV-topic06-deliverable/tests_main/branch/if_else.meta.json
new file mode 100644
index 0000000..fdacf6a
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/branch/if_else.meta.json
@@ -0,0 +1,10 @@
+{
+ "description": "if/else branch returns subtraction result when flag is zero.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "flag": 0,
+ "a": 9,
+ "b": 4
+ },
+ "expected_return": 5
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/branch/if_relu.dsl b/ScratchV-topic06-deliverable/tests_main/branch/if_relu.dsl
new file mode 100644
index 0000000..1346060
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/branch/if_relu.dsl
@@ -0,0 +1,8 @@
+# Branch selects whether to apply ReLU after an add.
+sum = add(a, b)
+if (use_relu != 0):
+result = relu(sum)
+return result
+else
+return sum
+endif
diff --git a/ScratchV-topic06-deliverable/tests_main/branch/if_relu.meta.json b/ScratchV-topic06-deliverable/tests_main/branch/if_relu.meta.json
new file mode 100644
index 0000000..1925fb5
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/branch/if_relu.meta.json
@@ -0,0 +1,10 @@
+{
+ "description": "if/else branch combined with add and relu.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "use_relu": 1,
+ "a": -8,
+ "b": 3
+ },
+ "expected_return": 0
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/branch/if_then.dsl b/ScratchV-topic06-deliverable/tests_main/branch/if_then.dsl
new file mode 100644
index 0000000..c305c3d
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/branch/if_then.dsl
@@ -0,0 +1,8 @@
+# Branch takes the then path when flag is non-zero.
+if (flag != 0):
+result = add(a, b)
+return result
+else
+result = sub(a, b)
+return result
+endif
diff --git a/ScratchV-topic06-deliverable/tests_main/branch/if_then.meta.json b/ScratchV-topic06-deliverable/tests_main/branch/if_then.meta.json
new file mode 100644
index 0000000..04748d1
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/branch/if_then.meta.json
@@ -0,0 +1,10 @@
+{
+ "description": "if/else branch returns add result when flag is non-zero.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "flag": 1,
+ "a": 9,
+ "b": 4
+ },
+ "expected_return": 13
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain.dsl b/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain.dsl
new file mode 100644
index 0000000..d3917c9
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain.dsl
@@ -0,0 +1,4 @@
+# Chain of two vector adds
+x = add(a, b)
+y = add(x, c)
+return y
\ No newline at end of file
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain.meta.json b/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain.meta.json
new file mode 100644
index 0000000..b07adaf
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain.meta.json
@@ -0,0 +1,10 @@
+{
+ "description": "Add a and b, then add c to the intermediate result.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 2,
+ "b": 3,
+ "c": 4
+ },
+ "expected_return": 9
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain_3.dsl b/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain_3.dsl
new file mode 100644
index 0000000..aeca4df
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain_3.dsl
@@ -0,0 +1,5 @@
+# Chain of three add operations
+x = add(a, b)
+y = add(x, c)
+z = add(y, d)
+return z
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain_3.meta.json b/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain_3.meta.json
new file mode 100644
index 0000000..368b3aa
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/add_chain_3.meta.json
@@ -0,0 +1,11 @@
+{
+ "description": "Chain three add operations across four symbolic inputs.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 2,
+ "b": 3,
+ "c": 4,
+ "d": 5
+ },
+ "expected_return": 14
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/add_fan_in_4.dsl b/ScratchV-topic06-deliverable/tests_main/elementwise/add_fan_in_4.dsl
new file mode 100644
index 0000000..0e4449e
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/add_fan_in_4.dsl
@@ -0,0 +1,5 @@
+# Fan-in add over four inputs
+x = add(a, b)
+y = add(c, d)
+result = add(x, y)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/add_fan_in_4.meta.json b/ScratchV-topic06-deliverable/tests_main/elementwise/add_fan_in_4.meta.json
new file mode 100644
index 0000000..3eb1f4a
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/add_fan_in_4.meta.json
@@ -0,0 +1,11 @@
+{
+ "description": "Compute two independent adds and then merge them with a final add.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 1,
+ "b": 2,
+ "c": 3,
+ "d": 4
+ },
+ "expected_return": 10
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/add_reuse.dsl b/ScratchV-topic06-deliverable/tests_main/elementwise/add_reuse.dsl
new file mode 100644
index 0000000..4618948
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/add_reuse.dsl
@@ -0,0 +1,4 @@
+# Reuse intermediate add result
+x = add(a, b)
+result = add(x, x)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/add_reuse.meta.json b/ScratchV-topic06-deliverable/tests_main/elementwise/add_reuse.meta.json
new file mode 100644
index 0000000..2d2c396
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/add_reuse.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Reuse the same intermediate add result on both operands of a second add.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 2,
+ "b": 3
+ },
+ "expected_return": 10
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/vector_add.dsl b/ScratchV-topic06-deliverable/tests_main/elementwise/vector_add.dsl
new file mode 100644
index 0000000..36ac648
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/vector_add.dsl
@@ -0,0 +1,3 @@
+# Vector add
+result = add(a, b)
+return result
\ No newline at end of file
diff --git a/ScratchV-topic06-deliverable/tests_main/elementwise/vector_add.meta.json b/ScratchV-topic06-deliverable/tests_main/elementwise/vector_add.meta.json
new file mode 100644
index 0000000..7347fff
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/elementwise/vector_add.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Single add over two symbolic vector inputs.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 2,
+ "b": 3
+ },
+ "expected_return": 5
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/loop/loop_add_4.dsl b/ScratchV-topic06-deliverable/tests_main/loop/loop_add_4.dsl
new file mode 100644
index 0000000..381f1c0
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/loop/loop_add_4.dsl
@@ -0,0 +1,5 @@
+# Loop with repeated add body over 4 iterations
+for i = 0, 4
+x = add(a, b)
+endfor
+return x
diff --git a/ScratchV-topic06-deliverable/tests_main/loop/loop_add_4.meta.json b/ScratchV-topic06-deliverable/tests_main/loop/loop_add_4.meta.json
new file mode 100644
index 0000000..fc31fae
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/loop/loop_add_4.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Run a four-iteration loop whose body computes one add; final returned value is the last loop-body result.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 2,
+ "b": 3
+ },
+ "expected_return": 5
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/loop/loop_add_chain_4.dsl b/ScratchV-topic06-deliverable/tests_main/loop/loop_add_chain_4.dsl
new file mode 100644
index 0000000..eca595b
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/loop/loop_add_chain_4.dsl
@@ -0,0 +1,6 @@
+# Loop with chained add inside the body
+for i = 0, 4
+x = add(a, b)
+y = add(x, c)
+endfor
+return y
diff --git a/ScratchV-topic06-deliverable/tests_main/loop/loop_add_chain_4.meta.json b/ScratchV-topic06-deliverable/tests_main/loop/loop_add_chain_4.meta.json
new file mode 100644
index 0000000..154d91d
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/loop/loop_add_chain_4.meta.json
@@ -0,0 +1,10 @@
+{
+ "description": "Run a four-iteration loop whose body computes two chained adds; final returned value is the last loop-body result.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 2,
+ "b": 3,
+ "c": 4
+ },
+ "expected_return": 9
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/loop/loop_relu_add_4.dsl b/ScratchV-topic06-deliverable/tests_main/loop/loop_relu_add_4.dsl
new file mode 100644
index 0000000..44f3080
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/loop/loop_relu_add_4.dsl
@@ -0,0 +1,6 @@
+# Loop with add followed by ReLU in the body
+for i = 0, 4
+x = add(input, bias)
+y = relu(x)
+endfor
+return y
diff --git a/ScratchV-topic06-deliverable/tests_main/loop/loop_relu_add_4.meta.json b/ScratchV-topic06-deliverable/tests_main/loop/loop_relu_add_4.meta.json
new file mode 100644
index 0000000..8baca28
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/loop/loop_relu_add_4.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Run a four-iteration loop whose body computes add followed by ReLU; final returned value is the last loop-body result.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "input": -4,
+ "bias": 6
+ },
+ "expected_return": 2
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/reduction/dot_4.dsl b/ScratchV-topic06-deliverable/tests_main/reduction/dot_4.dsl
new file mode 100644
index 0000000..2118c72
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/reduction/dot_4.dsl
@@ -0,0 +1,3 @@
+# Dot product of two 4-element vectors
+result = dot(a, b, len:4)
+return result
\ No newline at end of file
diff --git a/ScratchV-topic06-deliverable/tests_main/reduction/dot_4.meta.json b/ScratchV-topic06-deliverable/tests_main/reduction/dot_4.meta.json
new file mode 100644
index 0000000..748f260
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/reduction/dot_4.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Compute the dot product of two symbolic vectors of length 4.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": [1, 2, 3, 4],
+ "b": [5, 6, 7, 8]
+ },
+ "expected_return": 70
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/reduction/dot_8.dsl b/ScratchV-topic06-deliverable/tests_main/reduction/dot_8.dsl
new file mode 100644
index 0000000..e1b71fa
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/reduction/dot_8.dsl
@@ -0,0 +1,3 @@
+# Dot product of two 8-element vectors
+result = dot(a, b, len:8)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/reduction/dot_8.meta.json b/ScratchV-topic06-deliverable/tests_main/reduction/dot_8.meta.json
new file mode 100644
index 0000000..d3172a2
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/reduction/dot_8.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Compute the dot product of two symbolic vectors of length 8.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": [1, 2, 3, 4, 5, 6, 7, 8],
+ "b": [1, 1, 1, 1, 1, 1, 1, 1]
+ },
+ "expected_return": 36
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_4.dsl b/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_4.dsl
new file mode 100644
index 0000000..279fc19
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_4.dsl
@@ -0,0 +1,4 @@
+# Dot product followed by ReLU
+x = dot(a, b, len:4)
+result = relu(x)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_4.meta.json b/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_4.meta.json
new file mode 100644
index 0000000..24910bc
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_4.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Compute a length-4 dot product and pass it through ReLU.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": [1, -2, 3, -4],
+ "b": [2, 3, 4, 5]
+ },
+ "expected_return": 0
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_8.dsl b/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_8.dsl
new file mode 100644
index 0000000..d2b906d
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_8.dsl
@@ -0,0 +1,4 @@
+# Dot product of length 8 followed by ReLU
+x = dot(a, b, len:8)
+result = relu(x)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_8.meta.json b/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_8.meta.json
new file mode 100644
index 0000000..776fab5
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/reduction/dot_relu_8.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Compute a length-8 dot product and pass it through ReLU.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": [1, 0, 1, 0, 1, 0, 1, 0],
+ "b": [2, 2, 2, 2, 2, 2, 2, 2]
+ },
+ "expected_return": 8
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/tensor/matmul_2x2.dsl b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_2x2.dsl
new file mode 100644
index 0000000..f0a80d9
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_2x2.dsl
@@ -0,0 +1,3 @@
+# Matrix multiplication: 2x2 * 2x2
+result = matmul(A, B, m:2, n:2, k:2)
+return result
\ No newline at end of file
diff --git a/ScratchV-topic06-deliverable/tests_main/tensor/matmul_2x2.meta.json b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_2x2.meta.json
new file mode 100644
index 0000000..f970a61
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_2x2.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Compute a symbolic 2x2 by 2x2 matrix multiplication.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "A": [[1, 2], [3, 4]],
+ "B": [[5, 6], [7, 8]]
+ },
+ "expected_return": [[19, 22], [43, 50]]
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/tensor/matmul_4x4.dsl b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_4x4.dsl
new file mode 100644
index 0000000..ab5a865
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_4x4.dsl
@@ -0,0 +1,3 @@
+# Matrix multiplication: 4x4 * 4x4
+result = matmul(A, B, m:4, n:4, k:4)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/tensor/matmul_4x4.meta.json b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_4x4.meta.json
new file mode 100644
index 0000000..ead52b9
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_4x4.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Compute a symbolic 4x4 by 4x4 matrix multiplication.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "A": [[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 0], [0, 0, 0, 1]],
+ "B": [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
+ },
+ "expected_return": [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]]
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/tensor/matmul_add_2x2.dsl b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_add_2x2.dsl
new file mode 100644
index 0000000..d41a2cb
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_add_2x2.dsl
@@ -0,0 +1,4 @@
+# Matrix multiplication followed by add
+x = matmul(A, B, m:2, n:2, k:2)
+result = add(x, bias)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/tensor/matmul_add_2x2.meta.json b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_add_2x2.meta.json
new file mode 100644
index 0000000..9bca125
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_add_2x2.meta.json
@@ -0,0 +1,10 @@
+{
+ "description": "Compute a 2x2 matmul and then add a symbolic bias term.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "A": [[1, 2], [3, 4]],
+ "B": [[5, 6], [7, 8]],
+ "bias": [[1, 1], [1, 1]]
+ },
+ "expected_return": [[20, 23], [44, 51]]
+}
diff --git a/ScratchV-topic06-deliverable/tests_main/tensor/matmul_relu_2x2.dsl b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_relu_2x2.dsl
new file mode 100644
index 0000000..28c7c61
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_relu_2x2.dsl
@@ -0,0 +1,4 @@
+# Matrix multiplication followed by ReLU
+x = matmul(A, B, m:2, n:2, k:2)
+result = relu(x)
+return result
diff --git a/ScratchV-topic06-deliverable/tests_main/tensor/matmul_relu_2x2.meta.json b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_relu_2x2.meta.json
new file mode 100644
index 0000000..352dfa6
--- /dev/null
+++ b/ScratchV-topic06-deliverable/tests_main/tensor/matmul_relu_2x2.meta.json
@@ -0,0 +1,9 @@
+{
+ "description": "Compute a 2x2 matmul and then apply ReLU to its result.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "A": [[-1, 2], [-3, 4]],
+ "B": [[1, 0], [0, 1]]
+ },
+ "expected_return": [[0, 2], [0, 4]]
+}
diff --git "a/ScratchV-topic06-deliverable/\350\257\276\351\242\2306\350\256\276\350\256\241\346\226\207\346\241\243.md" "b/ScratchV-topic06-deliverable/\350\257\276\351\242\2306\350\256\276\350\256\241\346\226\207\346\241\243.md"
new file mode 100644
index 0000000..334990c
--- /dev/null
+++ "b/ScratchV-topic06-deliverable/\350\257\276\351\242\2306\350\256\276\350\256\241\346\226\207\346\241\243.md"
@@ -0,0 +1,472 @@
+# ScratchV 课题 06 性能测试套件设计文档
+
+> 文档版本:v1.1
+> 编写日期:2026-07-22
+> 更新日期:2026-08-06
+> 交付目录:`ScratchV-topic06-deliverable`
+> 核心文件:`run_tests.py`、`tests_main/`、`reports/`、`README.md`
+> 功能范围:DSL 用例管理、自动编译、TinyFive 模拟验证、期望输出检查、指令数统计、耗时统计、Benchmark、性能基线对比、Markdown/HTML 报告生成、CI 示例
+
+---
+
+## 一、功能介绍
+
+### 1.1 功能概述
+
+本测试套件用于验证 ScratchV 编译器在多个 DSL 程序上的编译结果和性能表现。测试脚本会自动遍历 `tests_main/` 下的 DSL 用例,调用 ScratchV 编译器生成 RISC-V 汇编,再通过 TinyFive 适配层进行模拟验证,并生成可读的测试报告。
+
+当前套件包含 23 个 DSL 测试用例,覆盖以下类别:
+
+| 类别 | 说明 |
+|---|---|
+| `activation` | ReLU 等激活函数组合 |
+| `elementwise` | 标量/向量加法、链式加法、复用输入 |
+| `loop` | `for/endfor` 循环类用例 |
+| `branch` | `if/else/endif` 分支类用例 |
+| `reduction` | `dot` 等归约类用例 |
+| `tensor` | `matmul` 等张量计算类用例 |
+
+测试输出不仅包含 PASS/FAIL,还包含指令数、编译耗时、模拟耗时、总耗时、Benchmark 平均值、95% 置信区间、性能基线对比结果等信息。
+
+### 1.2 设计目标
+
+- **自动化**:一条命令完成所有用例的编译、模拟、结果比对和报告生成。
+- **可扩展**:新增测试只需要添加 `.dsl` 和 `.meta.json` 文件,不需要修改主测试逻辑。
+- **可解释**:报告中展示每个用例的状态、预期输出、实际输出、模拟后端、指令数和耗时。
+- **可对比**:支持保存 Benchmark 基线,并在后续运行中检查性能变化率。
+- **避免卡死**:TinyFive 模拟阶段设置 5 秒超时,分支用例即使卡住也不会阻塞整套测试。
+
+---
+
+## 二、目录结构
+
+交付目录当前结构如下:
+
+```text
+ScratchV-topic06-deliverable/
+ README.md
+ 课题6设计文档.md
+ LICENSE
+ requirements-topic06.txt
+ run_tests.py
+ setup.py
+ tests_main/
+ activation/
+ branch/
+ elementwise/
+ loop/
+ reduction/
+ tensor/
+ reports/
+ report.md
+ report.html
+ course_report_instructions.png
+ benchmark_baseline.json
+ .github/
+ workflows/
+ benchmark.yml
+```
+
+其中:
+
+- `run_tests.py` 是测试主程序。
+- `tests_main/` 保存 DSL 测试用例。
+- `reports/` 保存测试运行后生成的报告和基线文件。
+- `requirements-topic06.txt` 保存本套件额外依赖。
+- `.github/workflows/benchmark.yml` 是 GitHub Actions 示例配置。
+
+`build/` 目录如果存在,是运行测试时生成的汇编输出目录,不是必须提交的源码内容。
+
+---
+
+## 三、测试用例格式
+
+每个测试用例由两个文件组成:
+
+```text
+tests_main/{category}/{name}.dsl
+tests_main/{category}/{name}.meta.json
+```
+
+示例 DSL:
+
+```text
+result = add(a, b)
+return result
+```
+
+示例元数据:
+
+```json
+{
+ "description": "Simple scalar add.",
+ "expected_output_type": "return_value",
+ "inputs": {
+ "a": 2,
+ "b": 3
+ },
+ "expected_return": 5
+}
+```
+
+这里没有采用单独的 `.expected` 和 `.desc` 文件,而是使用 `.meta.json` 统一保存输入、描述和期望输出。这样结构更集中,也方便 Python 脚本直接解析。
+
+---
+
+## 四、核心实现设计
+
+### 4.1 编译阶段
+
+函数:`run_compile(dsl_file: Path)`
+
+作用:
+
+1. 接收一个 `.dsl` 文件路径。
+2. 调用 ScratchV 编译器:
+
+```powershell
+python -m scratchv.main -o --optimize all --dump-ir
+```
+
+3. 将生成的 RISC-V 汇编写入 `build/{case_name}.s`。
+4. 返回编译进程结果和汇编输出路径。
+
+这一阶段使用 `subprocess.run()` 调用编译器,能够捕获 `stdout`、`stderr` 和返回码。
+
+当前版本的编译阶段还没有设置 `timeout` 参数。如果编译器本身在某个 DSL 上卡死,测试进程仍可能被阻塞。mentor review 中建议为编译阶段增加 30 秒超时,这是后续需要补齐的架构风险点。模拟阶段已经有 5 秒超时,但它只能保护 TinyFive 模拟,不能保护编译器调用。
+
+### 4.2 模拟验证阶段
+
+函数:`run_simulation(asm_file: Path, timeout: float = 5.0)`
+
+作用:
+
+1. 读取编译生成的 `.s` 汇编文件。
+2. 在独立 Python 子进程中调用:
+
+```python
+from scratchv.simulator.tinyfive import verify_assembly
+```
+
+3. 获取 TinyFive 适配层返回的模拟结果。
+4. 如果模拟超过 5 秒,则终止该子进程,并返回:
+
+```python
+{
+ "success": False,
+ "backend": "timeout",
+ "error": "simulation timeout after 5s"
+}
+```
+
+这个设计解决了 branch 分支用例在真实 TinyFive 路径下可能长时间不返回的问题。超时后该用例会被标记为 FAIL,但后续用例仍会继续执行,整套测试不会卡死。
+
+### 4.3 期望输出辅助检查
+
+函数:`execute_dsl_reference(dsl_file: Path, inputs)`
+
+当前 `run_tests.py` 中仍保留了一个简化 DSL 执行器,用来根据 `.dsl` 文件和 `.meta.json` 输入计算一个辅助输出,再和 `expected_return` 比较。它的作用是检查测试用例自身的输入和期望值是否写对。
+
+需要注意:这个辅助执行器不是最终后端。最终后端仍然是 TinyFive 适配层,整体 PASS 必须满足 TinyFive 模拟成功。如果 TinyFive 超时或返回失败,即使辅助输出与期望值匹配,该用例仍然判定为 FAIL。
+
+例如:
+
+```text
+result = add(a, b)
+return result
+```
+
+输入:
+
+```json
+{
+ "a": 2,
+ "b": 3
+}
+```
+
+辅助执行器会直接算出:
+
+```text
+5
+```
+
+报告中的“实际输出”和“输出是否匹配”来自这个辅助执行器,不是 TinyFive 的寄存器返回值。这样设计可以帮助发现 `.meta.json` 写错的问题,但也带来 mentor review 中提到的语义脱节风险:同一 DSL 语义同时存在编译器路径和 Python 辅助执行路径。后续更理想的方案是让 TinyFive 直通结果成为输出比较来源,辅助执行器只作为调试工具或被移除。
+
+### 4.4 PASS/FAIL 判断
+
+一个用例通过需要同时满足:
+
+```python
+ok = (
+ result.returncode == 0
+ and output_file.exists()
+ and sim_result["success"]
+ and matched
+ and not regression["regressed"]
+)
+```
+
+也就是说,必须同时满足:
+
+- 编译成功。
+- 汇编文件生成成功。
+- TinyFive 模拟成功。
+- 辅助输出检查与期望输出匹配。
+- Benchmark 模式下没有超过性能退化阈值。
+
+如果出现:
+
+```text
+状态: FAIL
+模拟后端: timeout
+输出是否匹配: True
+```
+
+含义是:`.meta.json` 期望值和辅助输出是一致的,但生成的汇编在 TinyFive 模拟阶段没有正常结束,因此整体仍然失败。
+
+### 4.5 指令数统计
+
+模拟结果中包含 `instr_count` 字段。测试脚本会把该字段写入报告。
+
+在普通模式下,报告显示单次指令数:
+
+```text
+指令数
+```
+
+在 Benchmark 模式下,报告显示:
+
+```text
+平均指令数
+最小指令数
+最大指令数
+95% 置信区间
+```
+
+相关函数:
+
+- `summarize_benchmark_runs(instr_counts)`
+- `detect_regression(avg_instr_count, baseline_instr_count)`
+
+报告字段采用同一套结果字典保存,普通模式和 Benchmark 模式共用字段名。普通模式下 `benchmark_runs` 为 1,平均/最小/最大指令数等字段使用单次模拟结果或默认值;Benchmark 模式下这些字段填入重复运行后的统计值。这样可以减少报告解析脚本在两种模式之间切换时出现 schema 不一致的问题。
+
+### 4.6 耗时统计
+
+测试脚本使用 `time.perf_counter()` 记录每个用例的耗时。
+
+当前记录的字段包括:
+
+| 字段 | 说明 |
+|---|---|
+| `compile_time_sec` | 编译 DSL 到 RISC-V 汇编的耗时 |
+| `simulation_time_sec` | TinyFive 模拟验证耗时 |
+| `reference_time_sec` | 辅助输出检查耗时 |
+| `total_time_sec` | 单个用例完整处理耗时 |
+
+这些字段会输出到 Markdown 和 HTML 报告中,便于观察哪些用例编译慢、模拟慢或因为 timeout 导致耗时较长。
+
+### 4.7 Benchmark 与性能基线
+
+运行:
+
+```powershell
+python run_tests.py --benchmark 3
+```
+
+表示每个用例重复模拟 3 次,统计平均指令数和 95% 置信区间。
+
+运行:
+
+```powershell
+python run_tests.py --benchmark 3 --update-baseline
+```
+
+表示将当前 Benchmark 结果保存为性能基线:
+
+```text
+reports/benchmark_baseline.json
+```
+
+后续再次运行:
+
+```powershell
+python run_tests.py --benchmark 3
+```
+
+脚本会读取基线文件,并计算变化率:
+
+```text
+变化率 = (当前平均指令数 - 基线指令数) / 基线指令数 * 100%
+```
+
+如果变化率超过 5%,则认为发生性能退化。当前阈值在代码中由 `REGRESSION_THRESHOLD_PCT = 5.0` 固定配置,暂时还没有命令行参数。mentor review 建议后续增加 `--regression-threshold`,让不同环境可以自行调整退化判断标准。
+
+Benchmark 模式和 timeout 用例存在交互问题:当前 `--benchmark 3` 会对每个编译成功的用例重复模拟 3 次。如果某个 branch 用例每次都会 5 秒超时,那么一个用例会等待约 15 秒。后续建议在首次 timeout 后跳过该用例的剩余 Benchmark 重复次数,避免更新基线时长时间等待。
+
+---
+
+## 五、报告生成设计
+
+### 5.1 Markdown 报告
+
+输出文件:
+
+```text
+reports/report.md
+```
+
+内容包括:
+
+- 测试概览。
+- 用例总数、通过数量、失败数量、通过率。
+- 每个用例的状态、模拟后端、指令数、耗时、预期输出、实际输出。
+- Benchmark 模式下的平均指令数、置信区间、基线、变化率和是否退化。
+- Mermaid 指令数图表。
+- 每个用例的详细说明。
+
+### 5.2 HTML 报告
+
+输出文件:
+
+```text
+reports/report.html
+```
+
+HTML 报告主要用于浏览器查看,报告中会使用不同颜色标识 PASS 和 FAIL。当前实现已经改为在脚本内生成简单 HTML,不强制依赖复杂模板文件。
+
+### 5.3 图表
+
+输出文件:
+
+```text
+reports/course_report_instructions.png
+```
+
+图表由 `matplotlib` 生成,展示不同测试用例的指令数对比。mentor review 中指出图表依赖会增加 CI 安装成本,因此后续可以把 Markdown 文本报告作为默认产物,把 HTML 和图表作为完整报告模式输出。
+
+---
+
+## 六、运行方式
+
+### 6.1 安装依赖
+
+```powershell
+pip install -r requirements-topic06.txt
+```
+
+### 6.2 普通测试
+
+```powershell
+python run_tests.py
+```
+
+适合快速查看所有用例的 PASS/FAIL 和基本报告。
+
+### 6.3 Benchmark 测试
+
+```powershell
+python run_tests.py --benchmark 3
+```
+
+适合统计平均指令数和置信区间。
+
+### 6.4 更新性能基线
+
+```powershell
+python run_tests.py --benchmark 3 --update-baseline
+```
+
+适合在修改编译器前或稳定版本上保存基线,供后续性能回归对比使用。
+
+---
+
+## 七、Mentor Review 对应状态
+
+mentor review 中提出的问题分为架构、API、实现和测试覆盖四类。当前文档按真实代码状态整理如下:
+
+| 编号 | 问题 | 当前处理状态 | 说明 |
+|---|---|---|---|
+| S1 | 参考解释器与编译器语义脱节 | 部分处理,仍需收敛 | 当前真实模拟由 TinyFive 执行,但 `execute_dsl_reference()` 仍用于辅助检查 `.meta.json` 期望值。后续应让 TinyFive 返回值成为输出比较来源,减少双语义实现。 |
+| S2 | 编译阶段缺 timeout | 待处理 | 模拟阶段已有 5 秒超时,编译阶段暂未设置 timeout,建议后续为 `run_compile()` 增加约 30 秒限制。 |
+| S3 | Benchmark/普通模式报告格式冲突 | 已在文档中明确 | 两种模式共用结果字段,普通模式填单次结果或默认值,Benchmark 模式填统计字段。 |
+| A1 | `.meta.json` 期望值表达能力有限 | 部分支持,待规范 | 当前 `expected_return` 可保存标量,也可保存数组类 JSON 值,但文档还需要进一步明确向量/张量输出的标准写法。 |
+| A2 | `verify_assembly` 调用风险 | 已补充说明 | 即使 TinyFive 存在内部指令上限,外层仍需要 5 秒子进程 timeout 兜底。 |
+| I1 | 报告依赖过重 | 待优化 | 当前保留 Markdown、HTML、PNG 图表。后续可把 Markdown 设为默认轻量输出,把 HTML/图表改为可选完整报告。 |
+| I2 | 5% 退化阈值过于刚性 | 待优化 | 当前阈值固定为 `REGRESSION_THRESHOLD_PCT = 5.0`,后续建议增加 `--regression-threshold` 参数。 |
+| I3 | Benchmark 重复运行 timeout 用例 | 待优化 | 当前 timeout 用例会重复等待,后续建议首次 timeout 后跳过剩余重复次数。 |
+| T1 | 编译器 crash 无上下文保存 | 待处理 | 当前只捕获进程输出,后续可把 crash 的 stderr/stdout 保存到独立日志文件。 |
+| T2 | 不支持增量/子集测试 | 待处理 | 当前递归运行所有 DSL 用例,后续建议增加 `--filter` 或 `--category`。 |
+
+---
+
+## 八、当前测试现状
+
+最近一次运行结果:
+
+```text
+Total: 23
+Passed: 20
+Failed: 3
+```
+
+失败的 3 个用例均位于 `branch/` 目录:
+
+```text
+branch/if_else.dsl
+branch/if_relu.dsl
+branch/if_then.dsl
+```
+
+失败原因是 TinyFive 模拟阶段 5 秒超时:
+
+```text
+simulation timeout after 5s
+```
+
+这 3 个用例的辅助输出检查与期望输出是匹配的,说明测试用例输入和期望值本身没有暴露问题;失败点主要在当前 ScratchV 分支汇编与 TinyFive 适配层之间没有完全对齐,导致真实模拟阶段无法正常结束。
+
+其余 20 个用例可以完成编译、模拟和结果对比。
+
+---
+
+## 九、CI 集成
+
+目录中包含 GitHub Actions 示例:
+
+```text
+.github/workflows/benchmark.yml
+```
+
+CI 的目标是在 push 或 pull request 时自动运行测试套件,并上传测试报告,方便查看每次修改后的正确性和性能变化。
+
+---
+
+## 十、已完成工作对照
+
+| 阶段 | 完成情况 |
+|---|---|
+| W1 学习编译命令和 TinyFive 用法 | 已完成 |
+| W2 使用 subprocess 自动调用编译器和模拟器 | 已完成 |
+| W3 设计测试用例格式 | 已完成,采用 `.dsl + .meta.json` |
+| W4 遍历用例并输出 PASS/FAIL | 已完成 |
+| W5 提取指令数并加入报告 | 已完成 |
+| W6 matplotlib 图表和 HTML 报告 | 已完成 |
+| W7 扩充到 15 个以上用例并覆盖分支循环 | 已完成,共 23 个 |
+| W8 使用 `time.perf_counter()` 统计耗时 | 已完成 |
+| W9 回归测试模式和 5% 退化阈值 | 已完成 |
+| W10 `--benchmark` 重复运行和置信区间 | 已完成 |
+| W11 GitHub Actions CI 示例 | 已完成 |
+| W12 完整文档 | 已完成 |
+
+---
+
+## 十一、后续改进建议
+
+1. 修复 branch 分支汇编与 TinyFive 适配层的跳转/label 对齐问题,使 23 个用例都能在真实 TinyFive 路径下通过。
+2. 为 `run_compile()` 增加编译阶段 timeout,避免编译器异常时卡住整套测试。
+3. 在 Benchmark 模式下对已知 timeout 用例减少重复模拟次数,避免更新基线时等待时间过长。
+4. 增加 `--filter` 或 `--category` 参数,支持只运行部分用例。
+5. 增加 `--regression-threshold` 参数,让性能退化阈值可配置。
+6. 将 `.meta.json` 格式进一步文档化,例如明确支持的 `expected_output_type`、输入类型和算子范围。
+7. 在报告中增加“失败原因汇总”模块,集中展示 timeout、编译失败和输出不匹配的用例。