Skip to content

[AI Test Toolkit] Export full eval run results (cost, time, dataset)#8442

Draft
onbuyuka wants to merge 1 commit into
mainfrom
bugs/633442-ait-export-full-results
Draft

[AI Test Toolkit] Export full eval run results (cost, time, dataset)#8442
onbuyuka wants to merge 1 commit into
mainfrom
bugs/633442-ait-export-full-results

Conversation

@onbuyuka
Copy link
Copy Markdown
Contributor

@onbuyuka onbuyuka commented Jun 3, 2026

Summary

The Export Results / Download Test Summary action on the AI Eval log pages (report 149030 "AIT Test Summary") did not export the full run results. It only emitted: codeunit/test name, status, accuracy, turns, input, output, and error. The reporter needs the cost and timing data to analyse a run.

Root cause

The export passes the complete filtered AIT Log Entry view to the report (no row/marked-row limiting) — so the limitation is purely the report's column set. The dataset omitted fields that already exist on the table and are shown on screen:

  • Cost/time: Tokens Consumed, Duration (ms), Start Time, End Time
  • Identity: Version, Tag, Test Input Group Code, Test Input Code, Test Input Description, Run ID, Entry No.

Fix

Add those columns to the dataitem(Results; "AIT Log Entry") dataset. A modern Excel-layout report writes every dataset column into the workbook's Data sheet, so the new fields are exported in the raw data without changing the formatted .xlsx layout.

Follow-up (optional, not in this PR)

If the formatted/pretty sheet of AITestSummary.xlsx should also display the new columns, the layout file can be regenerated from the updated report (Report → Run → Edit layout). The raw Data sheet already carries them.

Impact / risk

Additive only — no codeunit/page changes; the export already passed the full filtered view. Field names verified against AITLogEntry.Table.al.

Verification

  • Verified each added field exists on AIT Log Entry (fields 1, 4, 5, 9, 13, 15, 18, 24, 25, 26, 50).
  • ⚠️ Not runtime-verified on an NST (AL does not run locally here); relying on PR CI for compile. A manual export check is recommended to confirm the Data sheet content.

Linked work item: AB#633442

🤖 Generated with Claude Code

@github-actions github-actions Bot added this to the Version 29.0 milestone Jun 3, 2026
…dataset)

Report 149030 "AIT Test Summary" (the "Export Results" / "Download Test
Summary" action on the AI Eval log pages) only emitted a subset of the
AIT Log Entry data: codeunit/test name, status, accuracy, turns, input,
output and error. It omitted the cost and timing data the user needs to
analyse a run - tokens consumed, duration, start/end time - as well as the
version, tag and dataset identity (test input group/code/description) that
are shown on screen.

Add those columns to the report dataset so the exported workbook contains
the full per-entry results. Because a modern Excel-layout report writes
every dataset column into the workbook's Data sheet, the new fields are
exported without needing to change the formatted layout.

Fixes AB#633442

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@onbuyuka onbuyuka force-pushed the bugs/633442-ait-export-full-results branch from 0e67942 to 06027af Compare June 3, 2026 19:07
@onbuyuka
Copy link
Copy Markdown
Contributor Author

onbuyuka commented Jun 4, 2026

CI note: the Clean build failure is a known BCApps incremental-build infra issue — not this change

The ... (Clean) build failure is not caused by this PR. The incremental Clean build compiles the modified AI Test Toolkit app against the System Application bundled in the BC platform artifact (sandbox/29.0.51094.0) instead of the freshly-built System Application. That artifact predates #8381, which added Agent Task.OpenAgentTaskLogEntries / GetStepsDoneCount (called by AITAgentLogEntryPart.Page.al), so the compiler reports AL0132 ... does not contain a definition for ....

Evidence it is infrastructure, not this change:

No code change is appropriate (a #if not CLEAN29 guard on non-obsolete methods would be wrong). This goes green once the platform-artifact pin advances past #8381. Holding for that.

@onbuyuka onbuyuka closed this Jun 4, 2026
@onbuyuka onbuyuka reopened this Jun 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant