feat: eval - #68

Merged

schiwekM merged 49 commits into

mainfrom

Sep 4, 2026

Commits on Jul 30, 2026

First try
SirSimon04
committed
4c27c07

Commits on Jul 31, 2026

update impl
SirSimon04
committed
8047147
add mocking
SirSimon04
committed
8bfedab
Some changes
SirSimon04
committed
5704a1a

Commits on Aug 7, 2026

.
SirSimon04
committed
bf53299

Commits on Aug 10, 2026

set header
SirSimon04
committed
25ae909

Commits on Aug 27, 2026

Basic setup
schiwekM
committed
39381af
Merge branch 'main' into tryout/llm-as-a-judge
schiwekM
committed
1df9768
Fixes
schiwekM
committed
c3bf8e2
Formatting
schiwekM
committed
52f98fe
Update .gitignore
schiwekM
committed
f158f9e
Added changelog
schiwekM
committed
3088d5f

Commits on Aug 28, 2026

Cleanup
schiwekM
committed
8ebcbec
Better metrics
schiwekM
committed
8a67a23
Docs
schiwekM
committed
66b1c3c
Fix for multi turn
schiwekM
committed
4eb770d
Fixes
schiwekM
committed
118ca09

Commits on Aug 30, 2026

Multi turn eval
schiwekM
committed
e2805f8

Commits on Aug 31, 2026

Renamed to chat
schiwekM
committed
1914b1d

Commits on Sep 1, 2026

Update package.json
schiwekM
committed
7a25d81
Small cleanup
schiwekM
committed
6690eb1
Restructure and export prompts
schiwekM
committed
34be355
Fix
schiwekM
committed
caaa620
Cleanup
schiwekM
committed
6d2f9d5
fix
schiwekM
committed
0835360
lint
schiwekM
committed
c668905

Commits on Sep 2, 2026

Adjustments based on review
schiwekM
committed
641c443
Merge branch 'main' into feat/eval
schiwekM
committed
653acd4
Update CHANGELOG.md
schiwekM
committed
85bf9b0
Cleanup mlflow.js
schiwekM
committed
8ab12ec
Further cleanup
schiwekM
committed
10b2ee0

Commits on Sep 3, 2026

Cleanup
schiwekM
committed
3b19a30
Merge branch 'main' into feat/eval
schiwekM
committed
d380fbf

Commits on Sep 4, 2026

fix
schiwekM
committed
9435dd0
Review
schiwekM
committed
f5d1241
One Judge
schiwekM
committed
f03a1f0
Delete vitest.config.evals.js
schiwekM
committed
12df419
Merge branch 'main' into feat/eval
schiwekM
committed
f558c63
fix
schiwekM
committed
87cc99c
Update bookshop.eval.test.js
schiwekM
committed
10955dc
vitest as devDep for faster ci
schiwekM
committed
f793541
Update test.yml
schiwekM
committed
a27c522
Fix
schiwekM
committed
2683dc5
Update package-lock.json
schiwekM
committed
fc17bfe
Delete telemetry-v1.test.js
schiwekM
committed
93b2936
Merge branch 'main' into feat/eval
schiwekM
committed
13e34b5
Update Judge.js
schiwekM
committed
4ca3c1a
Fix custom prompt
schiwekM
committed
01a79c2
Update eval-framework.test.js
schiwekM
committed
f03c048