feat: store eval results in a Supabase project (AI-922) - #90
Closed
Rodriguespn wants to merge 6 commits into
Closed
Conversation
AI-922 Store eval results in a Supabase project (durable leaderboard store)
Store eval results in a dedicated Supabase project (org: Supabase Dev) as the durable, queryable source of truth for the public leaderboard, replacing the committed Complements [[AI-921]] (mirror results to Braintrust for analysis/traces): Braintrust reads are free but retention is limited (14d free / 30d Pro), so it isn't a durable store for a public leaderboard. A Supabase project is durable + free-to-read and lets the web app query results directly. Scope (rough):
|
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Rodriguespn
marked this pull request as ready for review
July 14, 2026 18:26
Add the schema for a durable, queryable eval-results store in a dedicated Supabase project (supabase-evals-results), intended to replace the committed eval-results.json as the leaderboard source of truth. Complements AI-921. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… path (AI-922)
Replace the committed apps/web/src/data/eval-results.json with a durable Supabase
store as the leaderboard source of truth.
- Real Supabase project in-repo (`supabase/`): config + migration for
public.eval_results with RLS ("public read"), anon/authenticated SELECT grant,
and service_role write grants (secure-by-default needs explicit grants).
- Uploader `pnpm upload:supabase`: upserts the exported snapshot by
(experiment, eval); --dry for a credential-free preview.
- Web reads from Supabase at runtime via the anon key (data/eval-results.ts +
a runtime result store), replacing the static JSON import.
- CI: publish-results uploads to Supabase (service-role secret) instead of
committing/PR-ing the JSON; stop tracking the JSON (now gitignored).
Verified locally end-to-end: migration applies, uploader writes 144 rows
(idempotent), anon REST read returns all rows; web typechecks and builds.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename VITE_SUPABASE_ANON_KEY -> VITE_SUPABASE_PUBLISHABLE_KEY; publishable keys are preferred over legacy anon keys for frontend code. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- web: validate DB rows per-row with safeParse and drop failures instead of throwing (a single drifted row no longer blanks the leaderboard); mount even if the fetch rejects, so an outage renders the empty state, not a blank page. - ci: reduce workflow permissions to contents:read (commit/PR steps are gone); fail loudly when SUPABASE_* secrets are missing (Supabase is now the only persistence path); refresh stale merge/concurrency/empty-state comments. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rodriguespn
force-pushed
the
pedrorodrigues/ai-922-supabase-results-store
branch
from
July 14, 2026 18:49
070ebeb to
15d526f
Compare
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rodriguespn
added
run-evals-changed
Contributor
Author
|
Closing this as we're not going forward with this (thread) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces
apps/web/src/data/eval-results.jsonwith a durable postgres database hosted on Supabase as the leaderboard's source of truth, as Braintrust retention is just 30d for Pro plans.The results are stored on project
supabase-evals-results(refwuoyotyfgfnxqqraadih) belonging toSupabase Devorg and hosted onus-east-1.Config needed
GitHub secrets
SUPABASE_URL+SUPABASE_SERVICE_ROLE_KEYfor CI;VITE_SUPABASE_URL+VITE_SUPABASE_PUBLISHABLE_KEYfor the web build.Closes AI-922.