The Wire · Showcase
JUDGE UPGRADE: evals now reasons at medium effort
By RepoJournal · Filed · About Supabase
Supabase's eval judge is getting smarter, but the RLS guide fix is the one that will keep your docs green.
The evals repo's default judge moved from gpt-5.5 at low reasoning to gpt-5.6-sol at medium effort [1], a shift that directly addresses the weakest link in your scoring pipeline. Separately, the RLS guide eval softened its pgTAP coverage rubric after a bonus test-quality check kept turning the whole doc red while all 40 RLS checks passed [2]. Over in the CLI, the monorepo consolidated on TypeScript 7 and bumped the entire Effect family to RC.111, removing stale beta pins and parallel aliases [3]. The stack runtime is now Effect-native across Node and Bun, with on-demand slim resource preparation via `off`, `lazy`, and `eager` policies [4]. Studio shipped three quality-of-life wins: permission presets for scoped PATs [5], preserved assistant tool previews after completion [6], and a restored notebook diff preview for already-applied updates [7]. ETL now handles flexible BigQuery column names across initial copy, CDC, schema changes, partitioning, and clustering [8], and Realtime surfaces the Postgres Changes connection pool in settings [9]. The MCP server instructions are now intent-based, fixing broken tool references on hosted platforms [10].
One email a day. Unsubscribe in one click.
Keep up with Supabase in about 3 minutes a day: what actually shipped — the commits, pull requests, releases, and security advisories that matter.
One email a day. Unsubscribe in one click. Read a past issue →
Action items
- → Pull the latest evals to get the gpt-5.6-sol judge and the softened RLS rubric supabase/evals [immediate]
- → Review the CLI Effect RC.111 upgrade for flag and output changes before adopting supabase/cli [plan]
- → Watch BigQuery flexible name support for any CDC or partitioning edge cases supabase/etl [monitor]
- → Check the Realtime dashboard for the new Postgres Changes pool setting supabase/supabase [monitor]
References
- [1] fix(core): default the judge to gpt-5.6-sol at medium reasoning effort (#222) supabase/evals
- [2] fix(evals): soften the pgTAP coverage rubric in the RLS guide eval (#224) supabase/evals
- [3] chore(deps): consolidate TypeScript and update Effect (#6289) supabase/cli
- [4] feat(stack): prepare slim resources on demand (#6250) supabase/cli
- [5] feat(studio): add permission presets to scoped pat creation form ↗ supabase/supabase
- [6] feat(studio): preserve assistant tool previews after completion ↗ supabase/supabase
- [7] Restore notebook diff preview for completed updates ↗ supabase/supabase
- [8] feat(bigquery): Add support for flexible names ↗ supabase/etl
- [9] fix(realtime): add pg changes pool to realtime settings ↗ supabase/supabase
- [10] fix: make server instructions intent-based instead of naming tools (#372) supabase/mcp