feat: replace GPT-5.4-mini experiments with GPT-5.6 Luna - #269
Merged
Rodriguespn merged 1 commit intoSep 9, 2026
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
mattrossman
approved these changes
Sep 7, 2026
mattrossman
reviewed
Sep 7, 2026
Rodriguespn
added a commit
that referenced
this pull request
Sep 8, 2026
Clarifies naming alongside the new codex-gpt-5.6-luna experiments, per review feedback on #269. Pure rename: model, suite, skills, and runtime config are unchanged.
Replaces the four GPT-5.4-mini experiments with GPT-5.6 Luna, the new Assistant default model (supabase/supabase#49749), using reasoningEffort medium to match. Renames the existing codex-gpt-5.6 experiments to codex-gpt-5.6-sol for naming clarity alongside the new luna experiments (per PR review). Removes 8 experiments never wired into any experiment suite (benchmark/no-skills/regression): they carry no historical results and no live references. Refreshed eval-results.json / regression-eval-results.json to match via the run-evals CI label.
Rodriguespn
force-pushed
the
Rodriguespn/ai-1190-replace-54-mini-with-56-luna
branch
from
September 9, 2026 17:14
a35be8c to
6cd27a7
Compare
mattrossman
added a commit
that referenced
this pull request
Sep 9, 2026
Moves the nightly regression suite from Claude Code / Sonnet 5 to Codex / GPT-5.6 Luna at medium effort, reusing the existing experiments from #269 . See [thread](https://supabase.slack.com/archives/C051L8U2EJF/p1788983215168069) for context on the decision. This PR doesn't update results, that will come from the next scheduled run. Closes AI-1200 --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces the four GPT-5.4-mini experiments with GPT-5.6 Luna, the new Assistant default model, so the model change in supabase/supabase#49749 is covered by the public eval suite.
Why
The Assistant default model changed to
gpt-5.6-luna(supabase/supabase#49749); the public evals suite should track that model.Closes AI-1190