Skip to content

fix(ci): use gpt-6.1-terra for AI translations and fail on translation errors - #2356

Merged
amilcarlucas merged 1 commit into
masterfrom
fix_i18n_LLM
Nov 9, 2026
Merged

amilcarlucas merged 1 commit into
masterfrom
fix_i18n_LLM

Conversation

@amilcarlucas

@amilcarlucas amilcarlucas commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Description

GitHub retired the gpt-5.4 Copilot model on 2026-10-19. Since then every "Run AI translation" job failed with:

Copilot CLI stderr: Error: Model "gpt-5.4" from --model flag is not available.

The failure was invisible: the step uses continue-on-error, the check step exits 0, and only a ::warning:: annotation was emitted. Workflow runs stayed green while the generated PR (merge-i18n-po-strings) contained no .po translations, only recompiled .mo files.

Changes:

  • Switch the actions/ai-inference model from gpt-5.4 to gpt-6.1-terra.
  • Make "Report translation status" exit 1 and emit ::error:: when a chunk was not translated, so the ai_translate job (and the workflow run) is marked as failed. The placeholder artifact is still uploaded beforehand, and the other matrix chunks keep running (fail-fast is false).
  • Run "Report translation status" unless the job is cancelled, instead of always().
  • Let finalize_translations run even if some ai_translate jobs failed (!cancelled() and extract_strings succeeded), so the .pot/.po string merge and the successful translations still reach the PR. Without this, failing translate jobs would skip the finalize job and no PR would be created.

Checklist

  • Run pre-commit checks locally
  • Verified by a human programmer
  • All commits are signed off (use git commit --signoff)
  • Code follows our coding standards
  • Documentation updated if needed
  • No breaking changes or properly documented

Testing

Describe how you tested these changes:

  • Unit tests pass
  • Integration tests pass
  • Manual testing performed
  • Tested on flight controller hardware

Copilot AI balanced review requested due to automatic review settings November 9, 2026 09:45
@amilcarlucas amilcarlucas added the AIReview Request an automated AI review; picked up by the reviewprs sweep label Nov 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Translation failure paths can bypass failure markers, allowing English text to be inserted as translations or empty uploads to abort partial-results PR creation.

1 open issue
What changed in this PR

Updates the automated GUI translation workflow to use a replacement AI model and expose translation failures while allowing finalization to proceed with partial results.

Changes:

  • Switches the inference model from gpt-5.4 to gpt-6.1-sol.
  • Reports unsuccessful translation chunks as job failures unless cancelled.
  • Allows finalization after translation-job failures when string extraction succeeded and .po files changed.
File Description
.github/​workflows/​ai-translation.yml Updates the AI model, failure reporting, and conditions for partial-results finalization.

Review effort: Balanced

if: needs.extract_strings.outputs.po-files-changed == 'true'
# Run even if some ai_translate jobs failed, so the .pot/.po merge and the successful translations still reach the PR.
# The failed ai_translate jobs already turn the workflow run red.
if: ${{ !cancelled() && needs.extract_strings.result == 'success' && needs.extract_strings.outputs.po-files-changed == 'true' }}
@AP-Review

AP-Review commented Nov 9, 2026 •

Copy link
Copy Markdown

Deprecated — see below for the updated review.

Previous review (2026-11-09)

Automated review note — AI-generated (Claude+Codex), validated against the live diff. Please sanity-check before acting.
Verdict: REQUEST CHANGES

Reviewed at head 07d82404f3.
Full report: https://firmware.ardupilot.org/Tools/APReview/DevCallReviews/PRReviews/ardupilot/methodicconfigurator/2356/1.html#prMethodicConfigurator-2356

This needs one change before merging: letting finalize run after failed translation jobs exposes a path where untranslated English is committed as translations.

Blocking

  1. ai-translation.yml:400: with the new finalize condition, a hard failure in 'Check AI translation success' (e.g. bash syntax error from an AI response like 0:Selecione "Modo (automático)") skips 'Handle translation failure', uploads the untouched English chunk, and finalize writes the English msgids as msgstr. Fix the placeholder condition at line 361: if: ${{ !cancelled() && (failure() || steps.check_translation.outputs.translation_successful != 'true') }} (same as the Copilot suggestion).

Low

  • ai-translation.yml:400: a 'Save translation result' failure uploads an empty chunk, and finalize then exits 1 before creating the partial-results PR, defeating the goal stated in the comment at line 398. The same line-361 change fixes it.

@amilcarlucas amilcarlucas changed the title fix(ci): use gpt-6.1-sol for AI translations and fail on translation errors fix(ci): use gpt-6.1-terra for AI translations and fail on translation errors Nov 9, 2026
@AP-Review

AP-Review commented Nov 9, 2026 •

Copy link
Copy Markdown

Deprecated — see below for the updated review.

Previous review (2026-11-09)

Automated review note — AI-generated (Claude+Codex), validated against the live diff. Please sanity-check before acting.
Verdict: COMMENT

Reviewed at head 3a3a93fc43.
Full report: https://firmware.ardupilot.org/Tools/APReview/DevCallReviews/PRReviews/ardupilot/methodicconfigurator/2356/2.html#prMethodicConfigurator-2356

The earlier blocker is fixed and nothing here blocks merging; two low-severity gaps in the new partial-results path are worth a look. Since the last review the head moved from 07d8240 to 3a3a93f: 'Handle translation failure' now also runs on failure(), so hard Check or Save failures upload a failure marker instead of English or empty chunks, and the model is now gpt-6.1-terra.

Low

  • ai-translation.yml:400: finalize now runs after failed chunks, but load_translations() in .github/skills/update-gui-translations/scripts/insert_missing_translations.py:85-98 stops at the first missing _N file. After the marker for a failed _1 is deleted, valid _2, _3… are silently skipped (this predates the PR, but it contradicts the line-398 comment). Iterate over all existing numbered files instead of stopping at the first gap.
Minor (1)
  • ai-translation.yml:400: if one leg is cancelled or times out before Save, Handle (line 361) is skipped but the always() upload at line 374 still ships the untouched English chunk, and finalize could insert it as msgstr. Making the upload if: ${{ !cancelled() }} closes this. Whether a single-job timeout leaves finalize's !cancelled() true is unconfirmed, and it needs a 6-hour hang.

Withdrawn: the Report step's success message after a Save failure is the same on master and the job still fails; the missing commit-message bullet for the Handle change is just style.

…n errors

GitHub retired the gpt-5.4 Copilot model on 2026-10-19. Since then every
"Run AI translation" job failed with:

  Copilot CLI stderr: Error: Model "gpt-5.4" from --model flag is not available.

The failure was invisible: the step uses continue-on-error, the check step
exits 0, and only a ::warning:: annotation was emitted. Workflow runs stayed
green while the generated PR (merge-i18n-po-strings) contained no .po
translations, only recompiled .mo files.

Changes:
- Switch the actions/ai-inference model from gpt-5.4 to gpt-6.1-terra.
- Make "Report translation status" exit 1 and emit ::error:: when a chunk
  was not translated, so the ai_translate job (and the workflow run) is
  marked as failed. The placeholder artifact is still uploaded beforehand,
  and the other matrix chunks keep running (fail-fast is false).
- Run "Report translation status" unless the job is cancelled, instead of
  always().
- Let finalize_translations run even if some ai_translate jobs failed
  (!cancelled() and extract_strings succeeded), so the .pot/.po string
  merge and the successful translations still reach the PR. Without this,
  failing translate jobs would skip the finalize job and no PR would be
  created.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

☂️ Code Coverage

current status: ✅

Overall Coverage

Statements Covered Coverage Threshold Status
30665 29142 95% 89% 🟢

New Files

No new covered files...

Modified Files

No covered modified files...

updated for commit: 3cc540c by action🐍

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Test Results

     3 files       3 suites   40m 33s ⏱️
 9 528 tests  9 523 ✅  5 💤 0 ❌
28 452 runs  28 417 ✅ 35 💤 0 ❌

Results for commit 3cc540c.

@AP-Review

Copy link
Copy Markdown

Automated review note — AI-generated (Claude+Codex), validated against the live diff. Please sanity-check before acting.
Verdict: ACCEPT

Reviewed at head 3cc540cb44.
Full report: https://firmware.ardupilot.org/Tools/APReview/DevCallReviews/PRReviews/ardupilot/methodicconfigurator/2356/3.html#prMethodicConfigurator-2356

Both remaining points from the last review are fixed, and nothing at head 3cc540c needs changing. Since the previous review at 3a3a93f, load_translations() reads every numbered chunk in numeric order instead of stopping at the first missing one, with a test that fails on the old loader (insert_missing_translations.py:81). The upload step is now if: ${{ !cancelled() }} (ai-translation.yml:374), so a cancelled leg no longer uploads its untranslated English chunk.

Not raised: the commit body's Changes list doesn't mention the loader rewrite or the Handle/Upload condition changes, but no repository rule requires a bullet per change.

@amilcarlucas
amilcarlucas merged commit 43fc06a into master Nov 9, 2026
31 of 33 checks passed
@amilcarlucas
amilcarlucas deleted the fix_i18n_LLM branch November 9, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AIReview Request an automated AI review; picked up by the reviewprs sweep

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants