Skip to content

builder.sh: propagate the build failure instead of always exiting 0 - #136

Merged
widgetii merged 1 commit into
OpenIPC:masterfrom
bneigher:builder-sh-propagate-failure
Aug 29, 2026
Merged

builder.sh: propagate the build failure instead of always exiting 0#136
widgetii merged 1 commit into
OpenIPC:masterfrom
bneigher:builder-sh-propagate-failure

Conversation

@bneigher

Copy link
Copy Markdown
Contributor

make BOARD=${DEVICE} is unguarded and nothing checks its status, so builder.sh
always exits 0. copy_to_archive then runs regardless and prints
Assembled firmware available in: over an empty directory.

The misleading banner is the small part. The expensive part is what it does to CI,
which @widgetii pointed out on #119master.yml calls this inside a retry loop:

for sleep_for in $backoffs ""; do
  bash builder.sh ${NAME} && break
  ...
  echo "::error::build failed after ${attempt} attempts"
  exit 1
done

A script that cannot fail satisfies && break on the first attempt. So the
six-step backoff budget, added specifically to absorb transient toolchain and CDN
flakes, has never retried anything, and the exit 1 after the loop is
unreachable. A genuinely failed build reports green rather than being
re-attempted.

Until #135 this was masked downstream: Stage artifacts is gated on env.NORFW,
so a failed build merely skipped the upload rather than failing the job. Now that
a PR keeps the image it built, a green job with no fw-* attached is a reliable
tell that the build failed silently — which makes propagating the status more
useful, not less.

How I hit it

Building gk7202v300_lite_generic-w7 locally, make died partway through
(unrelated host problem — an emulated x86_64 toolchain segfaulting gcc). The
script still printed the archive banner and returned 0, and I briefly believed I
had an image. The directory was empty.

The change

make BOARD=${DEVICE}
BUILD_RC=$?
if [ ${BUILD_RC} -ne 0 ]; then
    echo_c 31 "\nBuild FAILED (make exited ${BUILD_RC}) - not archiving"
    exit ${BUILD_RC}
fi

Deliberately not set -e at the top, per @widgetii's suggestion and for the
same reason: this script does a lot of unguarded cp/rm/cd, and enabling
errexit globally would change failure behaviour well beyond this line. The
explicit form touches exactly the status that callers depend on.

size-report keeps its || true — it is genuinely best-effort against older
pinned firmware refs.

sh -n clean. Verified the new path returns the make status rather than 0.

Branched from 222db9e rather than current master only because my token cannot
create the newer workflow files on my fork; builder.sh is byte-identical at both
commits, so the diff is exactly the twenty lines above.

Refs: #119

`make BOARD=${DEVICE}` was unguarded and nothing checked its status, so the
script always returned 0. copy_to_archive then ran regardless and printed
"Assembled firmware available in:" over an empty directory.

The expensive part is what that did to CI rather than the misleading banner.
master.yml calls this inside a retry loop:

    for sleep_for in $backoffs ""; do
      bash builder.sh ${NAME} && break
      ...
      echo "::error::build failed after ${attempt} attempts"
      exit 1
    done

A script that cannot fail satisfies `&& break` on the first attempt. So the
six-step backoff budget, added specifically to absorb transient toolchain and
CDN flakes, never retried anything, and the `exit 1` after the loop was
unreachable. A genuinely failed build reported green instead of being
re-attempted.

Until OpenIPC#135 this was masked downstream: Stage artifacts is gated on env.NORFW,
so a failed build merely skipped the upload. Now that a PR keeps the image it
built, a green job with no fw-* attached is a reliable tell that the build
failed silently -- which makes propagating the status more useful, not less.

Found while building a device profile locally, where make died and the script
still reported success. Deliberately NOT `set -e` at the top: this script does a
lot of unguarded cp/rm/cd and enabling errexit globally would change failure
behaviour well beyond this line.

Refs: OpenIPC#119
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Propagate firmware build failures from builder.sh

🐞 Bug fix 🕐 Less than 10 minutes

Grey Divider

AI Description

• Propagates firmware make failures to callers using the original nonzero exit status.
• Skips size reporting and archiving when the primary build fails.
• Restores CI retry/backoff behavior and prevents false-green builds.
Diagram

graph TD
  CI["CI Retry Loop"] --> Builder["builder.sh"] --> Make["Firmware Make"] --> Result{"Make succeeds?"}
  Result -->|Yes| Report["Size Report"] --> Archive["Firmware Archive"]
  Result -->|No, status propagated| CI
Loading
High-Level Assessment

The targeted status capture and exit is the safest approach because it preserves make's exact failure code and avoids archiving invalid output. Global set -e was appropriately rejected because it would alter unrelated failure behavior across existing unguarded shell commands.

Files changed (1) +20 / -0

Bug fix (1) +20 / -0
builder.shReturn make failures before reporting or archiving +20/-0

Return make failures before reporting or archiving

• Captures the primary firmware build's exit status and exits with that same code when make fails. The failure path emits a clear diagnostic and prevents best-effort reporting, empty archiving, misleading completion output, and false CI success.

builder.sh

@qodo-free-for-open-source-projects

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Grey Divider

Tip of the day
💡 Did you know, you can reply 'qodo' on any finding to push back, ask questions, or dig deeper

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

@widgetii
widgetii merged commit 15d0049 into OpenIPC:master Aug 29, 2026
115 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants