excludes: say when a prune entry no longer matches anything

[openipc-ai]

One of two PRs adding drift detection between this repo and OpenIPC/builder, after #2308 (comment) turned out to have three separate instances behind it. This is the cheap half and it lives here because it benefits every consumer of rootfs_script.sh, not just builder. The other half is OpenIPC/builder#129.

The problem

scripts/excludes/<model>_<variant>.list names files by hand, so it goes stale in one direction with nothing to show for it: a package renames or drops a sensor blob, the entry that used to prune it now prunes nothing, and the board keeps paying for whatever replaced it. The step was one line:

xargs -a ${LIST} -I % rm -f ${TARGET_DIR}%

which cannot tell "removed a file" from "removed nothing" — and fed the # separator lines these lists use to rm as literal paths besides.

The case that prompted it

OpenIPC/builder's hi3518ev200_lite.list names 25 sensor .so files where hisilicon-osdrv-hi3516cv200 now ships 17. Eight entries have been pruning nothing for however long. Meanwhile six sensor libraries it does not name — 188KB uncompressed — ship on a camera with one sensor:

libsns_gc2033.so  36K   libsns_imx323_i2c_dc_v2.so  44K   libsns_imx327.so  20K
libsns_imx307.so  20K   libsns_imx323_i2c_dc_v3.so  44K   libsns_sc2232.so  24K

That device is 4KB over its rootfs cap. Nothing in any build log said so.

What changes

Each entry is checked before removal, misses are named, and a count is printed at the end:

excludes: /usr/lib/sensors/libsns_gone.so matched no file
excludes: 1 of 4 entries in testsoc_testvariant.list matched no file

Report, never fail. An image carrying a few kB it meant to drop is something to look at, not a reason to break a build. Boards whose lists are accurate print nothing new, so this is silent for everyone it does not concern.

The loop also fixes two things the xargs form got away with by accident rather than intent: comment and blank lines are skipped explicitly, and a final entry with no trailing newline is still read.

Deliberately not here: reporting files that ship but are not named. That needs the built image and a notion of which directories a list is responsible for, and it would fire on every board at once. This half is free.

Test

test_excludes_report.sh drives the real rootfs_script.sh against a synthetic TARGET_DIR — no build:

$ bash .github/scripts/test_excludes_report.sh
ok   exit status is 0
ok   present entry was removed
ok   entry on a line without a trailing newline was removed
ok   dangling symlink was removed
ok   absent entry is reported
ok   summary counts 1 stale of 4 real entries
ok   comment lines are not treated as paths
ok   excludes step still routes the list through a reporting loop

Against the pre-change script it fails 4 of 8 and passes the other 4:

$ SCRIPT_UNDER_TEST=/tmp/old_rootfs.sh bash .github/scripts/test_excludes_report.sh
ok   exit status is 0
ok   present entry was removed
ok   entry on a line without a trailing newline was removed
ok   dangling symlink was removed
FAIL absent entry was not reported
FAIL summary line wrong or missing: <none>
FAIL a comment line was treated as a path
FAIL excludes step no longer reports stale entries -- reverted to a bare xargs?

The four that pass are the part worth stating: removal behaviour is unchanged, including the dangling symlink (-e is false for one, so there is an explicit -L arm) and the unterminated last line. Only the reporting is new.

Part 2 of the test is a drift check anchored on the loop rather than the message text, so wording can change freely but a revert to a bare xargs fails immediately.

Test plan

• bash .github/scripts/test_excludes_report.sh — 8/8
• same test against the pre-change script — 4 fail, 4 pass, as above
• python3 .github/scripts/lint-workflow-shell.py — 49 run blocks parse clean
• python3 .github/scripts/ci-matrix.py --self-test — ok (99 boards, 132 packages, 52 cases)
• bash .github/scripts/test_sysupgrade.sh — all checks passed
• bash -n general/scripts/rootfs_script.sh
• Two clean builds of ssc333_lite_meari-speed-6s proving the pruning is unchanged — see below

Proof that pruning is unchanged

rootfs_script.sh runs on the build host and can only ever rm -f listed paths, so the claim to test is whether the set of removed files moved. Two clean builds of ssc333_lite_meari-speed-6s — a device that actually has an excludes list — from identical trees differing only in general/scripts/rootfs_script.sh:

Target file lists are identical: 665 paths, no diff. That is the only thing rm -f can influence.

Every remaining byte difference between the two trees is accounted for and none of it is this change:

difference	files	cause
TIME_STAMP=1787691263 vs …65	usr/lib/os-release	stamped by date +TIME_STAMP=%s, the same line in both versions
NT_GNU_BUILD_ID (20 bytes)	9 .ko	non-reproducible build
embedded build clock	busybox (1 byte, 20:50:15→20:50:17), 8188fu.ko (20:53:53→20:53:57)	__TIME__

9 of the 11 differing binaries are byte-identical once the build-id note is masked; the other two differ only in an embedded clock string. The script never rewrites a file, so none of this is reachable from it.

No board in this repository has an excludes list. general/scripts/excludes/ does not exist here, so [ -f "${LIST}" ] is false for all 99 boards and the changed branch never executes for any of them. The consumers are OpenIPC/builder's retail devices — which is why the verification above uses one.

And it earns its keep immediately on that device:

excludes: /etc/sensors/sc2338.bin matched no file
excludes: /lib/modules/4.9.84/sigmastar/sensor_jxq03_mipi.ko matched no file
excludes: /lib/modules/4.9.84/sigmastar/sensor_sc2338_mipi.ko matched no file
excludes: 3 of 36 entries in ssc333_lite.list matched no file

Three stale entries in a shipping device's list that nothing could have surfaced before.

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Report stale rootfs exclusion entries

✨ Enhancement 🧪 Tests ⚙️ Configuration changes 🕐 20-40 Minutes

AI Description

• Reports stale exclusion entries without failing root filesystem builds.
• Skips comments and blanks while preserving unterminated entries and dangling symlink removal.
• Adds synthetic regression coverage and a dedicated shell-tests CI job.

Diagram

graph TD
  E["Exclude List"] --> L["Reporting Loop"] --> M{"Target Exists?"}
  M -->|yes| R["Remove File"]
  M -->|no| W["Report Miss"] --> S["Stale Summary"]

Loading

High-Level Assessment

The per-entry loop is the appropriate approach because reporting requires distinguishing successful removals from misses while retaining comment filtering, dangling-symlink handling, and unterminated-line support. Retaining bulk xargs removal with a separate preflight would duplicate list parsing and risk inconsistent behavior.

Files changed (3) +148 / -2

Enhancement (1) +28 / -2

rootfs_script.shDetect and report exclusion entries matching no files +28/-2

Detect and report exclusion entries matching no files

• Replaces bulk xargs deletion with a line-aware loop that skips comments and blanks, removes existing files and dangling symlinks, and handles a final unterminated entry. Missing paths are reported individually and summarized while the build remains successful.

general/scripts/rootfs_script.sh

Tests (1) +104 / -0

test_excludes_report.shAdd regression coverage for exclusion drift reporting +104/-0

Add regression coverage for exclusion drift reporting

• Runs the real rootfs post-build script against a synthetic target and exclusion list. Verifies non-failing stale-entry diagnostics, summary counts, comment filtering, unterminated final lines, dangling symlink removal, and protection against reverting to bare xargs.

.github/scripts/test_excludes_report.sh

Other (1) +16 / -0

shell-tests.ymlRun exclusion reporting regression tests in CI +16/-0

Run exclusion reporting regression tests in CI

• Adds a guarded Ubuntu job that checks out the repository and runs the synthetic exclusion-report test without requiring a board build.

.github/workflows/shell-tests.yml

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (1) 📜 Skill insights (0)

1. Hardware verification is absent 📘 Rule violation ☼ Reliability

Description

The PR changes rootfs pruning behavior but provides only a synthetic-target test, while the test
plan leaves hardware verification unchecked and explicitly states that no hardware run occurred.
This fails the required real-camera before/after evidence and checked verification-box criteria.

Code

.github/scripts/test_excludes_report.sh[R40-43]

+ext="$work/general"
+mkdir -p "$ext/scripts/excludes"
+target="$work/target"
+mkdir -p "$target/usr/lib/sensors" "$target/etc/sensors" "$target/usr/lib"

Evidence

PR Compliance ID 1 requires a behavior-changing PR to show before/after output from a real camera
and requires every verification checkbox to be checked. The added test constructs a synthetic target
at .github/scripts/test_excludes_report.sh[40-43], and the PR description confirms this is the
only behavioral exercise by leaving No hardware run unchecked and stating the script cannot
execute on a camera.

Rule 1: Hardware evidence is present and honest
.github/scripts/test_excludes_report.sh[40-43]

---

Tip of the day

💡 Did you know, you can hide the parts of a finding you never read, like the evidence or the agent prompt

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗