package: stop selecting jsonfilter, mirroring firmware #2304 - #128
Conversation
OpenIPC/firmware dropped BR2_PACKAGE_JSONFILTER from every one of its
defconfigs in #2304, once the WebUI stopped calling it. builder's defconfigs are
its own files, so 97 of them kept selecting it. jsonfilter selects JSON_C and
LIBUBOX, and on a device that enables neither uqmi-openipc nor wifibroadcast-ng
nothing else selects libubox, so it and jsonfilter shipped for nothing.
That is what has had three devices over their rootfs cap:
hi3518ev200_lite_switcam-hs303 rootfs 5124KB/5120KB -4KB
ssc333_lite_babysense-see-hd-ip206 rootfs 5124KB/5120KB -4KB
ssc333_lite_meari-speed-6s rootfs 5128KB/5120KB -8KB
all three red on the nightly, meari since 2026-08-24. Bisecting meari across
firmware revisions lands on #2301, which added 408 bytes to the comment-stripped
sysupgrade script -- these images had no headroom left at all, and 408 bytes of
a genuine bug fix was enough to push one over a 4KB padding step. The fix is to
give them headroom, not to give back the bug fix.
Measured on ssc333_lite_meari-speed-6s, built the way builder.sh builds it --
devices/<item>/ copied over a firmware checkout at master, clean tree both times:
before rootfs.squashfs 5128KB/5120KB size exceeded by: 8KB
after rootfs.squashfs 5096KB/5120KB 24KB spare
-32KB, from /usr/bin/jsonfilter (18068 raw) and /usr/lib/libubox.so (30404 raw).
json-c does NOT go: majestic lists it in MAJESTIC_DEPENDENCIES, so buildroot
builds and installs it whether or not BR2_PACKAGE_JSON_C is set. Only the two
files above leave the image.
Two consumers genuinely need jsonfilter and both keep it:
* devices/apfpv's own etc/init.d/S99msposd reads majestic's config.json
through it, and ssc338q_apfpv and ssc378qe_apfpv do not enable
wifibroadcast-ng. Those two keep the symbol, with the reason written above
the line -- the same undeclared-runtime-dependency trap firmware #2305 had
to fix, and the reason this is 95 defconfigs and not 97.
* every other FPV device enables wifibroadcast-ng, whose Config.in selects
BR2_PACKAGE_JSONFILTER (added by #2305) for the same script. They keep it
without saying so.
LTE devices keep libubox through uqmi-openipc, which selects it independently.
Nothing else in this repository calls jsonfilter: grepping devices/ and
package/ finds only that one S99msposd line, and firmware's own tree has no
caller left outside wifibroadcast-ng.
Code Review by Qodo🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)
Great, no issues found!Qodo reviewed your code and found no material issues that require reviewTip of the day💡 Did you know, you can hide the parts of a finding you never read, like the evidence or the agent prompt |
PR Summary by QodoDrop redundant jsonfilter selections from device defconfigs
AI Description
Diagram
High-Level Assessment
Files changed (97)
|
…129) * ci: watch for drift from firmware instead of finding it by accident builder.sh copies devices/<item>/* over a fresh firmware clone, so a device directory that ships its own copy of a firmware file replaces it outright. A fleet-wide change in firmware reaches every device that only references a path and none that ships its own copy, silently, because nothing in either repository is looking. One comment on OpenIPC/firmware#2308 turned up three instances of that in an afternoon: 13 board configs that missed the CONFIG_VT sweep (#126), 97 defconfigs that kept BR2_PACKAGE_JSONFILTER after firmware retired it and put three devices over their rootfs cap (#128), and excludes lists naming files that no longer exist (firmware#2313 reports those at build time). The trigger is the design. Every one of those was caused by a commit in firmware while this repository sat untouched, so a pull_request check would never have fired for any of them -- nobody opens a builder PR when firmware changes. firmware-drift.yml runs on a schedule and files an issue rather than failing: drift someone else introduced is not a reason to redden a nightly that is otherwise fine. Only the self-test gates a merge here, out of lint.yml. Two checks, because the two kinds of drift are not the same shape. Shadowed files: comparing content is useless, a builder copy is *supposed* to differ. What matters is whether firmware's version moved since a human last reconciled them, so each of the 13 entries pins the blob it was reconciled against. Re-pinning is the act of looking again. The mapping is hand-authored -- nothing can infer that gk7205v200.generic-fpv derives from gk7205v200.generic. Defconfig symbols: nothing here copies a firmware defconfig, so pinning does not apply. Every BR2_PACKAGE_*=y is resolved against buildroot, firmware and this tree and sorted into "resolves nowhere" (a dead line kconfig ignores) and "resolves, but no firmware defconfig selects it" (needs an allowlist entry). An allowlist entry can also fence a symbol to the devices that need it; without that, writing JSONFILTER down as builder-only for devices/apfpv would equally bless it creeping back onto 95 unrelated defconfigs, which is #128 verbatim. Replayed against the three regressions, each reproduced by hand on a clean tree: firmware moves a shadowed config -> br-ext-chip-goke/board/gk7205v200/gk7205v200.generic.config moved in firmware since this copy was reconciled (2026-08-25). builder copy: devices/common/.../gk7205v200.generic-fpv.config pinned 7450c6803f6d, firmware now f97dce84c0f8 firmware commits since: 85233f0e kernel: drop CONFIG_VT and the busybox applets ... (#2308) jsonfilter creeps back onto a non-apfpv device -> BR2_PACKAGE_JSONFILTER is allowlisted only for apfpv/*/configs/*_defconfig, but 1 other defconfig(s) select it. firmware retires a symbol builder still selects -> BR2_PACKAGE_LIBUBOX is selected here but by no firmware defconfig. Resolving against buildroot is not optional and getting it wrong is how a checker lies. Buildroot is vendored by neither repository, so a first pass over OpenIPC packages alone called HOSTAPD, IW, PHP, UHTTPD, LIBZIP and BWM_NG dead; all six are upstream. Matching only "Config.in" then called PHP_EXT_ZIP dead, because buildroot declares php's extensions in package/php/Config.ext. Without --buildroot that half is skipped and says so rather than inventing findings. Two symbols really do resolve nowhere today -- BR2_PACKAGE_APFPV_GREG on the four greg-generic devices and BR2_PACKAGE_WIFIBROADCAST_EXT in devices/common. Both are recorded as known-dead and reported as a notice on every run rather than removed: whether those devices still want the packages is a maintainer's call, not a side effect of adding a check. The three new paths are classified in ci-matrix.py so this PR and every future re-pin cost no device builds. Without that a blob update is a 107-device run. * ci: discover shadowed files instead of listing them by hand Qodo caught the shadow list omitting the device-local t21 and t31 kernel configs. It was right, and the hole was much bigger than the five it named: a hand-written list of 13 board configs against 93 same-path shadows covering 35 firmware paths. The missing ones are not incidental -- load_goke, load_hisilicon, load_sigmastar and four vendor .mk files are all shadowed by some device, and those are exactly the shared files a fleet-wide fix lands in. firmware regression-tests load_hisilicon's os_mem_size derivation; two devices here ship their own copy of that script and would not have seen the fix. Listing them by hand was the wrong shape. builder.sh copies devices/<item>/* over the firmware clone, so any file at a path firmware also has replaces it -- which makes same-path shadows discoverable rather than something to remember. discover_shadows() walks devices/ and reports anything sitting on a firmware path with no entry in firmware-drift.json, so the list cannot silently rot and a device added tomorrow cannot quietly introduce an unreconciled shadow. The manifest goes from 13 entries to 101: the 88 discovered ones, plus the 13 hand-authored entries that discovery cannot find because the name differs from the file they replace -- gk7205v200.generic-fpv shadows gk7205v200.generic, and nothing can infer that. Two self-test cases for the new path: a same-path shadow with no entry is reported, and an entry silences it. Verified against the tree by dropping the load_hisilicon entries, which surfaces both devices immediately.
1 participant
Fixes the three devices that have been red on the nightly, and unblocks #126 and #127 — whose only failures are these same three.
What is wrong
OpenIPC/firmware dropped
BR2_PACKAGE_JSONFILTERfrom every one of its defconfigs in #2304, once the WebUI stopped calling it. builder's defconfigs are its own files, so 97 of them kept selecting it.jsonfilterselectsJSON_CandLIBUBOX, and on a device that enables neitheruqmi-openipcnorwifibroadcast-ng, nothing else selectslibubox— so it andjsonfiltershipped for nothing.That is what has had three devices over their rootfs cap:
hi3518ev200_lite_switcam-hs303ssc333_lite_babysense-see-hd-ip206ssc333_lite_meari-speed-6sHow little it took
Bisecting
meariacross firmware revisions: the last passing nightly (08-23) built firmware4223e6ff, the first failing one (08-24) builtf5c75187. Two commits separate them, one CI-only. So it is #2301, which added 408 bytes to the comment-strippedsysupgrade:408 bytes of a genuine bug fix pushed the image over a 4KB padding step, because it had no headroom at all. The fix is to give these images headroom, not to give back the bug fix.
Measured
ssc333_lite_meari-speed-6s, built the waybuilder.shbuilds it —devices/<item>/copied over a firmware checkout atmaster, clean tree both times:−32KB, from
/usr/bin/jsonfilter(18068 raw) and/usr/lib/libubox.so(30404 raw).And the hisilicon one, built the same way — different SoC, different vendor SDK, same result:
Neither
/usr/bin/jsonfilternor/usr/lib/libubox.sois in either target tree afterwards.ssc333_lite_babysense-see-hd-ip206is not built here: it is the same SoC, variant and cap asmeariand needs 4KB wheremearineeded 8KB, so it is covered by the smaller of the two measurements.json-cdoes not go:majesticlists it inMAJESTIC_DEPENDENCIES, so buildroot builds and installs it whether or notBR2_PACKAGE_JSON_Cis set. Only those two files leave the image — worth stating because the size report attributes 17KB tojson-cand that is not part of this saving.What keeps it
Two consumers genuinely need
jsonfilter, and both keep it:devices/apfpv— its ownetc/init.d/S99msposdreads majestic'sconfig.jsonthrough it, andssc338q_apfpv/ssc378qe_apfpvdo not enablewifibroadcast-ng. Those two keep the symbol with the reason written above the line, which is why this is 95 defconfigs and not 97. Same undeclared-runtime-dependency trap firmware #2305 had to fix.wifibroadcast-ng, whoseConfig.inselectsBR2_PACKAGE_JSONFILTER(added by #2305) for the same script. They keep it without saying so.LTE devices keep
libuboxthroughuqmi-openipc, which selects it independently.Nothing else here calls
jsonfilter: greppingdevices/andpackage/finds only that oneS99msposdline, and firmware's tree has no caller left outsidewifibroadcast-ng.Related
Third instance of the same root cause @flyrouter raised on firmware#2308: builder keeps its own copies of files firmware also maintains, and a fleet-wide change to firmware does not reach them. #126 is the kernel-config instance; this is the defconfig instance.