Skip to content

system/libarchive: Add libarchive 3.7.7 with NuttX patches. - #4076

Draft
txy-21 wants to merge 28 commits into
apache:masterfrom
txy-21:upstream-libarchive
Draft

txy-21 wants to merge 28 commits into
apache:masterfrom
txy-21:upstream-libarchive

Conversation

@txy-21

@txy-21 txy-21 commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Why: the zipfs file system of nuttx reads and writes zip archives with
    libarchive, and nuttx-apps has no libarchive. The nuttx pull request is !fs/zipfs: Read zip files with libarchive and add the wzipfs writer. nuttx#21950.
  • What: new directory system/libarchive, with the Kconfig, Makefile,
    CMakeLists.txt, Make.defs and config.h files, and 13 patch files.
  • How:
    • The libarchive source is not in the repository. When UTILS_LIBARCHIVE is
      enabled, the Make build downloads libarchive-3.7.7.zip at the context
      step, and the CMake build fetches it with FetchContent. Then every file
      that matches 0*.patch is applied, in the order of its name.
    • 0001 fixes names that clash with stdio.h and a redefined MAX_SIGNO.
      0002 to 0013 make parts of libarchive optional (Unicode normalization,
      error strings, PPMd, Windows paths, Mac extensions, encryption, symbolic
      links, ZIP64, the ZIP xl field), fix the handling of the normalization
      flags, and remove a signed/unsigned comparison warning of newer clang.
      Patch files are used because the source is downloaded.
    • UTILS_LIBARCHIVE depends on FILE_STREAM. The command line programs
      (bsdunzip, bsdtar, bsdcat, cpio, minitar, tarfilter) are separate
      options.
  • The 28 commits keep their original authors. Commits 1 to 15 are the history
    of the Kconfig, config.h and build files, commit 16 makes the build apply
    every numbered patch, and commits 17 to 28 add the patch files 0002 to
    0013. Their subjects repeat those of the Kconfig commits.
  • Related: system/libarchive: build libarchive support #3318 was closed. It proposed the same libarchive
    port.

Impact

  • Is new feature added? Is existing feature changed? YES. The library is new.
    No existing option or code changes.
  • Impact on user (will user need to adapt to change)? NO. UTILS_LIBARCHIVE is
    off by default.
  • Impact on build (will build process change)? NO for the existing
    configurations. YES when UTILS_LIBARCHIVE is enabled, because the build
    downloads libarchive from GitHub. After a build, changing the configuration
    and running "make" again can fail, because the Makefile extracts the zip file
    again and applies the patches a second time. "make distclean" and removing
    the extracted libarchive directory fix it.
  • Impact on hardware (will arch(s) / board(s) / driver(s) change)? NO.
  • Impact on documentation (is update required / provided)? NO. The nuttx pull
    request documents zipfs.
  • Impact on security (any sort of implications)? YES. libarchive is third
    party code that parses archives, and the option for encrypted entries is off
    by default.
  • Impact on compatibility (backward/forward/interoperability)? NO.
  • Anything else to consider or add? The question whether UTILS_LIBARCHIVE
    should depend on ALLOW_BSD_COMPONENTS was asked in system/libarchive: build libarchive support #3318, and I did not
    find an answer. The Kconfig does not have that dependency.

Testing

I confirm that changes are verified on local setup and works as intended:

  • Build Host(s): Linux 7.0.0-34-generic x86_64, Intel i7-14700,
    gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0, GNU Make 4.3, CMake 3.28.3,
    Ninja 1.11.1.
  • Target(s): sim:zipfs, with the nuttx pull request. Real hardware was not
    tested.
  • The 28 commits were replayed on nuttx-apps 72937e7 and built with nuttx
    e9eefaa0e9a plus the nuttx pull request. Make and CMake with Ninja both
    download libarchive and zlib, apply the 13 patches, and build sim:zipfs
    without errors or warnings. In the CMake build, libarchive has 1442 files,
    no .rej file, and the string UTILS_LIBARCHIVE_ZIP64 appears 7 times in
    libarchive/archive_read_support_format_zip.c.

Testing logs before change: not applicable, the directory is new.

Testing logs after change (the nuttx binary built by CMake with Ninja), NSH
session as printed:

after
nsh> mount -t hostfs -o fs=/tmp/zh.GX1FlK/host /host
nsh> mount -t zipfs -o /host/test.zip /zip
nsh> echo ---ls-zip
---ls-zip
nsh> ls /zip
/zip:
 a/
 a/1
 a/2
 top.txt
 文件.txt
nsh> echo ---cat-top
---cat-top
nsh> cat /zip/top.txt
top level file
nsh> echo ---cat-a1
---cat-a1
nsh> cat /zip/a/1
this is zipfs test 1
nsh> echo ---cat-utf8
---cat-utf8
nsh> cat /zip/文件.txt
utf8 name file
nsh> echo ---ls-l-top
---ls-l-top
nsh> ls -l /zip/top.txt
 -rw-rw-r--          15 2026-11-17 12:18 /zip/top.txt
nsh> umount /zip
nsh> echo done
done
nsh> poweroff

Not tested: real hardware, ostest, other architectures, the command line
programs (they were not built or run), and an encrypted archive. The nine
libarchive options (default, all on, each alone) were built before the last
rebase onto the current master, and the default and the all-on builds read an
archive in the simulator.

PR verification Self-Check

  • This PR introduces only one functional change.
  • I have updated all required description fields above.
  • My PR adheres to Contributing Guidelines and Documentation (git commit title and message, coding standard, etc).
  • My PR is still work in progress (not ready for review).
  • My PR is ready for review and can be safely merged into a codebase.

🤖 Generated with Claude Code

anjiahao1 and others added 28 commits November 17, 2026 20:32
Add system/libarchive.  It downloads libarchive 3.7.7 when the build is
configured, applies 0001-fix-conflict-libarchive-for-nuttx.patch to it, and
builds it for the Make and the CMake build.  The library and some of its
programs can be enabled in Kconfig.

Assisted-by: Claude Code:claude-sonnet-5.5
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Define HAVE_LIBLZMA and HAVE_LIBLZMADEC in config.h when CONFIG_UTILS_XZ is
set, and HAVE_LIBZ when CONFIG_LIB_ZLIB is set, so that libarchive uses the
compression libraries that are enabled.

Assisted-by: Claude Code:claude-sonnet-5.5
Signed-off-by: anjiahao <anjiahao@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
libarchive needs the locale support of the C library, so UTILS_LIBARCHIVE
depends on LIBC_LOCALE.

Assisted-by: Claude Code:claude-sonnet-5.5
Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
1. add UTILS_LIBARCHIVE_UNICODE_NORMALIZATION
    Enable Unicode NFC/NFD normalization for filename handling.
    Disabling it can reduce the size of about 17K code (under LTO),
    or 29K without LTO.
2. disable HAVE_LANGINFO, HAVE_ICONV according to CONFIG_LIBC_LOCALE

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Reformat config.h to follow the NuttX coding style: add the Included Files
and Pre-processor Definitions sections, and write the comments in the NuttX
style.

Assisted-by: Claude Code:claude-sonnet-5.5
Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Add a config which enables error strings, and default it to n.
This saves flash size of about 4K when enabling libarchive.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Add config of ppmd and default to n.
This saves flash size of about 6K when enabling libarchive.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 0.11K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 1.38K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 2.2K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 0.35K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 0.44K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 0.18K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
libarchive unconditionally compiles libarchive_fe/*.c (line_reader.c,
err.c) and core archive_read_open_file.c / archive_write_open_file.c
which use FILE* / fopen / stdin. When CONFIG_FILE_STREAM is disabled,
the lib_get_stream() prototype in nuttx/lib/lib.h is hidden, so the
stdin macro (stdio.h) expands to a call to an undeclared function
returning int, triggering -Werror=int-conversion in line_reader.c.

Add an explicit dependency so UTILS_LIBARCHIVE only appears when
CONFIG_FILE_STREAM is enabled, preventing the invalid config combo.

Signed-off-by: haopengxiang <haopengxiang@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
The iso9660 writer passes an enum parameter to va_start(),
which undergoes default argument promotion. Clang treats this
as an error with -Werror,-Wvarargs. Add -Wno-varargs to both
CMakeLists.txt and Makefile to suppress it.

Signed-off-by: cuiziwei <cuiziwei@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
The sources of libarchive are downloaded at build time, and only
0001-fix-conflict-libarchive-for-nuttx.patch was applied to them.

Apply all the files that match 0*.patch in name order, in the Make and
in the CMake build, so that further patches can be added next to it.

Assisted-by: Claude Code:claude-sonnet-5.5
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
add UTILS_LIBARCHIVE_UNICODE_NORMALIZATION
Enable Unicode NFC/NFD normalization for filename handling.
Disabling it can reduce the size of about 17K code (under LTO),
or about 29K without LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Add a config which enables error strings, and default it to n.
This saves flash size of about 4K when enabling libarchive.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Add config of ppmd and default to n.
This saves flash size of about 6K when enabling libarchive.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disable the entire archive_string_normalize_C and archive_string_normalize_D
functions instead of only their internal implementations. Otherwise the
converter is still executed, leaving variables uninitialized and causing
null pointer dereference and memfault.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Problem:
After mounting, files could not be found due to a failure in string conversion.
This issue occurred when `CONFIG_UTILS_LIBARCHIVE_UNICODE_NORMALIZATION` is disabled:
- The `SCONV_NORMALIZATION_C` and `SCONV_NORMALIZATION_D` flags
  were still being set in some cases, even though
  `archive_string_normalize_C` and `archive_string_normalize_D` are disabled.
- This caused the `setup_converter` function to skip adding the
  `strncat_from_utf8_to_utf8` converter.
- As a result, no converter was added, and `nconverter` was set to 0, leading to the failure.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 0.11K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 1.38K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 2.2K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 0.35K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 0.44K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Disabling it saves 0.18K under LTO.

Signed-off-by: zhengyu16 <zhengyu16@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Newer Clang versions emit -Werror on the INT_MIN <= O_NOFOLLOW
comparison because INT_MIN (negative) gets implicitly converted to
unsigned. Simplify the check to O_NOFOLLOW > INT_MAX which is
logically equivalent and avoids the diagnostic.

Signed-off-by: cuiziwei <cuiziwei@xiaomi.com>
Signed-off-by: tengshuangshuang <tengshuangshuang@xiaomi.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants