kernel/build: Fix the protected build #18262

Barto22 · 2026-01-29T11:50:02Z

Summary

This patch adds CMake build support for NuttX protected mode (kernel/user space separation) on STM32F4Discovery and similar ARM boards. Previously, protected mode configurations like stm32f4discovery:kostest could only be built with the Make build system. The CMake build system failed due to several architectural issues:

Key Changes:

Multiple Linker Script Support (CMakeLists.txt)
- Modified linker script preprocessing to handle multiple scripts via foreach loops
- Protected mode requires two linker scripts: memory.ld (memory regions) and kernel-space.ld/user-space.ld (section placements)
- Previous implementation assumed single linker script via get_filename_component()
- Now preprocesses each script individually and builds list with -T flags
User Space Binary Build (boards/.../kernel/CMakeLists.txt)
- Changed from nuttx_add_aux_library() to target_sources(nuttx_user PRIVATE ...)
- The nuttx_add_aux_library() was creating an empty target without source files
- Direct target_sources() properly adds userspace initialization code
Linker Script Selection (boards/.../src/CMakeLists.txt)
- Added conditional logic to select correct linker scripts based on CONFIG_BUILD_PROTECTED
- Protected mode: memory.ld + kernel-space.ld for kernel, memory.ld + user-space.ld for user
- Flat mode: ld.script (existing behavior)
- Removed hardcoded -funwind-tables flag that was adding ~4.7KB bloat per binary
Critical .userspace Section (boards/.../scripts/user-space.ld)
- Added KEEP(*(.userspace)) directive to prevent linker garbage collection
- Added EXTERN() declarations for memory management functions
- The .userspace section contains struct userspace_s with memory map and entry points
- Without KEEP(), CMake's static library linking discards unreferenced sections
Memory Layout Updates (boards/.../scripts/memory.ld)
- Fixed memory region sizes: ksram 4KB→16KB, usram 4KB→16KB, xsram 104KB→80KB
- Updated comments to reflect correct memory map for protected mode
- Aligns with kernel heap size requirement (CONFIG_MM_KERNEL_HEAPSIZE=16384)
Configuration Updates (boards/.../configs/kostest/)
- Added Make.defs for kostest configuration (defines dual linker scripts)
- Updated defconfig: enabled FPU, increased idle stack, excluded CCM, set kernel heap size

Technical Details:

The root cause was that Make builds link userspace object files directly to the linker, while CMake links from static library archives. When sections aren't explicitly referenced, the linker's garbage collector removes them from archives. The .userspace section at address 0x08020000 is critical—the kernel reads it during boot to initialize user space memory regions, entry points, and heap structures.

Impact

Positive Impacts:

Enables CMake build system for protected mode configurations on STM32 and similar ARM platforms
Maintains binary size parity with Make builds (~62KB user, ~60KB kernel for kostest)
No changes to existing flat mode builds or other architectures
Provides path forward for deprecating Make build system

Build Process:

CMake 3.x or higher required (tested with 4.2.3)
Ninja generator recommended for parallel builds
Build command: cmake -B build -DBOARD_CONFIG=stm32f4discovery:kostest -GNinja && ninja -C build
Compatible with existing Make builds: ./tools/configure.sh stm32f4discovery:kostest && make

Testing

Test Environment:

Host: Linux (Ubuntu/similar distribution)
CMake version: 4.2.3
Ninja version: 1.11.1
Toolchain: arm-none-eabi-gcc 15.2.1 (arm-gnu-toolchain)
Board: STM32F4Discovery (STM32F407VG, ARM Cortex-M4 @ 168MHz)
Configuration: stm32f4discovery:kostest (protected mode OS test suite)

acassis · 2026-01-29T13:25:52Z

@Barto22 please change the PR message to something like: kernel/build: fix the protected build

Barto22 · 2026-01-29T14:02:11Z

Could someone explain why build for arm-12 failed? It seems that everything is ok and then fail. And could it be rerun manually or I need to push some commit to trigger it once again?

acassis · 2026-01-29T14:10:59Z

Could someone explain why build for arm-12 failed? It seems that everything is ok and then fail. And could it be rerun manually or I need to push some commit to trigger it once again?

Seems like some defconfigs need to be normalized:

no changes added to commit (use "git add" and/or "git commit -a")
Configuring...
Disabling CONFIG_ARM_TOOLCHAIN_GNU_EABI
Enabling CONFIG_ARM_TOOLCHAIN_GNU_EABI
Building NuttX...
chip/stm32_gpio.c:44:11: note: '#pragma message: CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py'
44 | # pragma message "CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py"
| ^~~~~~~
[1/1] Normalize stm32f4discovery/mmcsdspi
HEAD detached at pull/18262/merge
Changes not staged for commit:
(use "git add ..." to update what will be committed)
(use "git restore ..." to discard changes in working directory)
modified: boards/arm/stm32/stm32f4discovery/configs/kostest/defconfig

@simbit18 do you see other reason?

Barto22 · 2026-01-29T14:14:33Z

Could someone explain why build for arm-12 failed? It seems that everything is ok and then fail. And could it be rerun manually or I need to push some commit to trigger it once again?

Seems like some defconfigs need to be normalized:

no changes added to commit (use "git add" and/or "git commit -a") Configuring... Disabling CONFIG_ARM_TOOLCHAIN_GNU_EABI Enabling CONFIG_ARM_TOOLCHAIN_GNU_EABI Building NuttX... chip/stm32_gpio.c:44:11: note: '#pragma message: CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py' 44 | # pragma message "CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py" | ^~~~~~~ [1/1] Normalize stm32f4discovery/mmcsdspi HEAD detached at pull/18262/merge Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git restore ..." to discard changes in working directory) modified: boards/arm/stm32/stm32f4discovery/configs/kostest/defconfig

@simbit18 do you see other reason?

But this warning and note is not the reason of fail. It shows in other builds also and didn't fail them.

simbit18 · 2026-01-29T14:14:37Z

Hi @Barto22 @acassis step
Normalize stm32f4discovery/kostest

====================================================================================
Configuration/Tool: stm32f4discovery/kostest,CONFIG_ARM_TOOLCHAIN_GNU_EABI
2026-01-29 13:25:59
------------------------------------------------------------------------------------
  Cleaning...
  Configuring...
  Disabling CONFIG_ARM_TOOLCHAIN_GNU_EABI
  Enabling CONFIG_ARM_TOOLCHAIN_GNU_EABI
  Building NuttX...
chip/stm32_gpio.c:44:11: note: '#pragma message: CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py'
   44 | #  pragma message "CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py"
      |           ^~~~~~~
  [1/1] Normalize stm32f4discovery/kostest
15d14
< CONFIG_ARCH_FPU=y
28a28
> CONFIG_MM_KERNEL_HEAPSIZE=16384
41a42
> CONFIG_STM32_CCMEXCLUDE=y
51,52d51
< CONFIG_STM32_CCMEXCLUDE=y
< CONFIG_MM_KERNEL_HEAPSIZE=16384
Saving the new configuration file
HEAD detached at pull/18262/merge
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   boards/arm/stm32/stm32f4discovery/configs/kostest/defconfig

no changes added to commit (use "git add" and/or "git commit -a")
==================================================================================
==

Barto22 · 2026-01-29T16:37:05Z

Hi @Barto22 @acassis step

Normalize stm32f4discovery/kostest


====================================================================================

Configuration/Tool: stm32f4discovery/kostest,CONFIG_ARM_TOOLCHAIN_GNU_EABI

2026-01-29 13:25:59

------------------------------------------------------------------------------------

  Cleaning...

  Configuring...

  Disabling CONFIG_ARM_TOOLCHAIN_GNU_EABI

  Enabling CONFIG_ARM_TOOLCHAIN_GNU_EABI

  Building NuttX...

chip/stm32_gpio.c:44:11: note: '#pragma message: CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py'

   44 | #  pragma message "CONFIG_STM32_USE_LEGACY_PINMAP will be deprecated migrate board.h see tools/stm32_pinmap_tool.py"

      |           ^~~~~~~

  [1/1] Normalize stm32f4discovery/kostest

15d14

< CONFIG_ARCH_FPU=y

28a28

> CONFIG_MM_KERNEL_HEAPSIZE=16384

41a42

> CONFIG_STM32_CCMEXCLUDE=y

51,52d51

< CONFIG_STM32_CCMEXCLUDE=y

< CONFIG_MM_KERNEL_HEAPSIZE=16384

Saving the new configuration file

HEAD detached at pull/18262/merge

Changes not staged for commit:

  (use "git add <file>..." to update what will be committed)

  (use "git restore <file>..." to discard changes in working directory)

	modified:   boards/arm/stm32/stm32f4discovery/configs/kostest/defconfig



no changes added to commit (use "git add" and/or "git commit -a")

==================================================================================

==

Ok, I will take a look at this later and fix.

boards/arm/stm32/stm32f4discovery/src/CMakeLists.txt

- Added support for multiple linker scripts preprocessing in CMake - Changed kernel/CMakeLists.txt to use target_sources for nuttx_user - Added KEEP() directive in user-space.ld for .userspace section - Removed hardcoded -funwind-tables to fix binary size bloat - Updated memory regions: ksram/usram 4KB→16KB, xsram 104KB→80KB - Enabled CONFIG_ARCH_FPU, CONFIG_STM32_CCMEXCLUDE, adjusted heap size Fixes protected mode CMake build for stm32f4discovery:kostest configuration. Signed-off-by: Bartosz <[email protected]>

Barto22 requested review from GUIDINGLI, jerpelea, masayuki2009, raiden00pl, simbit18 and xiaoxiang781216 as code owners January 29, 2026 11:50

github-actions bot added Area: Build system Board: arm Size: M The size of the change in this PR is medium labels Jan 29, 2026

acassis previously approved these changes Jan 29, 2026

View reviewed changes

Barto22 changed the title ~~Fix protected build~~ kernel/build: Fix the protected build Jan 29, 2026

xiaoxiang781216 previously approved these changes Jan 30, 2026

View reviewed changes

boards/arm/stm32/stm32f4discovery/src/CMakeLists.txt Show resolved Hide resolved

Barto22 dismissed stale reviews from xiaoxiang781216 and acassis via 95b9a3e January 30, 2026 05:08

Barto22 requested review from acassis and xiaoxiang781216 January 30, 2026 05:09

Barto22 force-pushed the feature/fix-protected-build branch 2 times, most recently from 9fa5a73 to 08c485e Compare January 30, 2026 11:37

Barto22 force-pushed the feature/fix-protected-build branch from 08c485e to d4c0e6d Compare January 30, 2026 11:50

xiaoxiang781216 approved these changes Jan 30, 2026

View reviewed changes

Donny9 approved these changes Feb 2, 2026

View reviewed changes

Donny9 merged commit fe184f7 into apache:master Feb 2, 2026
40 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kernel/build: Fix the protected build #18262

kernel/build: Fix the protected build #18262

Barto22 commented Jan 29, 2026

Uh oh!

acassis commented Jan 29, 2026

Uh oh!

Barto22 commented Jan 29, 2026 •

edited

Loading

Uh oh!

acassis commented Jan 29, 2026

Uh oh!

Barto22 commented Jan 29, 2026

Uh oh!

simbit18 commented Jan 29, 2026

Uh oh!

Barto22 commented Jan 29, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kernel/build: Fix the protected build #18262

kernel/build: Fix the protected build #18262

Conversation

Barto22 commented Jan 29, 2026

Summary

Key Changes:

Technical Details:

Impact

Testing

Uh oh!

acassis commented Jan 29, 2026

Uh oh!

Barto22 commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

acassis commented Jan 29, 2026

Uh oh!

Barto22 commented Jan 29, 2026

Uh oh!

simbit18 commented Jan 29, 2026

Uh oh!

Barto22 commented Jan 29, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Barto22 commented Jan 29, 2026 •

edited

Loading