Skip to content

Optimize upb_Arena memory allocation for repeated growths#28150

Draft
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_936228005
Draft

Optimize upb_Arena memory allocation for repeated growths#28150
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_936228005

Conversation

@copybara-service

@copybara-service copybara-service Bot commented Jun 22, 2026

Copy link
Copy Markdown

Optimize upb_Arena memory allocation for repeated growths

Introduce a memory pool within upb_Arena to recycle power-of-2 sized blocks (on 32-bit) and map-entry-aligned sizes (on 64-bit). This addresses the "arena leak" pattern where structures like repeated fields (arrays) and maps (hash tables) reallocate repeatedly and leave abandoned, unreclaimable blocks in the arena.

Key changes:

  • Implement upb_Arena_AllocPool and upb_Arena_FreePool in arena.c using an in-place sorted linked list of size classes (LIFO stacks of free blocks), avoiding any intermediate allocations.
  • Use dynamic size classes of the form UPB_MAP_ENTRY_SIZE * 2^k, which dynamically resolves to 24 * 2^k on 64-bit and 16 * 2^k on 32-bit (collapsing to pure powers of 2).
  • Achieve 0% memory overhead for all maps >= 48 bytes by matching the map resize growth system exactly.
  • Integrate the pool into upb_Array reallocation with Capacity Coalescing, allowing arrays to safely use these same size classes and share memory blocks with maps.
  • Fix a critical memory corruption bug by protecting the array's inline initial data block from being freed to the pool.
  • Integrate the pool into upb_table (strtable, exttable, inttable) resizes to reclaim old entry arrays.
  • Guard the implementation with extensive UPB_ASSERTs for alignment, sizes, sorting, and stack invariants.
  • Ensure full compatibility and safety under ASAN/HWASAN (handling tag-stripping and aligned poisoning).

Introduce a memory pool within upb_Arena to recycle power-of-2 sized blocks (on 32-bit) and map-entry-aligned sizes (on 64-bit). This addresses the "arena leak" pattern where structures like repeated fields (arrays) and maps (hash tables) reallocate repeatedly and leave abandoned, unreclaimable blocks in the arena.

Key changes:
- Implement upb_Arena_AllocPool and upb_Arena_FreePool in arena.c using an in-place sorted linked list of size classes (LIFO stacks of free blocks), avoiding any intermediate allocations.
- Use dynamic size classes of the form UPB_MAP_ENTRY_SIZE * 2^k, which dynamically resolves to 24 * 2^k on 64-bit and 16 * 2^k on 32-bit (collapsing to pure powers of 2).
- Achieve 0% memory overhead for all maps >= 48 bytes by matching the map resize growth system exactly.
- Integrate the pool into upb_Array reallocation with Capacity Coalescing, allowing arrays to safely use these same size classes and share memory blocks with maps.
- Fix a critical memory corruption bug by protecting the array's inline initial data block from being freed to the pool.
- Integrate the pool into upb_table (strtable, exttable, inttable) resizes to reclaim old entry arrays.
- Guard the implementation with extensive UPB_ASSERTs for alignment, sizes, sorting, and stack invariants.
- Ensure full compatibility and safety under ASAN/HWASAN (handling tag-stripping and aligned poisoning).

PiperOrigin-RevId: 936228005
@copybara-service copybara-service Bot changed the title Internal change Optimize upb_Arena memory allocation for repeated growths Jun 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant