Optimize upb_Arena memory allocation for repeated growths#28150
Draft
copybara-service[bot] wants to merge 1 commit into
Draft
Optimize upb_Arena memory allocation for repeated growths#28150copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
Introduce a memory pool within upb_Arena to recycle power-of-2 sized blocks (on 32-bit) and map-entry-aligned sizes (on 64-bit). This addresses the "arena leak" pattern where structures like repeated fields (arrays) and maps (hash tables) reallocate repeatedly and leave abandoned, unreclaimable blocks in the arena. Key changes: - Implement upb_Arena_AllocPool and upb_Arena_FreePool in arena.c using an in-place sorted linked list of size classes (LIFO stacks of free blocks), avoiding any intermediate allocations. - Use dynamic size classes of the form UPB_MAP_ENTRY_SIZE * 2^k, which dynamically resolves to 24 * 2^k on 64-bit and 16 * 2^k on 32-bit (collapsing to pure powers of 2). - Achieve 0% memory overhead for all maps >= 48 bytes by matching the map resize growth system exactly. - Integrate the pool into upb_Array reallocation with Capacity Coalescing, allowing arrays to safely use these same size classes and share memory blocks with maps. - Fix a critical memory corruption bug by protecting the array's inline initial data block from being freed to the pool. - Integrate the pool into upb_table (strtable, exttable, inttable) resizes to reclaim old entry arrays. - Guard the implementation with extensive UPB_ASSERTs for alignment, sizes, sorting, and stack invariants. - Ensure full compatibility and safety under ASAN/HWASAN (handling tag-stripping and aligned poisoning). PiperOrigin-RevId: 936228005
2f17ec3 to
527fe86
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Optimize upb_Arena memory allocation for repeated growths
Introduce a memory pool within upb_Arena to recycle power-of-2 sized blocks (on 32-bit) and map-entry-aligned sizes (on 64-bit). This addresses the "arena leak" pattern where structures like repeated fields (arrays) and maps (hash tables) reallocate repeatedly and leave abandoned, unreclaimable blocks in the arena.
Key changes: