feat(memtrack): collect RSS via rss_stat and folio-rmap reconstruction by not-matthias · Pull Request #453 · CodSpeedHQ/codspeed

not-matthias · 2026-07-13T13:04:00Z

Summary

Adds RSS (resident set size) collection to memtrack, in two layers:

Authoritative RSS from the kernel's kmem:rss_stat tracepoint — absolute per-mm resident bytes (anon/file/shmem/swap), latest-wins.
Reconstructed anonymous RSS from raw kernel folio-rmap events: fentry hooks on the anon folio-rmap add/remove functions emit signed page-count deltas, so anon RSS can be rebuilt over time as Σ(add − remove) × PAGE_SIZE.

The reconstruction is a total anon RSS delta-sum, not a per-vaddr resident map — the kernel remove hook (folio_remove_rmap_ptes) carries no address, so removals can't be attributed to a vaddr (the add hooks' faulting vaddr is emitted for observability only).

Commits

feat(memtrack): track RSS via kmem:rss_stat tracepoint — the baseline: EVENT_TYPE_RSS contract, MemtrackEventKind::Rss, parser arm, writer bench case, gated integration test.
feat(memtrack): reconstruct anon RSS from gated folio rmap fentry hooks — EVENT_TYPE_RMAP_ANON + RmapAnon event, five fentry programs (add_new / add_ptes / remove_ptes / remove_pmd / remove_pud), CO-RE folio helpers, and the load/attach gating.
test(memtrack): validate anon RSS reconstruction against rss_stat — ramps anon RSS via mmap/munmap and asserts the reconstructed estimate tracks the rss_stat MM_ANONPAGES peak within 25%.

What's on by default vs gated

rss_stat tracepoint: always on. Emitting Rss events is the intended new default behavior introduced by this change — the RSS tracepoint is not gated.
RmapAnon folio-rmap fentry programs: off by default, gated behind CODSPEED_MEMTRACK_TRACK_RMAP=1. When the flag is unset they are set_autoload(false) before load and never attached, so:
- the skeleton still loads on any kernel (a missing fentry BTF target would otherwise fail the whole load), and
- the folio-rmap reconstruction path stays out of production (--mode memory) and out of the existing test suites — no RmapAnon events are produced by default.

Verification

Run in a privileged, --pid=host container sharing the host kernel (7.0.12):

Reconstruction (flag on): ✅ real anon amplitude = 64 MiB, estimated peak = 64 MiB (ratio 1.00).
Regression (rss_tests, flag unset): ✅ passes — no RmapAnon events, folio-rmap programs stay unloaded.
Parser unit tests, cargo fmt, and clippy clean.
Kernel BTF signatures for all five folio_*_rmap* functions verified to match the BPF_PROG arg layouts.

Review notes (draft)

track_command ordering: the shared test helper spawns the child before enable()/track(root_pid). In practice the child's fork→execve→ld.so→libc-init far outlasts the two BPF-map updates, so tracking is armed before the workload allocates (both fixtures captured full event streams). Flagging in case we'd prefer a leading settle-usleep in the fixtures or an enable-before-spawn change in the helper.
PAGE_SIZE: hardcoded to 4096 (correct on x86_64). On a 16K/64K-page arm64 runner the estimate would need sysconf(_SC_PAGESIZE); rss_stat is already in bytes and unaffected. Happy to switch to sysconf if these tests run on arm64 CI.
The chart/inspection tooling used during development lives outside the tree (dev artifact) and is not part of this PR.

codspeed-hq · 2026-07-13T13:06:37Z

Merging this PR will not alter performance

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 17 untouched benchmarks

_{Comparing cod-3089-collect-rss-in-memtrack (e3a67b4) with main (26fb4b5)}

greptile-apps · 2026-07-13T13:07:01Z

Greptile Summary

This PR adds RSS collection and optional rmap-based reconstruction to memtrack. The main changes are:

New RSS, rmap, fork, exec, and exit event types.
kmem:rss_stat collection for per-process resident memory.
Optional folio-rmap fentry hooks for RSS reconstruction.
RSS integration fixtures, snapshots, and CI matrix coverage.
Memtrack artifact parsing and writer support for the new events.

Confidence Score: 4/5

This is close, but the startup failure path should be fixed before merging.

rss_stat attach failures are handled gracefully.
The new lifecycle hook attaches still return errors directly.
A host without one of those hooks can fail memtrack startup even when allocator tracking would otherwise work.

crates/memtrack/src/ebpf/memtrack/tracking.rs

Important Files Changed

Filename	Overview
crates/memtrack/src/ebpf/memtrack/tracking.rs	Adds RSS and rmap attachment logic, with lifecycle hook errors still aborting startup.
crates/memtrack/src/ebpf/c/rss.bpf.h	Adds RSS, rmap, and lifecycle BPF programs for resident-memory accounting.
crates/memtrack/src/ebpf/tracker.rs	Adds a no-allocator tracker path for RSS and rmap tests.
crates/memtrack/tests/rss_tests.rs	Adds integration coverage for RSS snapshots and rmap reconstruction.

Sequence Diagram

sequenceDiagram
    participant Caller as Tracker startup
    participant Attach as attach_tracepoints()
    participant Life as RSS lifecycle hooks
    participant RSS as rss_stat hook
    Caller->>Attach: start memtrack
    Attach->>Life: attach task_newtask / exec / exit
    Life-->>Attach: attach error
    Attach-->>Caller: startup fails
    Note over Attach,RSS: rss_stat is handled as best-effort, but lifecycle hooks still abort startup

Prompt To Fix All With AI

Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
crates/memtrack/src/ebpf/memtrack/tracking.rs:55-57
**Lifecycle Attaches Abort**

These lifecycle tracepoints still make tracker startup fail before allocator tracking can run. `rss_stat` is now best-effort, but `task_newtask`, `sched_process_exec`, and `sched_process_exit` still propagate attach errors through `attach_tracepoints()`. If a host can attach the existing allocator probes but cannot attach one of these RSS reconciliation tracepoints, `Tracker::new()` returns an error instead of starting memtrack with RSS lifecycle accounting disabled.

_{Reviews (25): Last reviewed commit: "ci: run memtrack bpf tests across the ke..." | Re-trigger Greptile}

not-matthias · 2026-07-20T16:52:22Z

@greptileai pause reviews for now until i enable them again

greptile-apps · 2026-07-20T16:52:41Z

Understood — pausing reviews. I'll hold off until you re-enable them.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

Set AllowShortFunctionsOnASingleLine: None in .clang-format and apply it, reformatting the allocator uprobe macros accordingly.

Sample the kernel's per-mm resident counter through the kmem:rss_stat tracepoint, emitting absolute byte values per mm member. Adds the EVENT_TYPE_RSS contract, MemtrackEventKind::Rss, the parser arm, and a writer bench case. An rss_stat update from reclaim or another process's madvise fires in the actor's context; track (mm_id, member) -> owning pid so those updates reach the owner. External events may only lower a counter, so stale reads and mm_id collisions cannot invent peaks.

Attach fentry hooks on the folio-rmap add/remove functions, emitting signed page-count deltas per MM_* bucket so anon, file, and shmem RSS can be reconstructed over time. Gated behind CODSPEED_MEMTRACK_TRACK_RMAP; the programs stay autoload-off by default so the skeleton loads on any kernel. Adds the EVENT_TYPE_RMAP contract, MemtrackEventKind::Rmap, parser arm, and bench case.

A forked child's inherited RSS is invisible to rss_stat: the fork-time counter copies fire outside the child's context, and anon COW faults are counter-neutral, so a child that only touches inherited memory never reports anything on its own. A fork event carrying the parent pid lets consumers seed the child from the parent's last absolutes; exec and exit mark where the address space is replaced or torn down.

greptile-apps · 2026-07-22T14:55:50Z

+        self.attach_task_newtask()?;
+        self.attach_sched_process_exec()?;
+        self.attach_sched_process_exit()?;


Lifecycle Attach Aborts

These lifecycle tracepoint attaches still return errors from attach_tracepoints(), while rss_stat only logs and continues. Tracker::new() and Tracker::new_without_allocators_with_rmap() call this during startup, so a host that can attach the allocator probes but cannot attach task:task_newtask, sched:sched_process_exec, or sched:sched_process_exit still fails before memory tracking starts. These hooks only support RSS reconciliation, so their attach failures should disable that RSS path instead of taking down the tracker.

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/memtrack/src/ebpf/memtrack/tracking.rs Line: 55-57 Comment: **Lifecycle Attach Aborts** These lifecycle tracepoint attaches still return errors from `attach_tracepoints()`, while `rss_stat` only logs and continues. `Tracker::new()` and `Tracker::new_without_allocators_with_rmap()` call this during startup, so a host that can attach the allocator probes but cannot attach `task:task_newtask`, `sched:sched_process_exec`, or `sched:sched_process_exit` still fails before memory tracking starts. These hooks only support RSS reconciliation, so their attach failures should disable that RSS path instead of taking down the tracker. How can I resolve this? If you propose a fix, please make it concise.

Add /proc-backed integration cases, the rmap late-enable case, and the shared harness helpers used across the rss test suite.

Recover the owning pid for rmap events run by another task (reclaim, process_madvise, khugepaged, KSM) from the mm_struct pointer, and maintain the ownership maps across exec and thread-group exit.

Add the exec_anon and foreign_add cases and extend the rss suite for the mm-ownership attribution and thread-group lifecycle paths.

…ap hooks The batched folio_*_rmap_ptes interface only exists since v6.8; older kernels (e.g. the codspeed-macro runners on 6.5) account the same mapping events through page_add_anon_rmap / page_add_file_rmap / page_remove_rmap. Hook those as an alternative tier. Selection is now per accounting family (anon add-new, anon add, file add, remove) with the newest available API generation winning, since kernels keep transitional wrappers around their successors and hooking two generations double-counts. If any family has no sound hook (pre-6.3, or 6.6/6.7 where file adds flow through folio_add_file_rmap_range whose wrapper inlining is build-dependent), the whole reconstruction is disabled loudly instead of emitting silently drifting partial data. page_remove_rmap also fires for hugetlb teardown, which never touches the MM_*PAGES counters; filtered via VM_HUGETLB to avoid the version-drifting hugetlb folio markers.

Run the rss/bpf tests on arm and codspeed-macro runners over old- and new-layout kernels.

greptile-apps · 2026-07-22T17:13:11Z

+        self.attach_task_newtask()?;
+        self.attach_sched_process_exec()?;
+        self.attach_sched_process_exit()?;


Lifecycle Attaches Abort

These lifecycle tracepoints still make tracker startup fail before allocator tracking can run. rss_stat is now best-effort, but task_newtask, sched_process_exec, and sched_process_exit still propagate attach errors through attach_tracepoints(). If a host can attach the existing allocator probes but cannot attach one of these RSS reconciliation tracepoints, Tracker::new() returns an error instead of starting memtrack with RSS lifecycle accounting disabled.

Prompt To Fix With AI

This is a comment left during a code review. Path: crates/memtrack/src/ebpf/memtrack/tracking.rs Line: 55-57 Comment: **Lifecycle Attaches Abort** These lifecycle tracepoints still make tracker startup fail before allocator tracking can run. `rss_stat` is now best-effort, but `task_newtask`, `sched_process_exec`, and `sched_process_exit` still propagate attach errors through `attach_tracepoints()`. If a host can attach the existing allocator probes but cannot attach one of these RSS reconciliation tracepoints, `Tracker::new()` returns an error instead of starting memtrack with RSS lifecycle accounting disabled. How can I resolve this? If you propose a fix, please make it concise.

greptile-apps Bot reviewed Jul 13, 2026

View reviewed changes

not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from b670b0a to 2f41984 Compare July 13, 2026 17:32

not-matthias marked this pull request as ready for review July 14, 2026 15:38

greptile-apps Bot reviewed Jul 14, 2026

View reviewed changes

Comment thread crates/memtrack/src/ebpf/memtrack.rs Outdated

not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch from 41945a5 to 64688a9 Compare July 17, 2026 14:23

greptile-apps Bot reviewed Jul 17, 2026

View reviewed changes

Comment thread crates/memtrack/src/ebpf/memtrack.rs Outdated

not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from b02f3cb to 411b713 Compare July 17, 2026 18:12

not-matthias changed the base branch from main to cod-1801-only-attach-to-used-libraries-in-memtrack July 20, 2026 09:07

not-matthias force-pushed the cod-1801-only-attach-to-used-libraries-in-memtrack branch from 4d3fbb0 to a7e1666 Compare July 20, 2026 10:11

Base automatically changed from cod-1801-only-attach-to-used-libraries-in-memtrack to main July 20, 2026 13:12

not-matthias force-pushed the cod-3089-collect-rss-in-memtrack branch 2 times, most recently from e00fb6b to a68424a Compare July 20, 2026 16:08

greptile-apps Bot reviewed Jul 20, 2026

View reviewed changes