add memory profiling tests for dm broadcasting

[insipx]

Add memprofile crate and a dm_broadcast test that asserts DM broadcasting heap usage stays below 100 MiB using dhat::Alloc

Introduce memory profiling tests for DM broadcasting with a new memprofile crate, set the global allocator to dhat::Alloc, and add a tokio test that checks max_bytes < 100 MiB; adjust xmtp_debug export to always emit [] and raise fd limit to 1024.

📍Where to Start

Start with the dm_broadcast test in dm_broadcast.rs.

---

Macroscope summarized cf0ec21.

[claude]

Claude finished @insipx's task —— View job

---

Code Review Complete ✓

Reviewed on: 2026-01-26 22:28 UTC

---

Summary

This PR introduces a memory profiling infrastructure for testing DM broadcasting memory usage. The implementation is solid with good isolation patterns, though there are a few areas for improvement.

Code Quality & Best Practices

Positive aspects:

• Excellent isolation strategy: Each memory test gets its own integration test file to prevent interference (crates/memprofile/tests/dm_broadcast.rs:2-3)
• Smart design using xdbg as an out-of-process helper to avoid polluting memory measurements (crates/memprofile/tests/dm_broadcast.rs:38-39)
• Good reuse of existing identities to speed up test execution (crates/memprofile/tests/dm_broadcast.rs:44-46)

Areas for improvement:

1. Error handling - pipe resources (crates/memprofile/tests/dm_broadcast.rs:20-34)

   • The get_ids() function doesn't explicitly close pipe file descriptors. While Rust's Drop handles cleanup, the spawned jq process could leak if it fails. Consider using wait() or timeout on the jq process.

2. Inconsistent empty check (crates/xmtp_debug/src/app/export.rs:69-74)

   • Line 69 uses groups.len() == 0 instead of groups.is_empty(). This triggers clippy warnings and is non-idiomatic Rust. The logic also appears redundant since serde_json::to_string already handles empty vecs correctly.

3. Missing error context (crates/memprofile/tests/dm_broadcast.rs:34)

   • The JSON deserialization error lacks context about what failed. Consider adding .wrap_err("failed to parse ethereum addresses from xdbg export") for better debugging.

4. Whitespace-only change (crates/xmtp_mls/Cargo.toml)

   • Removed blank line serves no functional purpose. These cosmetic changes add noise to PRs.

Potential Bugs

1. Race condition risk (crates/memprofile/tests/dm_broadcast.rs:32-34)

   • The test waits for xdbg_child.wait()? but spawns jq without waiting for it. If xdbg finishes before jq completes reading, the pipe could break. Should wait for jq child or handle potential SIGPIPE.

2. Magic number threshold (crates/memprofile/tests/dm_broadcast.rs:104)

   • The 100MB limit is hardcoded without justification. Consider making this configurable or documenting the reasoning (e.g., based on mobile device constraints).

Performance Considerations

1. Serial DM creation (crates/memprofile/tests/dm_broadcast.rs:89-100)

   • Creates and sends 300 DM messages sequentially in a loop. This accurately measures per-message overhead but makes the test slow. Consider documenting the expected runtime or using a smaller default count with env var override.

2. Unnecessary recompilation check (crates/memprofile/tests/dm_broadcast.rs:73-75)

   • Checks for xdbg binary existence and compiles if missing, but doesn't verify the binary is up-to-date. Could lead to testing against stale binaries if source changed.

Security Concerns

No significant security issues identified. The test uses generated test wallets and local processes appropriately.

Test Coverage

Good:

• Addresses the critical DM broadcasting memory usage scenario
• Uses realistic scale (300 watchers)
• Proper use of dhat for heap profiling

Missing:

• No test for memory cleanup after DM operations complete
• No verification that memory usage stays bounded during the loop (only checks peak)
• Consider testing memory behavior with concurrent DM sends vs sequential

Minor Issues

1. crates/xmtp_debug/src/app/generate/identity.rs:36 - The return type uses use<'_> which is unstable syntax. Ensure this is tested on stable Rust or document the requirement.

2. Edition 2024 (crates/memprofile/Cargo.toml:3) - Using edition = "2024" which doesn't exist yet. Should be 2021.

---