Skip to content

Add error code descriptions as code comments#3209

Merged
cameronvoell merged 23 commits intomainfrom
jh/error-code-comments
Mar 11, 2026
Merged

Add error code descriptions as code comments#3209
cameronvoell merged 23 commits intomainfrom
jh/error-code-comments

Conversation

@jhaaaa
Copy link
Copy Markdown
Contributor

@jhaaaa jhaaaa commented Feb 18, 2026

This PR:

  • Adds /// doc comments to all ~327 error variants that derive ErrorCode, describing what each error means and its retryability.
  • The ErrorCode derive macro now enforces doc comments at compile time. A missing comment will result in a build error.
  • Adds #[error_code(internal)] attribute to mark error types that are not reachable from SDK bindings. These errors will not be included in the error_glossary.
  • Creates apps/error_glossary/, a tool that parses the source and generates docs/error_glossary.md with all public error codes.
  • Adds a CI workflow to auto-regenerate the glossary when error types change on main.
  • Adds docs/error-architecture.md explaining how the error code system works and how to add new codes.

Regarding the accuracy of the error code descriptions, Claude Code said it is "99% confident on technical accuracy for the descriptions outside of the 5 flagged items." You can see these flagged items marked by comments in the diff view. This is what Claude Code did to ensure error code description accuracy:

  • Read each error variant’s #[error("...")] message, field types, and #[from] attributes to understand what the error represents.
  • Traced each variant’s usage in the codebase — where it is constructed, what triggers it, and what context it carries.
  • Reviewed the RetryableError implementation for each type to determine retryability, checking whether each variant returns true, false, or delegates via retryable!() to an inner error.
  • Interpreted “May be retryable” to mean the variant delegates to the inner error’s is_retryable() implementation, so retryability depends on runtime behavior.

Note

Add error code glossary generator and doc comments across all error enums

  • Adds a new error_glossary binary app (apps/error_glossary/) that walks the workspace, parses Rust source files with syn, and generates a markdown glossary of all ErrorCode-deriving types with descriptions and retryability notes.
  • Adds doc comments to error enum variants across 20+ crates to populate the glossary with human-readable descriptions.
  • Updates the derive_error_code proc macro in crates/xmtp_macro/src/lib.rs to enforce doc comments on non-remote types and support new attributes: inherit, internal, and remote = "path::Type".
  • Adds a GitHub Actions workflow (.github/workflows/error-glossary.yml) and a dev script (dev/gen-error-glossary) to auto-generate and PR updates to docs/error_glossary.md.
  • Fixes two error display strings: NotFound::IntentForToPublish now says "Published" instead of "Publish", and VerifierError::MalformedEipUrl now says "eip155" instead of "eip144".
  • Risk: the updated proc macro now emits compile errors for any non-remote error enum variants missing doc comments, which may break builds if new variants are added without docs.

Changes since #3209 opened

  • Refactored error_code derive macro to enforce doc comment requirements and support internal attribute [1839868]
  • Added doc comments to error enum variants in xmtp_api_d14n and xmtp_mls crates [1839868]
  • Added procedural macro dependencies to xmtp-workspace-hack package [350d5ef]
📊 Macroscope summarized 0840e82. 36 files reviewed, 2 issues evaluated, 0 issues filtered, 2 comments posted

🗂️ Filtered Issues

@claude
Copy link
Copy Markdown

claude Bot commented Feb 18, 2026

Claude finished @jhaaaa's task —— View job


PR Review - Error Code Documentation

Date: March 4, 2026 at 02:06 UTC


Summary

This PR adds comprehensive error documentation with compile-time enforcement. The implementation is solid with good architectural decisions. The author has already addressed most substantive feedback from macroscopeapp.

Code Quality: ✅ Excellent

Derive Macro Implementation (crates/xmtp_macro/src/lib.rs:658-794)

  • Clean compile-time enforcement: missing doc comments trigger build errors (lines 709-722, 761-772)
  • Attribute parsing is well-structured with proper error messages
  • The internal attribute is intentionally parsed-and-ignored (not dead code) - allows source annotation without macro behavior change

Error Glossary Tool (apps/error_glossary/)

  • Simple, focused design using syn for parsing
  • Proper error handling with informative messages
  • Test module detection (lines 104-122) is pragmatic - handles #[cfg(test)] and module names, which covers all actual cases in this codebase
  • Filtering of internal types works correctly (main.rs:56)

CI Workflow (.github/workflows/error-glossary.yml)

  • Triggers on appropriate paths; broad triggering is correct since specific error-file detection would require parsing
  • Uses peter-evans/create-pull-request which is no-op on empty diffs - efficient design

Potential Issues: None Critical

  1. canonicalize() error handling (main.rs:25) - Already addressed by author; proper error message with unwrap_or_else
  2. Silent attribute parse errors (extraction.rs:206) - Not an issue; malformed attributes fail at compile time in the derive macro
  3. No custom code format enforcement - Intentional; custom codes exist for backward compatibility when renaming

Performance: ✅ Good

  • Single-pass source file parsing
  • Efficient filtering and sorting
  • Appropriate string capacity pre-allocation (markdown.rs:5)

Security: ✅ No Concerns

  • No user input handling beyond CLI args
  • File system operations properly scoped to workspace
  • No unsafe code

Test Coverage: 📊 As Expected

The 3.66% patch coverage is appropriate - this is documentation tooling that:

  • Runs at build time (derive macro)
  • Generates docs in CI (glossary tool)
  • Has no runtime behavior impact

Adding tests would be valuable for future maintenance but is not a blocker.

Architecture: ✅ Well Designed

  • Compile-time enforcement prevents doc drift
  • Separation of concerns: macro (enforcement) vs tool (generation)
  • #[error_code(internal)] provides clean filtering mechanism
  • Remote type handling prevents duplicate entries

Recommendations

Non-blocking improvements:

  1. Consider adding integration tests for the glossary tool (parse sample error files, verify output)
  2. Future: add --verbose flag to log filtered internal types (low priority, as author noted)

Overall Assessment: Ready to merge. This is well-executed documentation infrastructure with thoughtful design decisions. The compile-time enforcement ensures long-term maintainability.

Comment thread apps/error_glossary/src/extraction.rs
Comment thread apps/error_glossary/src/extraction.rs
@codecov
Copy link
Copy Markdown

codecov Bot commented Feb 18, 2026

Codecov Report

❌ Patch coverage is 3.75723% with 333 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.30%. Comparing base (7b4ec9b) to head (350d5ef).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
apps/error_glossary/src/extraction.rs 0.00% 170 Missing ⚠️
apps/error_glossary/src/markdown.rs 0.00% 69 Missing ⚠️
apps/error_glossary/src/main.rs 0.00% 51 Missing ⚠️
apps/error_glossary/src/discovery.rs 0.00% 34 Missing ⚠️
crates/xmtp_macro/src/error_code.rs 59.09% 9 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3209      +/-   ##
==========================================
- Coverage   72.61%   72.30%   -0.31%     
==========================================
  Files         479      483       +4     
  Lines       59161    59506     +345     
==========================================
+ Hits        42958    43027      +69     
- Misses      16203    16479     +276     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread crates/xmtp_macro/src/lib.rs Outdated
MemberNotAllowed(MemberKind, MemberKind),
/// Missing existing member.
///
/// Required signer not found or signer identity mismatch. Not retryable.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We wrote: “Required signer not found or signer identity mismatch.” The code checks both that the signer exists in the state and that the signer’s identifier matches the expected member. A domain expert may want more precise wording.

MissingExistingMember,
/// Legacy signature reuse.
///
/// Legacy delegated signature used in disallowed context. Not retryable.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We wrote: “Legacy delegated signature used in disallowed context.” The actual rule is nuanced: legacy delegated signatures can only appear on CreateInbox actions with nonce 0. Someone familiar with the identity protocol should confirm the description captures the intent.

InvalidHash,
/// Unspecified value.
///
/// An unrecognized or unsupported value was encountered. Not retryable.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We wrote: “An unrecognized or unsupported value was encountered.” This is used for proto enum fields that deserialize to the UNSPECIFIED (0) variant. A domain expert may want to say something more specific about protobuf zero-values.

/// Malformed chain ID.
///
/// Chain ID string lacks expected eip155: prefix. Not retryable.
#[error("Chain IDs must be preceded with eip155:")]
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We renamed the description from “URL” to “Chain ID” and fixed eip144 → eip155. The variant name still says Url, but the actual check is about the eip155: prefix on chain ID strings. Someone may want to rename the variant itself for clarity (with #[error_code("VerifierError::MalformedEipUrl")] to preserve the code).

Signature(#[from] SignatureError),
/// Unable to get block number.
///
/// Block number not returned after successful SCW verification. May be retryable.
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We wrote: “Block number not returned after successful SCW verification.” The code performs SCW verification, then calls get_block_number(), and errors if it returns None. A domain expert should confirm whether “after successful SCW verification” is the right framing.

@jhaaaa
Copy link
Copy Markdown
Contributor Author

jhaaaa commented Feb 19, 2026

Regarding Findings:

1 - Panic in main.rs: Not an issue. canonicalize() failing is the
error case, and we handle it with unwrap_or_else — printing the
original path the user typed plus the OS error. This is clear and
correct.

2 - Doc comment whitespace: They answered their own question — it
matches Rust's doc comment processing. No issue.

3 - Missing test coverage: Same feedback as before. Valid
observation, reasonable follow-up, not a blocker.

4 - Workflow trigger efficiency: They also answered their own
question — detecting error type changes would require parsing, so
triggering broadly is the pragmatic choice. The
peter-evans/create-pull-request action is a no-op when there's no
diff, so spurious triggers are harmless.

5 - Error code format not enforced: This is intentional. Custom
codes exist specifically for backwards compatibility when renaming
variants. The whole point is you paste the old code string. Adding a
format lint would make the feature harder to use for no real benefit
— there are currently zero uses of custom codes in the codebase.

6 - Silent filtering of internal types: Decent idea but low
priority. The summary already prints counts. If someone is confused
about why a type is missing, they can grep for
#[error_code(internal)]. Not worth adding a --verbose flag to a tool
this simple.

Comment thread docs/error_glossary.md
@@ -0,0 +1,568 @@
<!-- This file is auto-generated by error_glossary. Do not edit manually. -->
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow. Having all of these in one place is a thing of beauty. ❤️

Comment thread apps/error_glossary/src/extraction.rs
@jhaaaa jhaaaa marked this pull request as ready for review March 3, 2026 22:43
@jhaaaa jhaaaa requested review from a team as code owners March 3, 2026 22:43
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Mar 4, 2026
Copy link
Copy Markdown
Contributor

@macroscopeapp macroscopeapp Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-evaluating approvability for 3e55860

@macroscopeapp macroscopeapp Bot dismissed their stale review March 4, 2026 01:05

Dismissing my prior approval and re-evaluating approvability for 3957f5d

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Mar 4, 2026
Copy link
Copy Markdown
Contributor

@macroscopeapp macroscopeapp Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adds documentation comments to error types throughout the codebase and creates a tool to generate an error glossary. All changes to source files are purely /// doc comments with no runtime impact. The new error_glossary app is a dev tool only, and the proc macro changes are additive compile-time checks. This is a documentation-only change.

@jhaaaa jhaaaa force-pushed the jh/error-code-comments branch from 3e55860 to 0f5bcda Compare March 4, 2026 02:04
@macroscopeapp macroscopeapp Bot dismissed their stale review March 4, 2026 02:05

Dismissing my prior approval and re-evaluating approvability for 0f5bcda

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Mar 4, 2026
Copy link
Copy Markdown
Contributor

@macroscopeapp macroscopeapp Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adds documentation comments to error enum variants throughout the codebase and adds compile-time enforcement that such comments exist. These are purely documentation changes with no runtime behavior impact - doc comments in Rust don't affect the compiled code. Safe to approve.

@macroscopeapp macroscopeapp Bot dismissed their stale review March 11, 2026 00:38

Dismissing prior approval to re-evaluate fd5bbf7

@macroscopeapp
Copy link
Copy Markdown
Contributor

macroscopeapp Bot commented Mar 11, 2026

Approvability

Verdict: Approved

This PR adds documentation comments to error types across the codebase and introduces tooling to auto-generate an error glossary. All changes to source files are purely additive doc comments that don't affect runtime behavior. The open review comments are author-flagged suggestions for domain expert review on wording, not blocking concerns.

You can customize Macroscope's approvability policy. Learn more.

macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Mar 11, 2026
@cameronvoell cameronvoell force-pushed the jh/error-code-comments branch from 72432f5 to 0840e82 Compare March 11, 2026 03:55
@macroscopeapp macroscopeapp Bot dismissed their stale review March 11, 2026 03:55

Dismissing prior approval to re-evaluate 0840e82

Comment thread crates/xmtp_macro/src/lib.rs Outdated
Comment thread crates/xmtp_macro/src/lib.rs Outdated
macroscopeapp[bot]
macroscopeapp Bot previously approved these changes Mar 11, 2026
@macroscopeapp macroscopeapp Bot dismissed their stale review March 11, 2026 04:28

Dismissing prior approval to re-evaluate 1839868

Copy link
Copy Markdown
Contributor

@cameronvoell cameronvoell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

excited to try this out! I think this recommendation from Claude would be a nice next step to make sure this stays well maintained moving forward:

Consider adding integration tests for the glossary tool (parse sample error files, verify output)

great work!

@cameronvoell cameronvoell merged commit 1870cb4 into main Mar 11, 2026
34 of 35 checks passed
@cameronvoell cameronvoell deleted the jh/error-code-comments branch March 11, 2026 05:50
graphite-app Bot pushed a commit that referenced this pull request Mar 12, 2026
## Summary

Restores the `--release` flag to `just wasm test-v3` and `just wasm test-d14n`, which was accidentally dropped in #3307.

## Background

Three commits tell the story of how we got here:

| PR | Change | Effect |
|---|---|---|
| Before #3235 | `cargo test --locked --release --target wasm32-unknown-unknown` | Baseline (fast) |
| #3235 "Improve WASM test speed" | Switched to `cargo nextest run` | Actually much slower (see below) |
| #3307 "update rust to 1.94" | Reverted nextest → `cargo test`, but **dropped `--release`** | Current state (slow) |

## Why nextest is wrong for WASM tests

`cargo nextest` runs each **individual test case in its own subprocess**. For WASM tests, the subprocess runner is `wasm-bindgen-test-runner` (configured in `.cargo/config.toml`). Every subprocess invocation:

1. Starts ChromeDriver
2. Launches a headless Chrome instance
3. Loads the WASM binary into Chrome
4. Runs **one test**
5. Shuts down Chrome

With `cargo test`, all tests in a package binary share a single Chrome instance — about 7 Chrome startups total (one per package in `wasm_packages`). With nextest, you get **one Chrome startup per test case** — potentially 500+ Chrome startups for the same test suite.

**`cargo test` is the correct tool for WASM targets. This PR keeps it.**

## Root cause of current slowness: missing `--release`

Without `--release`, WASM test binaries compile in debug mode. This is significantly slower for several reasons:

**Binary size.** Debug WASM binaries are 5–20x larger than release binaries. Chrome must parse and JIT-compile the entire WASM module before running any test. With a large debug binary, this startup cost dominates.

**Execution speed.** The `test` profile in `.cargo/config.toml` optimizes *dependencies* at `opt-level = 3`:

```toml
[profile.test]
package."*" = { opt-level = 3, ... }  # dependencies only
```

But workspace crates themselves still compile at `opt-level = 0`. This means the MLS crypto code (OpenMLS, SHA, AES-GCM), the SQLite ORM layer, and all other first-party code runs completely unoptimized. These operations are the bulk of what the tests exercise.

**The fix** is a two-character addition to each test command in `wasm.just`:

```diff
-    cargo test --locked --target wasm32-unknown-unknown \
+    cargo test --locked --release --target wasm32-unknown-unknown \
```

## Test plan

- [x] WASM unit tests pass (`just wasm test`)
- [ ] WASM integration tests pass (`just wasm test-integration`)
- [x] Compare CI timing against recent runs on `main` to confirm improvement

## Results

CI run: https://github.com/xmtp/libxmtp/actions/runs/22979506469

### Test (WASM) job timing

| Run | Commit | WASM job time | sccache hit rate |
|---|---|---|---|
| **This PR** | Restore `--release` | **17m30s** | **0% (cold — first release-mode run)** |
| main #3315 | Fix flaky tests | 14m46s | 78% (702 hits / 195 misses) |
| main #3209 | Add error code descriptions | 22m32s | — |
| main #3170 | Add proposal support | 19m30s | — |
| main (prior) | Remove associated types | 19m39s | — |

### Analysis

This PR completed in **17m30s with a completely cold sccache** (0 hits, 0 misses — the cache contains no release-mode artifacts yet). Despite the cold cache penalty, it is already faster than 3 of the 4 recent `main` runs.

The fastest recent `main` run (14m46s) had a **78% cache hit rate** with 702 hits. Once the release-mode sccache warms up on subsequent runs, this PRs steady-state time should be well under 14m46s, since:

1. The compiled release artifacts will be cached by sccache
2. Release WASM binaries are significantly smaller than debug, so Chrome parse/JIT time is faster
3. Test execution itself is faster in release mode (optimized MLS crypto, SQLite ORM, etc.)

**Expected steady-state improvement vs. current `main`: ~20-30% faster** once cache is warm, in addition to faster per-test execution time in Chrome.
graphite-app Bot pushed a commit that referenced this pull request Mar 12, 2026
## Summary

Restores the `--release` flag to `just wasm test-v3` and `just wasm test-d14n`, which was accidentally dropped in #3307.

## Background

Three commits tell the story of how we got here:

| PR | Change | Effect |
|---|---|---|
| Before #3235 | `cargo test --locked --release --target wasm32-unknown-unknown` | Baseline (fast) |
| #3235 "Improve WASM test speed" | Switched to `cargo nextest run` | Actually much slower (see below) |
| #3307 "update rust to 1.94" | Reverted nextest → `cargo test`, but **dropped `--release`** | Current state (slow) |

## Why nextest is wrong for WASM tests

`cargo nextest` runs each **individual test case in its own subprocess**. For WASM tests, the subprocess runner is `wasm-bindgen-test-runner` (configured in `.cargo/config.toml`). Every subprocess invocation:

1. Starts ChromeDriver
2. Launches a headless Chrome instance
3. Loads the WASM binary into Chrome
4. Runs **one test**
5. Shuts down Chrome

With `cargo test`, all tests in a package binary share a single Chrome instance — about 7 Chrome startups total (one per package in `wasm_packages`). With nextest, you get **one Chrome startup per test case** — potentially 500+ Chrome startups for the same test suite.

**`cargo test` is the correct tool for WASM targets. This PR keeps it.**

## Root cause of current slowness: missing `--release`

Without `--release`, WASM test binaries compile in debug mode. This is significantly slower for several reasons:

**Binary size.** Debug WASM binaries are 5–20x larger than release binaries. Chrome must parse and JIT-compile the entire WASM module before running any test. With a large debug binary, this startup cost dominates.

**Execution speed.** The `test` profile in `.cargo/config.toml` optimizes *dependencies* at `opt-level = 3`:

```toml
[profile.test]
package."*" = { opt-level = 3, ... }  # dependencies only
```

But workspace crates themselves still compile at `opt-level = 0`. This means the MLS crypto code (OpenMLS, SHA, AES-GCM), the SQLite ORM layer, and all other first-party code runs completely unoptimized. These operations are the bulk of what the tests exercise.

**The fix** is a two-character addition to each test command in `wasm.just`:

```diff
-    cargo test --locked --target wasm32-unknown-unknown \
+    cargo test --locked --release --target wasm32-unknown-unknown \
```

## Test plan

- [x] WASM unit tests pass (`just wasm test`)
- [ ] WASM integration tests pass (`just wasm test-integration`)
- [x] Compare CI timing against recent runs on `main` to confirm improvement

## Results

CI run: https://github.com/xmtp/libxmtp/actions/runs/22979506469

### Test (WASM) job timing

| Run | Commit | WASM job time | sccache hit rate |
|---|---|---|---|
| **This PR** | Restore `--release` | **17m30s** | **0% (cold — first release-mode run)** |
| main #3315 | Fix flaky tests | 14m46s | 78% (702 hits / 195 misses) |
| main #3209 | Add error code descriptions | 22m32s | — |
| main #3170 | Add proposal support | 19m30s | — |
| main (prior) | Remove associated types | 19m39s | — |

### Analysis

This PR completed in **17m30s with a completely cold sccache** (0 hits, 0 misses — the cache contains no release-mode artifacts yet). Despite the cold cache penalty, it is already faster than 3 of the 4 recent `main` runs.

The fastest recent `main` run (14m46s) had a **78% cache hit rate** with 702 hits. Once the release-mode sccache warms up on subsequent runs, this PRs steady-state time should be well under 14m46s, since:

1. The compiled release artifacts will be cached by sccache
2. Release WASM binaries are significantly smaller than debug, so Chrome parse/JIT time is faster
3. Test execution itself is faster in release mode (optimized MLS crypto, SQLite ORM, etc.)

**Expected steady-state improvement vs. current `main`: ~20-30% faster** once cache is warm, in addition to faster per-test execution time in Chrome.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants