Skip to content

Conversation

@Walnut356
Copy link
Contributor

Background

Almost all of the commands in lldb_commands used a regex to associate a type with the synthetic_lookup and summary_lookup python functions. When looking up a type, LLDB iterates through the commands in reverse order (so that new commands can overwrite old ones), stopping when it finds a match. These lookups are cached, but it's a shallow cache (e.g. when Vec<T> is matched by lldb, it will always point to synthetic_lookup, NOT the result of synthetic_lookup which would be StdVecSyntheticProvider).

This becomes a problem because within synthetic_lookup and summary_lookup we run classify_rust_type which checks exact same regexes again. This causes 2 issues:

  1. running the regexes via lldb commands is even more of a waste because the final check is a .* regex that associates with synthetic_lookup anyway
  2. Every time lldb wants to display a value, that value must run the entirety of synthetic_lookup and run its type through 19 regexes + some assorted checks every single time. Those checks take between 1 and 100 microseconds depending on the type.

On a 10,000 element Vec<i32> (which bypasses classify_struct and therefore the 19 regexes), ~30 milliseconds are spent on classify_rust_type. For a 10,000 element Vec<UserDefinedStruct> that jumps up to ~350 milliseconds.

The salt on the wound is that some of those 19 regexes are useless (BTreeMap and BTreeSet which don't even have synthetic/summary providers so it doesn't matter if we know what type it is), and then the results of that lookup function use string-comparisons in a giant if...elif...elif chain.

Solution

To fix all of that, the lldb_commands now point directly to their appropriate synthetic/summary when possible. In cases where there was extra logic, streamlined functions have been added that have much fewer types being passed in, thus only need to do one or two simple checks (e.g. classify_hashmap and classify_hashset).

Some of the lldb_commands regexes were also consolidated to reduce the total number of commands we pass to lldb (e.g. NonZero

An extra upshot is that summary_lookup could be completely removed due to being redundant.

@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 10, 2025
@rustbot
Copy link
Collaborator

rustbot commented Oct 10, 2025

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@Zalathar
Copy link
Member

While I'm not comfortable reviewing this myself (and it might be tricky to find someone who is), I do want to at least say thanks for working on it.

@Walnut356
Copy link
Contributor Author

Walnut356 commented Oct 13, 2025

If it helps the review any, this is more or less how it should have been written from the start. The type synthetic add -l technically expects a python class (-l is short for --python-class), it's sortof a hack that we exploit LLDB not being able to differentiate between an initializer function and a flat function. It's nice because we can check extra bits of the value without callback-based type matching (which was only introduced in lldb 19.0), but I'm not sure why it was written the way it was in cases like Vec<T> that are unambiguous.

The MSVC providers I added a while ago already point directly to the initializer rather than taking a trip through synthetic_lookup and summary_lookup first. This patch just applies that to all of the commands because it's significantly faster.

There shouldn't be any actual changes for end-users (aside from IndirectionSyntheticProvider fixing a minor bug with pointer types).

@Mark-Simulacrum
Copy link
Member

r=me if this is still ready to go (not sure if other changes have landed in the last month that make a rebase make sense). I think we can land this and revert/revisit if it runs into issues, the description makes sense to me and the changes seem reasonable.

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes also look sensible to me. Given that at worst we can revert, let's merge this.

View changes since this review

@jieyouxu
Copy link
Member

@bors r=Mark-Simulacrum,jieyouxu rollup

@bors
Copy link
Collaborator

bors commented Nov 12, 2025

📌 Commit 2e8e618 has been approved by Mark-Simulacrum,jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 12, 2025
@jieyouxu
Copy link
Member

Uh actually @Walnut356, this doesn't need a rebase or anything, right? If not, please r= us
@bors r-
@bors delegate+

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 12, 2025
@bors
Copy link
Collaborator

bors commented Nov 12, 2025

✌️ @Walnut356, you can now approve this pull request!

If @jieyouxu told you to "r=me" after making some further change, please make that change, then do @bors r=@jieyouxu

@Walnut356
Copy link
Contributor Author

@bors r=@jieyouxu

@bors
Copy link
Collaborator

bors commented Nov 15, 2025

📌 Commit 2e8e618 has been approved by jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 15, 2025
@jieyouxu
Copy link
Member

@bors r=Mark-Simulacrum,jieyouxu

@bors
Copy link
Collaborator

bors commented Nov 15, 2025

💡 This pull request was already approved, no need to approve it again.

@bors
Copy link
Collaborator

bors commented Nov 15, 2025

📌 Commit 2e8e618 has been approved by Mark-Simulacrum,jieyouxu

It is now in the queue for this repository.

@bors
Copy link
Collaborator

bors commented Nov 15, 2025

⌛ Testing commit 2e8e618 with merge 3d00472...

bors added a commit that referenced this pull request Nov 15, 2025
[Debugger Visualizers] Optimize lookup behavior

# Background

Almost all of the commands in `lldb_commands` used a regex to associate a type with the `synthetic_lookup` and `summary_lookup` python functions. When looking up a type, LLDB iterates through the commands in reverse order (so that new commands can overwrite old ones), stopping when it finds a match. These lookups are cached, but it's a shallow cache (e.g. when `Vec<T>` is matched by lldb, it will always point to `synthetic_lookup`, NOT the result of `synthetic_lookup` which would be `StdVecSyntheticProvider`).

This becomes a problem because within `synthetic_lookup` and `summary_lookup` we run `classify_rust_type` which checks exact same regexes again. This causes 2 issues:

1. running the regexes via lldb commands is even more of a waste because the final check is a `.*` regex that associates with `synthetic_lookup` anyway
2. Every time lldb wants to display a value, that value must run the entirety of `synthetic_lookup` and run its type through 19 regexes + some assorted checks every single time. Those checks take between 1 and 100 microseconds depending on the type.

On a 10,000 element `Vec<i32>` (which bypasses `classify_struct` and therefore the 19 regexes), ~30 milliseconds are spent on `classify_rust_type`. For a 10,000 element `Vec<UserDefinedStruct>` that jumps up to ~350 milliseconds.

The salt on the wound is that some of those 19 regexes are useless (`BTreeMap` and `BTreeSet` which don't even have synthetic/summary providers so it doesn't matter if we know what type it is), and then the results of that lookup function use string-comparisons in a giant `if...elif...elif` chain.

# Solution

To fix all of that, the `lldb_commands` now point directly to their appropriate synthetic/summary when possible. In cases where there was extra logic, streamlined functions have been added that have much fewer types being passed in, thus only need to do one or two  simple checks (e.g. `classify_hashmap` and `classify_hashset`).

Some of the `lldb_commands` regexes were also consolidated to reduce the total number of commands we pass to lldb (e.g. `NonZero`

An extra upshot is that `summary_lookup` could be completely removed due to being redundant.
@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Collaborator

bors commented Nov 15, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Nov 15, 2025
@Walnut356
Copy link
Contributor Author

That is_msvc check needed to be changed anyway because SBProcess.GetTriple doesn't work quite how we need it to, but it's pretty crazy that it can return None lol

@rustbot
Copy link
Collaborator

rustbot commented Nov 15, 2025

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@rustbot rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Nov 15, 2025
@Zalathar
Copy link
Member

@bors try jobs=aarch64-apple

rust-bors bot added a commit that referenced this pull request Nov 15, 2025
[Debugger Visualizers] Optimize lookup behavior

try-job: aarch64-apple
@rust-bors

This comment has been minimized.

@rust-bors
Copy link

rust-bors bot commented Nov 15, 2025

☀️ Try build successful (CI)
Build commit: c4dc492 (c4dc4926c5197ff47b02db345958f28abcc6afe6, parent: 733108b6d4acaa93fe26ae281ea305aacd6aac4e)

@Mark-Simulacrum Mark-Simulacrum added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 15, 2025
@bors
Copy link
Collaborator

bors commented Nov 20, 2025

☔ The latest upstream changes (presumably #89917) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants