[Feature]: Investigate Perf Wins From Custom HashTables

[wallstop]

Feature Description

See if we can learn anything from the Java world: https://sooniln.github.io/posts/hashmap-benchmarks-2026/ and replace any of our Dictionary uses with something faster?

Use Case / Motivation

More perf

Proposed Solution

Red/green+perf test implementations

Alternatives Considered

Don't do it

[wallstop]

Measured, on the real dispatch path

Investigated through the Unity MCP loop against the host editor (Unity 6000.4.6f1, Mono in-editor, warmed then one continuous window per variant, 4M lookups per cell).

First, where a hash lookup actually happens per emit

Most of the bus is already dictionary-free. MessageCache<TValue> is a sparse array indexed by a compact per-message-type integer (MessageHelperIndexer<TMessage>), so every type-keyed lookup is array indexing, not hashing. Steady-state dispatch then walks a frozen FlatDispatchEntry<TMessage>[] with plain entries[i].

That leaves exactly one hash lookup on the per-emit path, and only for two of the three kinds:

// MessageBus.TargetedBroadcast, fast lane
Dictionary<InstanceId, HandlerCache<int, HandlerCache>> fastTargeted = plan.contextHandle;
if (fastTargeted != null && fastTargeted.TryGetValue(target, out HandlerCache<int, HandlerCache> fastSorted) ...

SourcedBroadcast has the same shape. UntargetedBroadcast has no hash lookup at all -- it reads plan.scalarHandle directly. So this is a targeted/broadcast question only, and the Java HashMap benchmarks in the linked article are about a structure this codebase has already mostly designed away.

The measurement

InstanceId is a readonly struct holding int _id plus a managed UnityEngine.Object Object, with Equals/GetHashCode reading only _id. Three key shapes, identical identity semantics, 64 targets, strided realistic ids:

key shape	ns / lookup	vs today
InstanceId-shaped struct (int + Object ref) -- today	~23.3	--
struct carrying only the int	~25.2	none (slightly worse)
raw int	~12.3	1.9x faster
open-addressed int table (linear probe, pow2 mask)	~4.9	4.8x faster

Flat across 8, 64 and 1024 targets, so this is per-lookup constant cost, not distribution or collisions.

Root cause, and it is not what the issue assumed

The managed reference in the key is not the cost. A struct with only the int measured the same or marginally worse. The cost is Dictionary<TCustomStruct, V> versus Dictionary<int, V>: a custom struct key resolves through EqualityComparer<T>.Default -> GenericEqualityComparer<T>, whose Equals/GetHashCode are virtual calls Unity Mono does not devirtualize, while a primitive int key hits the specialized path. That is the same hazard already written down in the dispatch-hot-path skill ("Unity Mono has no guarded devirtualization"), showing up in a place nobody had looked: inside Dictionary's comparer rather than in our own call chain.

So the answer to "replace Dictionary with something faster" is no as posed, and yes in a narrower and cheaper form: keep Dictionary, change the key.

For scale

Published throughput is roughly 11-19M emits/sec, i.e. ~50-90 ns per emit. A ~23 ns lookup is a meaningful share of a targeted emit's budget; dropping it to ~12 ns (key change) or ~5 ns (open addressing) is worth having.

Caveat that must not be skipped: these are Mono in-editor numbers, and the published headline is Standalone IL2CPP Release. IL2CPP's generic-sharing and devirtualization behaviour differs, so the ratios must be re-measured on that leg before any of this is claimed as a win. The microbenchmark also isolates the lookup; end-to-end emit gain will be a fraction of these ratios.

Proposed work, and why it is not in #309

Key the context sinks by int (InstanceId.Id) instead of InstanceId:

• Dictionary<InstanceId, HandlerCache<int, HandlerCache>> -> Dictionary<int, HandlerCache<int, HandlerCache>> (59 of the 87 occurrences are this one shape)
• boundary conversion at insert/lookup only; InstanceId stays exactly as it is in the public API

Blast radius, counted rather than guessed:

file	occurrences
Runtime/Core/MessageBus/MessageBus.cs	60
Runtime/Core/MessageHandler.cs	8
Runtime/Core/MessageBus/Internal/BusSlots.cs	6
Runtime/Core/Pooling/DxPools.cs	4
Runtime/Core/Internal/TypedSlots.cs	1

Known hazards to design for, not discovered late:

1. _dirtyTargets (List<InstanceId>) and _dirtyTargetSets (HashSet<InstanceId>) participate in stale-target sweeping alongside these maps and would have to move in step.
2. Sweep and diagnostic paths that enumerate keys currently get a full InstanceId back, whose ToString() includes the Unity object name via .Object. An int key loses that, so any such surface needs the id-to-object resolution moved or the message reworded. (The dispatch path itself is safe: it logs the target parameter, not the key.)
3. MessageBus.ExpectedMessageCacheFieldCount, SweepableTypeCaches, and the memory-reclamation coverage contract all constrain this file.
4. Open addressing (the 4.8x variant) means owning a hashtable, including eviction interaction with Trim and the idle sweep. Worth a second look only if the int-key step measures well on IL2CPP and more is still wanted.

Deliberately not bundled into #309, which is a three-file CI unblock for a workflow that was failing every 25 minutes. An 87-site refactor of the hottest, most invariant-laden file needs the full 9-leg Unity matrix plus the perf-numbers.yml IL2CPP leg to validate, and mixing the two would hold the unblock hostage to perf review.

Next step

Land the int-key change on its own branch, gated on:

• perf-numbers.yml showing a Standalone IL2CPP gain on TargetedBroadcast / SourcedBroadcast and no loss on UntargetedBroadcast
• AllocationMatrixTests unchanged
• the lifecycle edge-case and memory-reclamation suites green across 2021.3 / 2022.3 / 6000.3

[wallstop]

IL2CPP decision: do not ship

PR #312 implemented the narrower primitive-key candidate, preserved the public InstanceId boundary, passed the full correctness/allocation matrix, and measured it twice in Standalone IL2CPP Release.

The final independent run versus the committed baseline was:

Scenario	Throughput delta
Untargeted, one handler	-6.02%
Untargeted, four handlers/four priorities	-11.03%
Targeted, no matching target	+7.03%
Targeted, one listener	+5.55%
Targeted, sixteen listeners	-1.75%
Source-bound broadcast, one handler	-0.05%

The earlier run was also directionally negative for targeted fan-out (-10.86%), source-bound broadcast (-8.90%), and the unaffected untargeted control (-3.25%). Both allocation legs remained exactly zero allocations and zero bytes on the affected rows.

That fails the stated gate: gains on both targeted and source-bound broadcast, no untargeted regression, and unchanged allocations. The primitive-key implementation has therefore been reverted. A custom hash table would add substantially more ownership, reclamation, and AOT risk without evidence that the cheaper intervention helps the published runtime, so this investigation is complete with a do not implement result.

The benchmark reporting gap discovered during this work is tracked separately in #313.