Skip to content

Isolate RD Gateway RPC bindings for in-process ActiveX connections - #188

Open
StefanKoell wants to merge 1 commit into
Devolutions:masterfrom
StefanKoell:gateway-isolation
Open

StefanKoell wants to merge 1 commit into
Devolutions:masterfrom
StefanKoell:gateway-isolation

Conversation

@StefanKoell

Copy link
Copy Markdown
Contributor

Problem

A second Microsoft RDP ActiveX connection through an RD Gateway can fail
while the first connection remains open in the same process.

Solution

Prevent RPC connection reuse by applying RPC_C_OPT_UNIQUE_BINDING
after successful authentication setup. The change is limited to
ncacn_http bindings originating from mstscax.dll or rdclientax.dll.

Isolation is applied once per binding lifetime. Repeated authentication
setup does not override subsequent application option changes. Native
return values and last-error state are preserved.

Binding tracking accounts for handle reuse and failed frees. The hooks
follow the normal attach/detach lifecycle, and the DLL remains unloadable.
Successful detach clears tracking; failed detach preserves it. During
process termination, Windows reclaims tracking records without acquiring
locks that may be owned by terminated threads.

Isolation operates independently of logging. Its only log output is a
warning containing the status when isolation cannot be applied.

Configuration and compatibility

Isolation is enabled by default. Hosts can disable it before opening
connections:

var core = new MsRdpEx.RdpCoreApi();
core.GatewayIsolationEnabled = false;

The property is thread-safe and shared by all core API objects in the
process. Changes affect previously untracked bindings at their first
successful authentication setup. Existing bindings retain their behavior,
including bindings first observed while isolation was disabled.

Native hosts access the setting through the new
IMsRdpExGatewaySettings COM extension. The existing
IMsRdpExCoreApi IID, signatures and vtable remain unchanged.

Both managed interop implementations support the property. Accessing it
with an older native DLL throws NotSupportedException, while existing
API usage remains compatible.

MSRDPEX_GATEWAY_UNIQUE_BINDING provides an optional startup override,
read once per DLL load. Subsequent property assignments take precedence.

Validation

  • Native Release builds succeeded for x64, x86 and ARM64.
  • All 30 native tests passed on each architecture.
  • Managed builds succeeded for net48 and net8.0-windows.
  • Six managed compatibility tests passed, including older-DLL behavior.
  • Regression coverage includes binding scope, repeated authentication,
    property changes, native error preservation, handle reuse, failed frees,
    hook transactions, COM identity and DLL unloading.
  • Shutdown coverage verifies recording finalization when a terminated
    worker owns the gateway binding lock.

Customer validation of the final build against the two-connection
scenario was successful.

Reference: RPC binding options.

Fix a connection failure where the first Microsoft RDP ActiveX session
through an RD Gateway succeeds but a second session in the same process
fails.

Apply RPC_C_OPT_UNIQUE_BINDING after successful authentication setup for
ncacn_http bindings originating from mstscax.dll or rdclientax.dll. Apply
the option once per binding lifetime, preserve native return values and
last-error state, and respect subsequent application option changes.

Enable isolation by default and expose a thread-safe, process-wide
RdpCoreApi.GatewayIsolationEnabled property. Provide native access through
a separate IMsRdpExGatewaySettings COM interface, preserving the existing
IMsRdpExCoreApi IID and vtable.

Support the property in both managed interop implementations. Access
against an older native DLL throws NotSupportedException. Retain
MSRDPEX_GATEWAY_UNIQUE_BINDING as an optional startup override, with
subsequent property assignments taking precedence.

Track bindings even when isolation is disabled. Handle binding-address
reuse and failed frees, and clear tracking after successful hook detach.
Skip tracking cleanup during process termination to avoid locks owned
by terminated threads.

Add regression coverage for isolation behavior, configuration, COM
compatibility, hook lifecycle and DLL unloading.

Validation:
- Native Release builds and 30 tests passed on x64, x86 and ARM64.
- Both managed target frameworks built successfully.
- Six managed compatibility tests passed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant