CredSSP/NLA fails with TLS alert 80 (internal_error) against Windows Server 2008 R2 — NTLM AUTHENTICATE rejected

[GOODBOY008]

Environment

Client

Component	Version
ironrdp	0.17.0
ironrdp-connector	0.10.0
ironrdp-tokio	0.10.0
sspi	0.21.3
picky	7.0.0-rc.25
TLS backend	OpenSSL 3.x (vendored, tokio-openssl)
Client platform	macOS 15.x (aarch64-apple-darwin, Apple Silicon)
Application	r-shell v2.5.0 (Tauri 2 desktop app)

Server

Property	Value
OS Version	Microsoft Windows Server 2008 R2 Enterprise
OS Build	6.1.7601 Service Pack 1 Build 7601
Configuration	Standalone Server (not domain-joined)
Account	Local Administrator (no KDC/Kerberos available)
NLA Required	Yes
RDP Security Layer	Negotiate (HYBRID/SSL)

Server TLS Configuration

• Supported protocols: TLS 1.0, TLS 1.1 only (TLS 1.2 not available on Server 2008 R2 by default)
• Negotiated cipher: AES128-SHA (TLS_RSA_WITH_AES_128_CBC_SHA)
• Certificate: Self-signed, RSA 2048-bit

Symptom

Client completes NTLM exchange through all three stages:

1. NEGOTIATE — sent successfully
2. CHALLENGE — received from server with full target info
3. AUTHENTICATE — sent with nego_tokens + pub_key_auth (32 bytes) + client_nonce (32 bytes), TsRequest version 6

Then the server immediately tears down the TLS session with TLS alert 80 (internal_error) instead of replying with the expected stage-4 pub_key_auth echo.

OpenSSL error:

Error { code: 167773240, library: "SSL routines", function: "ssl3_read_bytes", 
reason: "tlsv1 alert internal error", data: "SSL alert number 80" }

Relevant Logs

INFO connect_begin: Server confirmed connection selected_protocol=SecurityProtocol(HYBRID) 
     flags=ResponseFlags(EXTENDED_CLIENT_DATA_SUPPORTED)
INFO RDP: TLS upgrade succeeded (OpenSSL), cipher = AES128-SHA
INFO RDP: finalizing connection (credssp=true)
INFO connect_finalize:initialize_security_context{protocol="Kerberos"}: 
     sspi::negotiate::client: return=Ok(InitializeSecurityContextResult { 
     status: ContinueNeeded, flags: ClientResponseFlags(0x0), expiry: None })
ERROR RDP connection finalization failed: "SSL alert number 80"
WARN RDP CredSSP/NLA failed, retrying TLS-only
INFO connect_begin: Server confirmed connection selected_protocol=SecurityProtocol(SSL)
INFO RDP: connection finalized successfully (TLS-only)
INFO RDP connected — desktop 1920×1080
... (30 seconds later)
INFO RDP DeactivateAll — running reactivation sequence
ERROR RDP reactivation failed: received disconnect provider ultimatum: UserRequested

Note: The protocol shows "Kerberos" but this is the SPNEGO/Negotiate wrapper — actual auth is NTLM (no KDC available).

What Works (Control)

Client	NLA	Result
Microsoft Remote Desktop (macOS v10.9.x)	✅ Enabled	✅ Connects successfully
FreeRDP (xfreerdp 3.x)	✅ Enabled	✅ Connects successfully
ironrdp 0.17 + sspi 0.21.3	✅ Enabled	❌ TLS alert 80
ironrdp 0.17 + sspi 0.21.3	❌ Disabled (TLS-only)	⚠️ Connects, but server disconnects after ~30s (NLA enforcement)

Client TLS Configuration

// OpenSSL builder configuration
builder.set_min_proto_version(Some(openssl::ssl::SslVersion::TLS1));
builder.set_cipher_list("ALL:@SECLEVEL=0");  // Allow legacy CBC ciphers
builder.set_verify(SslVerifyMode::NONE);     // Accept self-signed cert

Suspected Causes

1. CredSSP version mismatch — Windows Server 2008 R2 may use an older CredSSP/TSRequest format that sspi-rs does not handle correctly
2. pub_key_auth encoding — Server 2008 R2 CredSSP implementation may expect a different ASN.1 encoding for the channel-bound public key
3. NTLMv2 flags — Server 2008 R2 may require specific NTLM negotiate flags that sspi-rs does not set
4. TsRequest version — Server 2008 R2 may require CredSSP version 2-5 instead of version 6

Related Issues

• sspi-rs build(deps): bump bitflags from 2.6.0 to 2.8.0 #651 (closed Apr 2026, but issue persists with sspi 0.21.3)
• sspi-rs Server example: [failed to negotiate security protocol] general error #652 (CredSSP pub_key_auth mismatch with gnome-remote-desktop)
• IronRDP Untangle ironrdp-session from ironrdp-connector and make sspi/NLA an injectable, optional dependency #1422 (CredSSP dependency injection refactor)

Versions Tested

ironrdp	sspi	Result
0.16.0	0.21.0	❌ TLS alert 80
0.17.0	0.21.3	❌ TLS alert 80 (same failure)

Additional Context

• Windows Server 2008 R2 is EOL (end of life Jan 2020) but still widely deployed in legacy environments
• Server requires NLA (cannot be disabled in production environments)
• The server_public_key passed to CredSSP is the full DER-encoded X.509 certificate from the TLS handshake
• Server only supports TLS 1.0/1.1 — cannot test with TLS 1.2+
• No domain controller available — pure NTLMv2 local authentication
• FreeRDP and Microsoft RDC both work with NLA enabled, confirming the issue is specific to ironrdp/sspi-rs CredSSP implementation
• Server 2008 R2 uses an older CredSSP implementation that may differ from newer Windows versions

[TheBestTvarynka]

Interesting. I got the same bug when trying to connect using MsRdpEx and injected credssp.dll (built from sspi-rs). I thought the problem might be with the TLS layer implemented in sspi-rs on top of rustls, but IronRDP uses rustls without any complex wrappers. Even more, I see your setup uses OpenSSL instead of rustls. I am surprised to see this error here too.

My test environment:

• TLS 1.3
• NLA: Kerberos.
• Target machine: Windows 11.

It may be a problem in rustls itself, but I have not confirmed it yet. I am still working on my issue. I will share my results here too

[TheBestTvarynka]

UPD: actually, it is a problem with sspi-rs. In some cases, sspi-rs increments a sequence number unnecessarily, which leads to an invalid sequence number. Somewhat, the Windows RDP server responds with a TLS error instead of a proper CredSSP TsRequest with an error code inside (I do not know why).

In any case, expect a fix soon.

[GOODBOY008]

UPD: actually, it is a problem with sspi-rs. In some cases, sspi-rs increments a sequence number unnecessarily, which leads to an invalid sequence number. Somewhat, the Windows RDP server responds with a TLS error instead of a proper CredSSP TsRequest with an error code inside (I do not know why).

In any case, expect a fix soon.

Pavlo Myroniuk (@TheBestTvarynka) Thanks for debugging this and identifying the root cause. I really appreciate it. Looking forward to the fix.