openhcl/tdx: inject machine check on ept exits caused by bad guest addresses #1340

chris-oo · 2025-05-12T19:05:03Z

On TDX, we see some cases where the guest attempts to access an address with an incorrect shared bit. It's unclear if this is an issue in OpenHCL, the guest, or the host, but fix OpenHCL crashing with an emulation failure due to a GuestMemory access failure, and instead inject a machine check into the guest.

For addresses outside of mmio and ram, continue to emulate but log that the guest did something strange. In the future, we may also inject a machine check on that path.

Tested via a uefi app that attempts to access a shared page at a private gpa (see https://github.com/chris-oo/openvmm/blob/uefi-tmk-write-to-shared/tmk/tmk_launch/src/main.rs) with the following crash:

[kmsg]: [3.058450] virt_mshv_vtl::processor::tdx: WARN  guest accessed inaccessible gpa, injecting MC gpa=0x666d9000 is_shared=false
[kmsg]: [3.061137] virt_mshv_vtl::processor: WARN  Guest has reported system crash crash=VtlCrash { vp_index: VpIndex(0), last_vtl: Vtl0, control: GuestCrashCtl { pre_os_id: 0, no_crash_dump: false, crash_message: true, crash_notify: true }, parameters: [12, 0, 0, 6790a820, 4d7] }
[kmsg]: [3.061758] virt_mshv_vtl::processor:
WARN  Guest has reported a system crash message "!!!! X64 Exception Type - 12(#MC - Machine-Check)  CPU Apic ID - 00000000 !!!!<5c>r<5c>nRIP  - 00000000666DB030, CS  - 0000000000000038, RFLAGS - 0000000000010202<5c>r<5c>nRAX  - 00000000666D9000, RCX - 0000000000000042, RDX - 3333333333333333<5c>r<5c>nRBX  - 0000000066E00018, RSP - 0000000033D99150, RBP - 0000000033D99190<5c>r<5c>nRSI  - 0000000066E54718, RDI - 0000000033DB8160<5c>r<5c>nR8   - 0000000000000000, R9  - 00000000666ED508, R10 - 00000000666ED5EE<5c>r<5c>nR11  - 000000000000000A, R12 - 0000000000000000, R13 - 0000000000000000<5c>r<5c>nR14  - 0000000033D99AA0, R15 - 0000000033D99A98<5c>r<5c>nDS   - 0000000000000030, ES  - 0000000000000030, FS  - 0000000000000030<5c>r<5c>nGS   - 0000000000000030, SS  - 0000000000000030<5c>r<5c>nCR0  - 0000000080010073, CR2 - 0000000000000000, CR3 - 0000000033801000<5c>r<5c>nCR4  - 0000000000000668, CR8 - 0000000000000000<5c>r<5c>nDR0  - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000<5c>r<5c>nDR3  -

#1426 tracks implementing this for SNP.

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs

vmm_core/virt_support_x86emu/src/emulate.rs

jstarks · 2025-05-28T17:03:46Z

openhcl/virt_mshv_vtl/src/processor/snp/mod.rs

@@ -1370,6 +1371,11 @@ impl UhProcessor<'_, SnpBacked> {
            }

            SevExitCode::NPF if has_intercept => {
+                // TODO SNP: This code needs to be fixed to not rely on the


Are you sure? I thought Alex said this was necessary. Doesn't the legacy HCL do this, too?

I talked with Jon offline and he said we shouldn't be relying on the hypervisor for anything - the hardware exit info on both SNP and TDX should have what we need. I haven't looked at the legacy HCL in detail but he said that's what he believes to be the case after some refactoring.

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs

vmm_core/virt_support_x86emu/src/emulate.rs

jstarks · 2025-05-28T22:03:10Z

vmm_core/virt_support_x86emu/src/emulate.rs

@@ -1014,6 +1041,11 @@ fn gpf_event() -> hvdef::HvX64PendingEvent {
    make_exception_event(Exception::GENERAL_PROTECTION_FAULT, Some(0), None)
 }

+/// Generates a machine check pending event
+fn mc_event() -> hvdef::HvX64PendingEvent {
+    make_exception_event(Exception::MACHINE_CHECK, None, None)


Do we need to make sure cr4.MCE is 1?

Also, are there any machine-check MSRs the guest is going to read that we need to populate so that the guest crash dump is useful?

IIRC in TDX MCE is always set to 1, although i think we do shadow the guest disabling it, we don't actaully allow the guest to disable the hardware TD state of MCE

…dresses (microsoft#1340) On TDX, we see some cases where the guest attempts to access an address with an incorrect shared bit. It's unclear if this is an issue in OpenHCL, the guest, or the host, but fix OpenHCL crashing with an emulation failure due to a `GuestMemory` access failure, and instead inject a machine check into the guest. For addresses outside of mmio and ram, continue to emulate but log that the guest did something strange. In the future, we may also inject a machine check on that path. Tested via a uefi app that attempts to access a shared page at a private gpa (see https://github.com/chris-oo/openvmm/blob/uefi-tmk-write-to-shared/tmk/tmk_launch/src/main.rs) with the following crash: ``` [kmsg]: [3.058450] virt_mshv_vtl::processor::tdx: WARN guest accessed inaccessible gpa, injecting MC gpa=0x666d9000 is_shared=false [kmsg]: [3.061137] virt_mshv_vtl::processor: WARN Guest has reported system crash crash=VtlCrash { vp_index: VpIndex(0), last_vtl: Vtl0, control: GuestCrashCtl { pre_os_id: 0, no_crash_dump: false, crash_message: true, crash_notify: true }, parameters: [12, 0, 0, 6790a820, 4d7] } [kmsg]: [3.061758] virt_mshv_vtl::processor: WARN Guest has reported a system crash message "!!!! X64 Exception Type - 12(#MC - Machine-Check) CPU Apic ID - 00000000 !!!!<5c>r<5c>nRIP - 00000000666DB030, CS - 0000000000000038, RFLAGS - 0000000000010202<5c>r<5c>nRAX - 00000000666D9000, RCX - 0000000000000042, RDX - 3333333333333333<5c>r<5c>nRBX - 0000000066E00018, RSP - 0000000033D99150, RBP - 0000000033D99190<5c>r<5c>nRSI - 0000000066E54718, RDI - 0000000033DB8160<5c>r<5c>nR8 - 0000000000000000, R9 - 00000000666ED508, R10 - 00000000666ED5EE<5c>r<5c>nR11 - 000000000000000A, R12 - 0000000000000000, R13 - 0000000000000000<5c>r<5c>nR14 - 0000000033D99AA0, R15 - 0000000033D99A98<5c>r<5c>nDS - 0000000000000030, ES - 0000000000000030, FS - 0000000000000030<5c>r<5c>nGS - 0000000000000030, SS - 0000000000000030<5c>r<5c>nCR0 - 0000000080010073, CR2 - 0000000000000000, CR3 - 0000000033801000<5c>r<5c>nCR4 - 0000000000000668, CR8 - 0000000000000000<5c>r<5c>nDR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000<5c>r<5c>nDR3 - ``` microsoft#1426 tracks implementing this for SNP.

…dresses (#1340) (#1434) On TDX, we see some cases where the guest attempts to access an address with an incorrect shared bit. It's unclear if this is an issue in OpenHCL, the guest, or the host, but fix OpenHCL crashing with an emulation failure due to a `GuestMemory` access failure, and instead inject a machine check into the guest. For addresses outside of mmio and ram, continue to emulate but log that the guest did something strange. In the future, we may also inject a machine check on that path. Tested via a uefi app that attempts to access a shared page at a private gpa (see https://github.com/chris-oo/openvmm/blob/uefi-tmk-write-to-shared/tmk/tmk_launch/src/main.rs) with the following crash: ``` [kmsg]: [3.058450] virt_mshv_vtl::processor::tdx: WARN guest accessed inaccessible gpa, injecting MC gpa=0x666d9000 is_shared=false [kmsg]: [3.061137] virt_mshv_vtl::processor: WARN Guest has reported system crash crash=VtlCrash { vp_index: VpIndex(0), last_vtl: Vtl0, control: GuestCrashCtl { pre_os_id: 0, no_crash_dump: false, crash_message: true, crash_notify: true }, parameters: [12, 0, 0, 6790a820, 4d7] } [kmsg]: [3.061758] virt_mshv_vtl::processor: WARN Guest has reported a system crash message "!!!! X64 Exception Type - 12(#MC - Machine-Check) CPU Apic ID - 00000000 !!!!<5c>r<5c>nRIP - 00000000666DB030, CS - 0000000000000038, RFLAGS - 0000000000010202<5c>r<5c>nRAX - 00000000666D9000, RCX - 0000000000000042, RDX - 3333333333333333<5c>r<5c>nRBX - 0000000066E00018, RSP - 0000000033D99150, RBP - 0000000033D99190<5c>r<5c>nRSI - 0000000066E54718, RDI - 0000000033DB8160<5c>r<5c>nR8 - 0000000000000000, R9 - 00000000666ED508, R10 - 00000000666ED5EE<5c>r<5c>nR11 - 000000000000000A, R12 - 0000000000000000, R13 - 0000000000000000<5c>r<5c>nR14 - 0000000033D99AA0, R15 - 0000000033D99A98<5c>r<5c>nDS - 0000000000000030, ES - 0000000000000030, FS - 0000000000000030<5c>r<5c>nGS - 0000000000000030, SS - 0000000000000030<5c>r<5c>nCR0 - 0000000080010073, CR2 - 0000000000000000, CR3 - 0000000033801000<5c>r<5c>nCR4 - 0000000000000668, CR8 - 0000000000000000<5c>r<5c>nDR0 - 0000000000000000, DR1 - 0000000000000000, DR2 - 0000000000000000<5c>r<5c>nDR3 - ``` #1426 tracks implementing this for SNP. Backport of #1340

benhillis · 2025-05-30T00:54:58Z

Backported in #1434

chris-oo requested a review from a team as a code owner May 12, 2025 19:05

smmalis37 reviewed May 12, 2025

View reviewed changes

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs Show resolved Hide resolved

jstarks reviewed May 12, 2025

View reviewed changes

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs Outdated Show resolved Hide resolved

jstarks reviewed May 12, 2025

View reviewed changes

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs Show resolved Hide resolved

jstarks reviewed May 12, 2025

View reviewed changes

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs Show resolved Hide resolved

chris-oo marked this pull request as draft May 15, 2025 18:23

chris-oo force-pushed the inject-mc-tdx branch 4 times, most recently from 57c5608 to d2d59f6 Compare May 28, 2025 15:28

chris-oo added the backport_2505 Change should be backported to the release/2505 branch label May 28, 2025

chris-oo marked this pull request as ready for review May 28, 2025 15:56

chris-oo force-pushed the inject-mc-tdx branch from d2d59f6 to fa4e04a Compare May 28, 2025 16:00

smalis-msft reviewed May 28, 2025

View reviewed changes

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs Outdated Show resolved Hide resolved

smalis-msft reviewed May 28, 2025

View reviewed changes

vmm_core/virt_support_x86emu/src/emulate.rs Outdated Show resolved Hide resolved

jstarks reviewed May 28, 2025

View reviewed changes

chris-oo added 3 commits May 28, 2025 18:16

save

ecfcb05

impl

aed3cdc

fix hide isolation has no vtom

322f4e8

chris-oo force-pushed the inject-mc-tdx branch from 6d9fb62 to 74eb588 Compare May 28, 2025 18:19

smalis-msft reviewed May 28, 2025

View reviewed changes

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs Outdated Show resolved Hide resolved

smalis-msft previously approved these changes May 28, 2025

View reviewed changes

jstarks reviewed May 28, 2025

View reviewed changes

openhcl/virt_mshv_vtl/src/processor/tdx/mod.rs Outdated Show resolved Hide resolved

jstarks reviewed May 28, 2025

View reviewed changes

vmm_core/virt_support_x86emu/src/emulate.rs Outdated Show resolved Hide resolved

jstarks reviewed May 28, 2025

View reviewed changes

vmm_core/virt_support_x86emu/src/emulate.rs Outdated Show resolved Hide resolved

jstarks reviewed May 28, 2025

View reviewed changes

vmm_core/virt_support_x86emu/src/emulate.rs Outdated Show resolved Hide resolved

jstarks reviewed May 28, 2025

View reviewed changes

chris-oo requested a review from smalis-msft May 28, 2025 22:21

chris-oo added 2 commits May 28, 2025 22:26

more targeted

1a1b687

port fixes from other branch

2469bdd

chris-oo dismissed smalis-msft’s stale review via 2469bdd May 28, 2025 22:33

chris-oo force-pushed the inject-mc-tdx branch from dfd20bd to 2469bdd Compare May 28, 2025 22:33

jstarks approved these changes May 28, 2025

View reviewed changes

chris-oo enabled auto-merge (squash) May 28, 2025 22:39

chris-oo merged commit 5c44faa into microsoft:main May 28, 2025
28 checks passed

chris-oo deleted the inject-mc-tdx branch May 28, 2025 23:10

chris-oo mentioned this pull request May 28, 2025

openhcl/tdx: inject machine check on ept exits caused by bad guest addresses (#1340) #1434

Merged

benhillis added the backport_2505_approved label May 30, 2025

benhillis added backported_2505 PR that has been backported to release/2505 and removed backport_2505 Change should be backported to the release/2505 branch labels May 30, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

openhcl/tdx: inject machine check on ept exits caused by bad guest addresses #1340

openhcl/tdx: inject machine check on ept exits caused by bad guest addresses #1340

chris-oo commented May 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jstarks May 28, 2025

Uh oh!

chris-oo May 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jstarks May 28, 2025

Uh oh!

chris-oo May 28, 2025

Uh oh!

Uh oh!

benhillis commented May 30, 2025

Uh oh!

Uh oh!

openhcl/tdx: inject machine check on ept exits caused by bad guest addresses #1340

openhcl/tdx: inject machine check on ept exits caused by bad guest addresses #1340

Conversation

chris-oo commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jstarks May 28, 2025

Choose a reason for hiding this comment

Uh oh!

chris-oo May 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jstarks May 28, 2025

Choose a reason for hiding this comment

Uh oh!

chris-oo May 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

benhillis commented May 30, 2025

Uh oh!

Uh oh!

chris-oo commented May 12, 2025 •

edited

Loading