Skip to content

Add cloud-hypervisor v51.1 with backwards-compatible version flag (CVE-2026-27211)#200

Open
ulziibay-kernel wants to merge 10 commits into
mainfrom
hypeship/upgrade-ch-v50.1
Open

Add cloud-hypervisor v51.1 with backwards-compatible version flag (CVE-2026-27211)#200
ulziibay-kernel wants to merge 10 commits into
mainfrom
hypeship/upgrade-ch-v50.1

Conversation

@ulziibay-kernel
Copy link
Copy Markdown
Contributor

@ulziibay-kernel ulziibay-kernel commented Apr 24, 2026

Summary

Adds Cloud Hypervisor v51.1 alongside existing v49.0 with a config flag to control which version new instances use. This enables a safe, no-downtime upgrade path for CVE-2026-27211.

What changed

  • Both v49.0 and v51.1 binaries are embedded -- no need to drain standby instances before deploy
  • New config flag: hypervisor.cloud_hypervisor_version (default: v49.0)
    • Env override: HYPERVISOR__CLOUD_HYPERVISOR_VERSION=v51.1
  • Existing standby instances restore using their stored version (already the case)
  • New instances use whichever version the flag is set to
  • VMM client regenerated from v51.1 OpenAPI spec with ImageType: Raw on all disks (CVE fix)

Upgrade path

  1. Deploy this PR -- all instances continue on v49.0, no disruption
  2. When ready, set HYPERVISOR__CLOUD_HYPERVISOR_VERSION=v51.1
  3. New instances get v51.1; existing standbys still restore on v49.0
  4. Once all v49.0 standbys have expired, v49.0 can be removed in a follow-up

Security fix (CVE-2026-27211)

Fixes GHSA-jmr4-g2hv-mjj6: arbitrary host file exfiltration via crafted QCOW2 disk headers. Affects CH versions 34.0 through 50.0. Fixed in 50.1+.

v51.1 additionally includes:

  • QCOW2 v3 improvements (live resize, variable refcount, dirty bit)
  • DISCARD/WRITE_ZEROES support for virtio-blk
  • THP for anonymous shared memory (performance)
  • ACPI NUMA affinity for VFIO-PCI (GPU passthrough)

Files changed

  • Makefile -- download both v49.0 and v51.1 binaries
  • cmd/api/config/config.go -- add cloud_hypervisor_version field (default v49.0)
  • lib/vmm/binaries_linux.go -- embed both v49.0 and v51.1
  • lib/vmm/binaries_darwin.go -- version constants for compile compat
  • lib/vmm/version.go -- ParseVersion matches both versions
  • lib/vmm/vmm.go -- regenerated from v51.1 OpenAPI spec
  • lib/vmm/client_test.go -- tests for both versions
  • lib/hypervisor/cloudhypervisor/process.go -- SetDefaultVersion/GetDefaultVersion, configurable GetVersion
  • lib/hypervisor/cloudhypervisor/config.go -- ImageType: Raw on all disks (CVE fix, safe for v49.0)
  • lib/providers/providers.go -- wire config flag at startup
  • lib/instances/version_upgrade_test.go -- new E2E test: create v49.0 -> standby -> switch to v51.1 -> restore -> verify version preserved -> create new instance on v51.1

Test plan

  • go build ./lib/vmm/... ./lib/hypervisor/cloudhypervisor/...
  • go vet clean
  • go test ./lib/vmm/... -- both versions extract and start
  • go test ./lib/hypervisor/cloudhypervisor/... passes
  • TestCloudHypervisorVersionUpgradeRestore -- standby/restore across version change
  • Full go test ./lib/instances/... (requires KVM, CI)

Note

Medium Risk
Changes which Cloud Hypervisor binary can be selected for new VM instances and updates the generated VMM API client; mistakes could break VM start/restore or lead to mismatched snapshot restores across versions.

Overview
Adds Cloud Hypervisor v51.1 as an additional embedded VMM binary (keeping v49.0) and updates build tooling to download/ensure both versions, enabling backwards-compatible upgrades.

Introduces a new config field hypervisor.cloud_hypervisor_default_version and runtime override (cloudhypervisor.SetDefaultVersion/GetDefaultVersion) so new instances can use a chosen CH version, while existing instances continue to restore using the version stored in metadata; CreateInstanceRequest now supports an explicit HypervisorVersion with validation.

Regenerates the Cloud Hypervisor OpenAPI client/spec from v51.1 (including /vm.resize-disk and new schema fields) and sets CH disk ImageType to Raw in VM config generation; adds version-aware capabilities (e.g., disk resize on v51.1) and an integration test covering standby/restore across a default-version flip.

Reviewed by Cursor Bugbot for commit 32e6820. Bugbot is set up for automated code reviews on this repo. Configure here.

@ulziibay-kernel ulziibay-kernel marked this pull request as ready for review April 27, 2026 19:02
@firetiger-agent
Copy link
Copy Markdown

Firetiger deploy monitoring skipped

This PR didn't match the auto-monitor filter configured on your GitHub connection:

Any PR that changes the kernel API. Monitor changes to API endpoints (packages/api/cmd/api/) and Temporal workflows (packages/api/lib/temporal) in the kernel repo

Reason: PR updates Cloud Hypervisor binaries and related build tooling, but does not modify API endpoints (packages/api/cmd/api/) or Temporal workflows (packages/api/lib/temporal) that the filter targets.

To monitor this PR anyway, reply with @firetiger monitor this.

@ulziibay-kernel
Copy link
Copy Markdown
Contributor Author

ulziibay-kernel commented Apr 27, 2026

Changes between v49.0 and v51.1 -- reviewer notes

Update (May 12): This PR is now backwards-compatible. Both v49.0 and v51.1 are embedded. A config flag (hypervisor.cloud_hypervisor_version, default v49.0) controls which version new instances use. Existing standby instances restore using their stored version. No drain strategy needed.

Breaking changes addressed

Change Version Impact on us Action taken
Sector-zero write prevention on autodetected raw images v50.1+ BREAKING -- overlay disk (vdb) fails with I/O errors Fixed: set ImageType: Raw explicitly on all disk configs
backing_files defaults to off v50.1+ Low risk -- we use raw images, not QCOW2 No action needed
Snapshot restore requires exact CH version match v50.0+ HIGH -- standbys on v49.0 can't restore with v51.1 binary Fixed: both binaries embedded, restore uses stored version

Upgrade path

  1. Deploy this PR -- all instances continue on v49.0, no disruption
  2. Set HYPERVISOR__CLOUD_HYPERVISOR_VERSION=v51.1 when ready
  3. New instances get v51.1; existing standbys still restore on v49.0
  4. Once all v49.0 standbys expire, v49.0 can be removed in a follow-up

Medium Impact

Change Version Impact on us Action needed
Byte-range advisory locks on block devices (was whole-file locks) v50.0 Low risk -- better compat with network storage Monitor for lock-related errors
Seccomp filter fixes v50.0 Positive -- fixes seccomp violations in vsock thread Monitor for seccomp kills in CH vmm.log
CPUID fixes in guest v50.0 Positive -- unlikely to break anything None expected
Nested virtualization now configurable (`nested=on off, default on`) v50.0 No change in behavior

Low Impact / Positive (v50.0-v51.1)

  • Live migration performance improvement
  • Live disk resizing API (/vm.resize-disk)
  • QCOW2 v3 improvements (live resize, variable refcount, dirty bit)
  • DISCARD/WRITE_ZEROES support for virtio-blk
  • THP for anonymous shared memory (performance)
  • ACPI NUMA affinity for VFIO-PCI (GPU passthrough)

Comment thread lib/vmm/version.go Outdated
@ulziibay-kernel ulziibay-kernel changed the title Upgrade cloud-hypervisor to v50.1 (CVE-2026-27211) Upgrade cloud-hypervisor to v51.1 (CVE-2026-27211) Apr 28, 2026
@ulziibay-kernel ulziibay-kernel force-pushed the hypeship/upgrade-ch-v50.1 branch from bb7f53e to ef4be42 Compare May 12, 2026 17:19
@ulziibay-kernel ulziibay-kernel changed the title Upgrade cloud-hypervisor to v51.1 (CVE-2026-27211) Add cloud-hypervisor v51.1 with backwards-compatible version flag (CVE-2026-27211) May 12, 2026
Comment thread lib/hypervisor/cloudhypervisor/process.go
Copy link
Copy Markdown
Collaborator

@sjmiller609 sjmiller609 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

excellent, just a few small feedbacks

Comment thread cmd/api/config/config.go Outdated

var (
defaultVersionMu sync.RWMutex
defaultVersion = vmm.V49_0
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need this block and the default version will be somewhere else

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — removed the hardcoded default here. defaultVersionOverride is now a nullable pointer; when nil (the default), GetDefaultVersion() falls back to vmm.DefaultVersion defined in binaries_linux.go.

Comment thread lib/hypervisor/cloudhypervisor/process.go
Comment thread lib/instances/version_upgrade_test.go Outdated
Comment thread lib/vmm/README.md
Comment thread specs/cloud-hypervisor/api-v0.3.0/cloud-hypervisor.yaml
Comment thread Makefile
Comment thread cmd/api/config/config.go Outdated
Comment thread lib/hypervisor/cloudhypervisor/config.go
Comment thread lib/hypervisor/cloudhypervisor/process.go Outdated
Comment thread lib/hypervisor/cloudhypervisor/process.go
@ulziibay-kernel ulziibay-kernel removed the request for review from rgarcia May 12, 2026 20:03
Comment thread lib/vmm/version.go
Comment thread lib/instances/create.go
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4cb1544. Configure here.

Comment thread lib/hypervisor/cloudhypervisor/cloudhypervisor.go
ulziibay-kernel and others added 10 commits May 13, 2026 15:53
Fixes GHSA-jmr4-g2hv-mjj6 (CVE-2026-27211): VMM host file
exfiltration via malicious QCOW2 headers. Affects versions 34.0
through 50.0; fixed in 50.1.

- Drop embedded v48.0 and v49.0 binaries; embed v50.1 only
- Update Makefile downloads, spec source, and ensure-ch-binaries
  check to v50.1
- Update SupportedVersions, ParseVersion, and the default
  GetVersion() returned by the cloud-hypervisor Starter
- Update tests and docs to reference v50.1

Cloud Hypervisor API remains at v0.3.0 (new /vm.resize-disk
endpoint and optional `nested` field are additive, no regen
needed unless the new surface is used).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CH v50.1 prevents sector-zero writes on autodetected raw images as
part of the CVE-2026-27211 fix. Without explicit image_type, the
overlay disk (vdb) fails with I/O errors because CH treats it as a
potential QCOW2 spoof:

  I/O error, dev vdb, sector 0 op 0x1:(WRITE)
  EXT4-fs (vdb): mount failed
  FATAL: dropping to shell for debugging

Fix:
1. Regenerate lib/vmm/vmm.go from the v50.1 OpenAPI spec to pick up
   the new image_type and backing_files fields in DiskConfig
2. Fix malformed enum in the upstream spec (type: enum [...] -> type:
   string with enum list) matching cloud-hypervisor PR #7734
3. Set ImageType: Raw on all disk configs in ToVMConfig so CH skips
   format autodetection and allows sector-zero writes on raw images

Made-with: Cursor
v51.1 is the latest release and includes:
- Same CVE-2026-27211 fix as v50.1
- Fixed image_type enum in OpenAPI spec (PR #7734) -- we no longer
  need our manual spec patch
- QCOW2 v3 improvements (live resize, variable refcount, dirty bit)
- DISCARD/WRITE_ZEROES support for virtio-blk
- THP for anonymous shared memory (performance)
- ACPI NUMA affinity for VFIO-PCI (GPU passthrough)

Regenerated lib/vmm/vmm.go from v51.1's native spec (includes
Unknown as an additional ImageType variant).

Made-with: Cursor
Instead of replacing v49.0 with v51.1, embed both versions and add a
config flag (hypervisor.cloud_hypervisor_version) to control which
version new instances use. Default remains v49.0 for safe rollout.

- Re-add v49.0 binaries alongside v51.1 in Makefile and embed directives
- Add CloudHypervisorVersion field to HypervisorConfig (default "v49.0")
- Add SetDefaultVersion/GetDefaultVersion to CH starter, wired from
  providers.go at startup
- Existing standby instances restore using their stored version
  (already the case via stored.HypervisorVersion in restore.go)
- New E2E test (TestCloudHypervisorVersionUpgradeRestore) verifies:
  create on v49.0 -> standby -> switch default to v51.1 -> restore
  still uses v49.0 -> new instance uses v51.1
- ImageType: Raw (v51.1 CVE fix) is safe for v49.0 (serde ignores
  unknown fields)

Operators can flip to v51.1 when ready:
  HYPERVISOR__CLOUD_HYPERVISOR_VERSION=v51.1

No drain strategy needed -- snapshot restore works across both versions.

Made-with: Cursor
Cursor Bugbot correctly identified that RestoreSnapshot and
ForkFromSnapshot in snapshot.go call starter.GetVersion() and
overwrite HypervisorVersion. With the configurable default, flipping
to v51.1 would break standby snapshot restores of v49.0 instances.

Fix: only overwrite HypervisorVersion when the target hypervisor TYPE
differs from the source (stopped snapshot cross-hypervisor restore).
When the type is the same, preserve the source snapshot's version so
CH gets the exact binary match it requires.

Also fix ParseVersion to use regex extraction + SupportedVersions
lookup instead of substring Contains, addressing the Bugbot note about
"v50.1" matching "v50.10" (now moot but the pattern was still fragile).

Made-with: Cursor
- Move DefaultVersion to binaries_linux.go as single source of truth
  (not duplicated in config defaults)
- Rename config to cloud_hypervisor_default_version, default empty
  (empty = use vmm.DefaultVersion from code)
- SetDefaultVersion now returns error on invalid versions (Bugbot fix)
- Add HypervisorVersion to CreateInstanceRequest so callers can pin a
  specific version at creation time
- Add per-version Capabilities pattern: CapabilitiesForVersion() with
  SupportsDiskResize=true for v51.1+ (establishes the pattern)
- Add API versioning comment in Makefile (single spec for now)
- Update README with multi-version architecture docs
- Update E2E test to use explicit HypervisorVersion on create request

Made-with: Cursor
- Validate req.HypervisorVersion against supported versions before
  storing in metadata (Bugbot: unsupported version would persist and
  fail at VM start)
- Replace regex-based ParseVersion with direct string matching against
  SupportedVersions, longest-first to avoid "v49.0" matching "v49.0.1"
  (Bugbot: greedy regex could match three-part version strings)

Made-with: Cursor
@ulziibay-kernel ulziibay-kernel force-pushed the hypeship/upgrade-ch-v50.1 branch from 4cb1544 to 32e6820 Compare May 13, 2026 21:53
Copy link
Copy Markdown
Collaborator

@sjmiller609 sjmiller609 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional feedbacks. Looks great! Thanks

Comment on lines +90 to +92
// GetVersion returns the configured default Cloud Hypervisor version.
// This controls which version is used for new instances; existing instances
// use the version stored in their metadata.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

V51_1 CHVersion = "v51.1"
)

const DefaultVersion = V49_0
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be defaulting to V51_1?

Comment thread lib/vmm/binaries_linux.go
)

var SupportedVersions = []CHVersion{V48_0, V49_0}
const DefaultVersion = V49_0
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we be defaulting to V51_1? It would be OK in follow up if intentional

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants