File-based disk-only VM snapshot with KVM as hypervisor #10632

JoaoJandre · 2025-03-28T13:54:55Z

Description

This PR implements the spec available at #9524. For more information regarding it, please read the spec.

Furthermore, the following changes that are not contemplated in the spec were added:

The snapshot.merge.timeout agent property was added. It is only considered if libvirt.events.enabled is true;
A new snapshot merge process (which affects normal volume snapshots and this feature) was created. When libvirt.events.enabled is true, ACS will register to gather events from Libvirt and will collect information on the process, providing a progress report in the logs. If the configuration is false, the old process is used;
Volumes attached to VMs with file-based disk-only VM snapshots in KVM are able to be resized.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Basic Tests

I created a test VM to carry out the tests below. Additionally, after performing the relevant operations, the VM's XML and the storage were checked to observe if the snapshots existed.

Snapshot Creation

The tests below were also repeated with the VM stopped.

N	Test	Result
1	Take a snapshot of VM 1 without specifying quiesceVM	Snapshot created
2	Take a snapshot of VM 2 specifying quiesceVM	Snapshot created

Snapshot Reversion

N	Test	Result
1	Revert VM in Running state to any snapshot	Error thrown
2	Revert VM in Stopped state to snapshot 1 and start it	VM reverted and started successfully

Snapshot Removal

N	Test	Result
1	Create a new snapshot 3 after the second reversion test and delete snapshot 1	I verified that the snapshot was no longer listed and had the correct database metadata, the file still existed because more than one delta depended on it
2	Delete snapshot 2	Snapshot deleted; snapshot 1 was merged with snapshot 3 since it only had the latter as a dependency
3	Delete snapshot 3 (current)	Snapshot removed, merged with the VM's volume
4	Create 3 snapshots and remove the first one	Snapshot removed, merged with the second snapshot
5	Create two snapshots, revert to the first, and delete the second	Snapshot deleted

Advanced Tests

Deletion Test

All tests were carried out with the VM stopped.

I created 3 snapshots: s1, s2, and s3.
I reverted the VM to snapshot s2.
I created snapshot s4.
I removed snapshot s2.

The snapshot was marked as hidden and was not removed from storage.

I removed snapshot s3.

Snapshot s3 was removed normally. Snapshot s2 was merged with snapshot s4.

I created snapshot s5.
I reverted to snapshot s4.
I removed snapshot s4.

Snapshot s4 was marked as hidden and was not removed from storage.

I removed snapshot s5.
Snapshot s5 was removed normally. Snapshot s4 was merged with the delta of the VM's volume.
I removed the last remaining snapshot (s1). It was removed normally.

Reversion Test

I created two snapshots: s1 and s2.
I reverted to snapshot s1.
I removed snapshot s1.

Snapshot s1 was marked as hidden and was not removed from storage.

I reverted to snapshot s2. Snapshot s1 was merged with the base volume.

Concurrent Test

I created 4 VMs and took a VM snapshot of each. Then, I instructed to remove them all at the same time. All snapshots were removed simultaneously and successfully.

Test with Multiple Volumes

I created a VM with one datadisk and attached 8 more datadisks (10 volumes in total), took two VM snapshots, and then instructed to remove one at a time. The snapshots were removed successfully.

Tests Changing the `snapshot.merge.timeout` Config

I changed the config to 1 and restarted the host;
I created a VM, took a VM snapshot, accessed it, and wrote 4GB of data to it;
I tried to remove the snapshot, an error occurred, and looking at the logs, it was possible to observe that it timed out;
I manually aborted the blockcommit process;
I changed the config to 0 and restarted the host;
I tried to remove the snapshot, and it was performed correctly;

Tests Related to Volume Resize with Disk-Only VM Snapshots on KVM

Test	Result	Expected?
Create a VM, take a snapshot, resize the volume	Resize performed successfully, both in metadata and when checked with `qemu-img info`	Y
Stop the VM and revert the snapshot	Revert performed successfully, volume size returned to original, both in metadata and `qemu-img info`	Y
Remove the snapshot with the VM stopped	The delta of the volume was correctly merged with the snapshot's, and the final size was that of the volume	Y
Start the VM, take a new snapshot, resize the volume, and remove the snapshot	Deltas were correctly merged, and the final size was that of the volume	Y

The last two tests were repeated on a VM with several snapshots, so that a merge between snapshots was performed. The result was the same.

Tests Related to Events:

Create VM, take disk-only VM snapshot, resize the root volume by 1GB more, stop the VM, revert the snapshot. It was observed through the cloud.usage_event table that the resize event was correctly triggered, and it was also observed via GUI that the account's resource limit was updated.
Repeat the test above with a VM with two volumes, with only one resized. The test had the same result, and only one resize event was triggered, for the volume that had been resized.

JoaoJandre · 2025-03-28T14:02:31Z

@blueorangutan package

codecov · 2025-03-28T14:03:12Z

Codecov Report

Attention: Patch coverage is 10.34816% with 927 lines in your changes missing coverage. Please review.

Project coverage is 16.58%. Comparing base (6fdaf51) to head (92c3486).
Report is 98 commits behind head on main.

Files with missing lines	Patch %	Lines
...napshot/KvmFileBasedStorageVmSnapshotStrategy.java	0.24%	411 Missing and 1 partial ⚠️
...LibvirtCreateDiskOnlyVMSnapshotCommandWrapper.java	1.04%	95 Missing ⚠️
.../LibvirtMergeDiskOnlyVMSnapshotCommandWrapper.java	1.38%	71 Missing ⚠️
...ervisor/kvm/resource/LibvirtComputingResource.java	50.75%	62 Missing and 3 partials ⚠️
...LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.java	1.96%	50 Missing ⚠️
...d/hypervisor/kvm/resource/BlockCommitListener.java	28.12%	22 Missing and 1 partial ⚠️
...m/cloud/agent/api/storage/SnapshotMergeTreeTO.java	0.00%	21 Missing ⚠️
...tack/storage/snapshot/DefaultSnapshotStrategy.java	0.00%	19 Missing ⚠️
...java/org/apache/cloudstack/utils/qemu/QemuImg.java	0.00%	19 Missing ⚠️
...nt/api/storage/MergeDiskOnlyVmSnapshotCommand.java	0.00%	18 Missing ⚠️
... and 18 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #10632      +/-   ##
============================================
+ Coverage     16.41%   16.58%   +0.17%     
- Complexity    13629    13940     +311     
============================================
  Files          5702     5743      +41     
  Lines        503405   509243    +5838     
  Branches      60976    61886     +910     
============================================
+ Hits          82626    84472    +1846     
- Misses       411594   415334    +3740     
- Partials       9185     9437     +252

Flag	Coverage Δ
uitests	`3.93% <ø> (-0.07%)`	⬇️
unittests	`17.47% <10.34%> (+0.20%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-04-28T08:49:49Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

JoaoJandre · 2025-04-28T16:27:51Z

@blueorangutan package

blueorangutan · 2025-04-28T16:30:03Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-28T17:52:53Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13204

JoaoJandre · 2025-04-28T18:02:23Z

@rohityadavcloud @sureshanaparti @weizhouapache could we run the CI?

DaanHoogland · 2025-04-29T07:02:26Z

@blueorangutan test

blueorangutan · 2025-04-29T07:04:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-04-29T22:57:50Z

[SF] Trillian test result (tid-13177)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 54050 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr10632-t13177-kvm-ol8.zip
Smoke tests completed. 140 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_02_restore_vm_strict_tags_failure	`Failure`	53.35	test_vm_strict_host_tags.py
test_02_scale_vm_strict_tags_failure	`Failure`	54.75	test_vm_strict_host_tags.py
test_06_deploy_vm_on_any_host_with_strict_tags_failure	`Failure`	4.69	test_vm_strict_host_tags.py

github-actions · 2025-05-02T10:35:25Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

…shot

github-actions · 2025-05-12T14:37:39Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

rohityadavcloud · 2025-06-10T08:25:48Z

@blueorangutan package

blueorangutan · 2025-06-10T08:28:04Z

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-10T10:03:15Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13686

JoaoJandre · 2025-06-11T12:07:57Z

@blueorangutan package

blueorangutan · 2025-06-11T12:10:04Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-11T13:09:36Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13710

JoaoJandre · 2025-06-11T16:36:33Z

@blueorangutan package

blueorangutan · 2025-06-11T16:38:03Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-11T17:12:53Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13715

blueorangutan · 2025-06-12T09:23:11Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✖️ debian ✔️ suse15. SL-JID 13740

JoaoJandre · 2025-06-13T11:58:39Z

@blueorangutan package

bernardodemarco

@JoaoJandre, outstanding work! I haven’t gone through all the code yet, nor have I performed more advanced tests. However, here are the basic ones I’ve already done.

Tests descriptions

The following tests were performed to verify basic disk-only VM snapshot creation, deletion and reversion workflows:

Deployed a VM
Created snapshot s1
Created snapshot s2
Tried to revert the VM to the s1 snapshot. An error was thrown, as expected, because the VM was in the Running state
Successfully reverted the VM to the s1 snapshot
Initialized the VM again and created snapshot s3
Stopped the VM and reverted it to the snapshot s2
Performed the above step for the s1 and s3 snapshots
Deleted snapshot s1. Since it had two children (s2 and s3), verified it it was marked as Hidden in the DB and it remained stored in the storage:

select * from vm_snapshots where uuid = '76a90156-a9c8-4ff4-a99b-ae5473c36923'\G

*************************** 1. row ***************************
                 id: 27
               uuid: 76a90156-a9c8-4ff4-a99b-ae5473c36923
               name: i-2-26-VM_VS_20250615141733
       display_name: s1
        description: NULL
              vm_id: 26
         account_id: 2
          domain_id: 1
service_offering_id: 1
   vm_snapshot_type: Disk
              state: Hidden
             parent: NULL
            current: 0
       update_count: 8
            updated: 2025-06-15 14:22:37
            created: 2025-06-15 14:17:33
            removed: NULL
1 row in set (0.001 sec)

Deleted the s2 snapshot
Reverted the s3 snapshot
Created snapshot s4 and successfully reverted the VM to it

The following tests were performed to verify volume resizing behavior alongside disk-only VM snapshots:

Deployed a VM
Created snapshot s1
Successfully resized the VM's root disk from 8 GiB to 20 GiB

qemu-img info -U 777614fc-47bd-4d77-a904-19ad3a817eb1
image: 777614fc-47bd-4d77-a904-19ad3a817eb1
file format: qcow2
virtual size: 20 GiB (21474836480 bytes)
disk size: 3.38 MiB
cluster_size: 65536
backing file: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/66cbefff-a06b-4ea9-b3e0-aa7dacc0301a
backing file format: qcow2
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false
Child node '/file':
    filename: 777614fc-47bd-4d77-a904-19ad3a817eb1
    protocol type: file
    file length: 3.44 MiB (3604480 bytes)
    disk size: 3.38 MiB

Created snapshot s2
Reverted the VM to the s1 snapshot and verified that the root disk size changed accordingly
Reverted the VM to the s2 snapshot and verified that the root disk size changed accordingly

The following tests were performed to verify the behavior of disk-only VM snapshots with VMs with multiple volumes.

Verified that it was not possible to attach new volumes to VMs that already had VM snapshots
Deployed a VM with a root disk and data disk
Wrote data to both disks and created snapshot s1
Verified that it was not possible to detach the data disk, because the VM already had VM snapshots
Wrote data to both disks and created snapshot s2
Stopped the VM and reverted to s1
Stopped the VM and reverted to s2

Other tests:

Verified that it was not possible to create VM snapshots with memory when the VM already had disk-only snapshots
After deleting the disk-only VM snapshots, verified that it was possible to create VM snapshots with memory
Verified that it was not possible to create disk-only VM snapshots when the VM already had VM snapshots with memory
Verified the creation of disk-only VM snapshots with the Quiesce Instance parameter defined as true

bernardodemarco

@JoaoJandre, when testing some more advanced workflows of creation, reversion and deletion of disk-only VM snapshots, I created the following scenario:

Created snapshots s1, s2 and s3
Reverted to snapshot s2 and created snapshot s4

Then, I deleted snapshot s2 and verified that it was marked as Hidden in the DB. Deleted snapshot s1 and reverted to snapshot s3:

When reverting to the s3 snapshot, the operation failed:

The logs seem to indicate an error when creating the new delta for the volume:

2025-06-15 15:29:11,565 DEBUG [resource.wrapper.LibvirtRevertDiskOnlyVMSnapshotCommandWrapper] (AgentRequest-Handler-1:[]) (logid:59368419) Creating new delta for volume [4b80abb2-562c-4325-b68f-3655e933daeb] as part of the disk-only VM snapshot revert process for VM [i-2-14-VM].
2025-06-15 15:29:11,565 DEBUG [utils.script.Script] (AgentRequest-Handler-1:[]) (logid:59368419) Executing command [qemu-img create -f qcow2 -F qcow2 -b /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/d0dd008e-9458-4995-b9ca-e0f41bb73556 /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd ].
2025-06-15 15:29:11,584 WARN  [utils.script.Script] (AgentRequest-Handler-1:[]) (logid:59368419) Execution of process [8348] for command [qemu-img create -f qcow2 -F qcow2 -b /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/d0dd008e-9458-4995-b9ca-e0f41bb73556 /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd ] failed.
2025-06-15 15:29:11,584 DEBUG [utils.script.Script] (AgentRequest-Handler-1:[]) (logid:59368419) Exit value of process [8348] for command [qemu-img create -f qcow2 -F qcow2 -b /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/d0dd008e-9458-4995-b9ca-e0f41bb73556 /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd ] is [1].
2025-06-15 15:29:11,584 WARN  [utils.script.Script] (AgentRequest-Handler-1:[]) (logid:59368419) Process [8348] for command [qemu-img create -f qcow2 -F qcow2 -b /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/d0dd008e-9458-4995-b9ca-e0f41bb73556 /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd ] encountered the error: [qemu-img: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd: Could not open backing file: Could not open backing file: Could not open '/mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/4b80abb2-562c-4325-b68f-3655e933daeb': No such file or directoryCould not open backing image.].
2025-06-15 15:29:11,584 ERROR [resource.wrapper.LibvirtRevertDiskOnlyVMSnapshotCommandWrapper] (AgentRequest-Handler-1:[]) (logid:59368419) Exception while reverting disk-only VM snapshot for VM [i-2-14-VM]. Deleting leftover deltas. org.apache.cloudstack.utils.qemu.QemuImgException: qemu-img: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd: Could not open backing file: Could not open backing file: Could not open '/mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/4b80abb2-562c-4325-b68f-3655e933daeb': No such file or directoryCould not open backing image.
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:268)
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:200)
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:297)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.execute(LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.java:71)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.execute(LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.java:43)
--
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2025-06-15 15:29:11,591 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-1:[]) (logid:59368419) Seq 1-6557804007404863671:  { Ans: , MgmtId: 169551026600426, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":"false","details":"Exception: org.apache.cloudstack.utils.qemu.QemuImgException
Message: qemu-img: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd: Could not open backing file: Could not open backing file: Could not open '/mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/4b80abb2-562c-4325-b68f-3655e933daeb': No such file or directoryCould not open backing image.
Stack: org.apache.cloudstack.utils.qemu.QemuImgException: qemu-img: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd: Could not open backing file: Could not open backing file: Could not open '/mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/4b80abb2-562c-4325-b68f-3655e933daeb': No such file or directoryCould not open backing image.
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:268)
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:200)
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:297)

Below are uploaded the full workflow execution logs:

JoaoJandre · 2025-06-16T12:06:02Z

@JoaoJandre, when testing some more advanced workflows of creation, reversion and deletion of disk-only VM snapshots, I created the following scenario:

* Created snapshots `s1`, `s2` and `s3`

* Reverted to snapshot `s2` and created snapshot `s4`

Then, I deleted snapshot s2 and verified that it was marked as Hidden in the DB. Deleted snapshot s1 and reverted to snapshot s3:

When reverting to the s3 snapshot, the operation failed:

The logs seem to indicate an error when creating the new delta for the volume:

2025-06-15 15:29:11,565 DEBUG [resource.wrapper.LibvirtRevertDiskOnlyVMSnapshotCommandWrapper] (AgentRequest-Handler-1:[]) (logid:59368419) Creating new delta for volume [4b80abb2-562c-4325-b68f-3655e933daeb] as part of the disk-only VM snapshot revert process for VM [i-2-14-VM].
2025-06-15 15:29:11,565 DEBUG [utils.script.Script] (AgentRequest-Handler-1:[]) (logid:59368419) Executing command [qemu-img create -f qcow2 -F qcow2 -b /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/d0dd008e-9458-4995-b9ca-e0f41bb73556 /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd ].
2025-06-15 15:29:11,584 WARN  [utils.script.Script] (AgentRequest-Handler-1:[]) (logid:59368419) Execution of process [8348] for command [qemu-img create -f qcow2 -F qcow2 -b /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/d0dd008e-9458-4995-b9ca-e0f41bb73556 /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd ] failed.
2025-06-15 15:29:11,584 DEBUG [utils.script.Script] (AgentRequest-Handler-1:[]) (logid:59368419) Exit value of process [8348] for command [qemu-img create -f qcow2 -F qcow2 -b /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/d0dd008e-9458-4995-b9ca-e0f41bb73556 /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd ] is [1].
2025-06-15 15:29:11,584 WARN  [utils.script.Script] (AgentRequest-Handler-1:[]) (logid:59368419) Process [8348] for command [qemu-img create -f qcow2 -F qcow2 -b /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/d0dd008e-9458-4995-b9ca-e0f41bb73556 /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd ] encountered the error: [qemu-img: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd: Could not open backing file: Could not open backing file: Could not open '/mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/4b80abb2-562c-4325-b68f-3655e933daeb': No such file or directoryCould not open backing image.].
2025-06-15 15:29:11,584 ERROR [resource.wrapper.LibvirtRevertDiskOnlyVMSnapshotCommandWrapper] (AgentRequest-Handler-1:[]) (logid:59368419) Exception while reverting disk-only VM snapshot for VM [i-2-14-VM]. Deleting leftover deltas. org.apache.cloudstack.utils.qemu.QemuImgException: qemu-img: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd: Could not open backing file: Could not open backing file: Could not open '/mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/4b80abb2-562c-4325-b68f-3655e933daeb': No such file or directoryCould not open backing image.
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:268)
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:200)
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:297)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.execute(LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.java:71)
	at com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.execute(LibvirtRevertDiskOnlyVMSnapshotCommandWrapper.java:43)
--
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

2025-06-15 15:29:11,591 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-1:[]) (logid:59368419) Seq 1-6557804007404863671:  { Ans: , MgmtId: 169551026600426, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":"false","details":"Exception: org.apache.cloudstack.utils.qemu.QemuImgException
Message: qemu-img: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd: Could not open backing file: Could not open backing file: Could not open '/mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/4b80abb2-562c-4325-b68f-3655e933daeb': No such file or directoryCould not open backing image.
Stack: org.apache.cloudstack.utils.qemu.QemuImgException: qemu-img: /mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/f9fa34c3-993a-47d1-a7aa-6c08ec23f9dd: Could not open backing file: Could not open backing file: Could not open '/mnt/10d28cdf-71a7-33ad-802e-f4ec9042e4fd/4b80abb2-562c-4325-b68f-3655e933daeb': No such file or directoryCould not open backing image.
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:268)
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:200)
	at org.apache.cloudstack.utils.qemu.QemuImg.create(QemuImg.java:297)

Below are uploaded the full workflow execution logs:

* [ms-logs.txt](https://github.com/user-attachments/files/20746296/ms-logs.txt)

* [agent-logs.txt](https://github.com/user-attachments/files/20746297/agent-logs.txt)

@bernardodemarco thank you for the tests, I'll check it out as soon as possible.

sureshanaparti · 2025-06-16T12:47:54Z

@blueorangutan package

blueorangutan · 2025-06-16T12:50:04Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-16T14:16:16Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✔️ debian ✖️ suse15. SL-JID 13793

JoaoJandre · 2025-06-17T16:38:34Z

@bernardodemarco I've fixed the reported errors and validated that the use case you reported is working. Could you check?

JoaoJandre · 2025-06-17T16:38:39Z

@blueorangutan package

blueorangutan · 2025-06-17T16:40:06Z

@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-17T18:06:00Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13819

JoaoJandre · 2025-06-20T11:49:51Z

@sureshanaparti could we run the CI here?

DaanHoogland · 2025-06-23T08:48:30Z

@blueorangutan LLtest

blueorangutan · 2025-06-23T08:50:02Z

@DaanHoogland a [LL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

File-based disk-only VM snapshot with KVM as hypervisor

2027f25

JoaoJandre added the type:new-feature label Mar 28, 2025

JoaoJandre added this to the 4.22.0 milestone Mar 28, 2025

boring-cyborg bot added component:agent component:api component:kvm component:storage labels Mar 28, 2025

winterhazel self-requested a review March 28, 2025 17:09

rohityadavcloud requested review from sureshanaparti and weizhouapache April 1, 2025 10:21

github-actions bot added the status:has-conflicts label Apr 28, 2025

Merge remote-tracking branch 'origin' into kvm-disk-only-vm-snapshot

73b6820

github-actions bot removed the status:has-conflicts label Apr 28, 2025

github-actions bot added the status:has-conflicts label May 2, 2025

Merge remote-tracking branch 'origin/main' into kvm-disk-only-vm-snap…

3d2be3f

…shot

github-actions bot removed the status:has-conflicts label May 5, 2025

Merge remote-tracking branch 'origin/main' into kvm-disk-only-vm-snap…

0e0c6d8

…shot

github-actions bot added the status:has-conflicts label May 12, 2025

Add limits to avoid data loss

6a08bf2

fix validation

af9090e

JoaoJandre requested review from hsato03 and bernardodemarco June 13, 2025 11:59

bernardodemarco reviewed Jun 15, 2025

View reviewed changes

fix merge witthout events and parent search

92c3486

boring-cyborg bot added the component:backup label Jun 17, 2025

File-based disk-only VM snapshot with KVM as hypervisor #10632

Are you sure you want to change the base?

File-based disk-only VM snapshot with KVM as hypervisor #10632

Uh oh!

Conversation

JoaoJandre commented Mar 28, 2025

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Basic Tests

Snapshot Creation

Snapshot Reversion

Snapshot Removal

Advanced Tests

Deletion Test

Reversion Test

Concurrent Test

Test with Multiple Volumes

Tests Changing the snapshot.merge.timeout Config

Tests Related to Volume Resize with Disk-Only VM Snapshots on KVM

Tests Related to Events:

Uh oh!

JoaoJandre commented Mar 28, 2025

Uh oh!

codecov bot commented Mar 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Apr 28, 2025

Uh oh!

JoaoJandre commented Apr 28, 2025

Uh oh!

blueorangutan commented Apr 28, 2025

Uh oh!

blueorangutan commented Apr 28, 2025

Uh oh!

JoaoJandre commented Apr 28, 2025

Uh oh!

DaanHoogland commented Apr 29, 2025

Uh oh!

blueorangutan commented Apr 29, 2025

Uh oh!

blueorangutan commented Apr 29, 2025

Uh oh!

github-actions bot commented May 2, 2025

Uh oh!

github-actions bot commented May 12, 2025

Uh oh!

rohityadavcloud commented Jun 10, 2025

Uh oh!

blueorangutan commented Jun 10, 2025

Uh oh!

blueorangutan commented Jun 10, 2025

Uh oh!

JoaoJandre commented Jun 11, 2025

Uh oh!

blueorangutan commented Jun 11, 2025

Uh oh!

blueorangutan commented Jun 11, 2025

Uh oh!

JoaoJandre commented Jun 11, 2025

Uh oh!

blueorangutan commented Jun 11, 2025

Uh oh!

blueorangutan commented Jun 11, 2025

Uh oh!

blueorangutan commented Jun 12, 2025

Uh oh!

JoaoJandre commented Jun 13, 2025

Uh oh!

bernardodemarco left a comment

Choose a reason for hiding this comment

Tests descriptions

Uh oh!

bernardodemarco left a comment

Choose a reason for hiding this comment

Tests Changing the `snapshot.merge.timeout` Config

codecov bot commented Mar 28, 2025 •

edited

Loading