Save systemvm template details for cpvm, ssvm and vr's. #9721

BartJM · 2024-09-23T10:06:22Z

Description

This PR adds saving systemvm template settings to the user_vm_details table. Allows systemvms to start with details specified in the systemvm template settings.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Change systemvm template settings to add iothreads and io.policy.
Recreate ssvm, cpvm and vr and verified the details are in the db and the vms are running with the change.

Ran on a ubuntu 20.04 manager, 22.04 kvm host testing env and mbx kvm centos8 env.

How did you try to break this feature and the system with this change?

boring-cyborg · 2024-09-23T10:06:27Z

Congratulations on your first Pull Request and welcome to the Apache CloudStack community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/cloudstack/blob/main/CONTRIBUTING.md)
Here are some useful points:

In case of a new feature add useful documentation (raise doc PR at https://github.com/apache/cloudstack-documentation)
Be patient and persistent. It might take some time to get a review or get the final approval from the committers.
Pay attention to the quality of your code, ensure tests are passing and your PR doesn't have conflicts.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Issues, Mailing list and Slack.
Be sure to read the CloudStack Coding Conventions.
Apache CloudStack is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: [email protected] (https://cloudstack.apache.org/mailing-lists.html)
Slack: https://apachecloudstack.slack.com/

engine/schema/src/main/java/com/cloud/vm/dao/ConsoleProxyDaoImpl.java

engine/schema/src/main/java/com/cloud/vm/dao/UserVmDetailsDaoImpl.java

codecov · 2024-09-23T14:21:27Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 4.00%. Comparing base (f2a6a2f) to head (69d2347).
Report is 22 commits behind head on 4.20.

❗ There is a different number of reports uploaded between BASE (f2a6a2f) and HEAD (69d2347). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (f2a6a2f) HEAD (69d2347)

unittests 1 0

Additional details and impacted files

@@             Coverage Diff              @@
##               4.20   #9721       +/-   ##
============================================
- Coverage     16.13%   4.00%   -12.14%     
============================================
  Files          5656     401     -5255     
  Lines        497622   32598   -465024     
  Branches      60339    5788    -54551     
============================================
- Hits          80302    1305    -78997     
+ Misses       408370   31144   -377226     
+ Partials       8950     149     -8801

Flag	Coverage Δ
uitests	`4.00% <ø> (ø)`
unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

weizhouapache

code lgtm

not tested yet

engine/schema/src/main/java/com/cloud/vm/dao/UserVmDetailsDaoImpl.java

weizhouapache · 2024-09-24T07:44:33Z

@blueorangutan package

blueorangutan · 2024-09-24T07:46:04Z

@weizhouapache a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-09-24T08:47:29Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11190

DaanHoogland

nice thanks @BartJM CLGTM

DaanHoogland · 2024-09-27T09:39:04Z

@blueorangutan package

blueorangutan · 2024-09-27T09:40:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

BartJM · 2024-09-27T09:45:01Z

With the new changes also tested the following on mbx centos8 kvm for user vms. All changes were reflected in the database as expected.

Create a vm with ssh key,
Create password enabled vm with ssh key.
Reset ssh key and password.
Set iothreads enabled and io.policy to storage_specific.
Create a vm with custom offering, change the custom values and change offering to fixed
Change vm with fixed offering to custom offering

blueorangutan · 2024-09-27T10:59:43Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11217

DaanHoogland · 2024-09-27T11:56:07Z

@blueorangutan test

blueorangutan · 2024-09-27T11:58:03Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2024-09-28T06:59:13Z

[SF] Trillian test result (tid-11565)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 66133 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9721-t11565-kvm-ol8.zip
Smoke tests completed. 138 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_03_create_iso_with_checksum_md5	`Error`	66.48	test_iso.py
ContextSuite context=TestISOUsage>:setup	`Error`	0.00	test_usage.py
test_disable_oobm_ha_state_ineligible	`Error`	288.31	test_hostha_kvm.py
test_hostha_configure_default_driver	`Error`	77.00	test_hostha_kvm.py
test_hostha_enable_ha_when_host_disabled	`Error`	87.68	test_hostha_kvm.py
test_hostha_enable_ha_when_host_disconected	`Error`	156.97	test_hostha_kvm.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	151.03	test_hostha_kvm.py
test_hostha_kvm_host_degraded	`Error`	76.92	test_hostha_kvm.py
test_hostha_kvm_host_fencing	`Error`	35.70	test_hostha_kvm.py
test_hostha_kvm_host_recovering	`Error`	34.50	test_hostha_kvm.py
test_remove_ha_provider_not_possible	`Error`	37.59	test_hostha_kvm.py

DaanHoogland · 2024-09-30T13:28:22Z

I cannot relate any of the errors to the code (yet) trying insanity first.

DaanHoogland · 2024-09-30T13:28:45Z

@blueorangutan package

blueorangutan · 2024-09-30T13:30:03Z

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2024-09-30T14:46:19Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11243

DaanHoogland · 2024-09-30T15:10:19Z

@blueorangutan test

blueorangutan · 2024-09-30T15:12:04Z

@DaanHoogland a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2024-10-01T05:45:36Z

[SF] Trillian test result (tid-11580)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 50195 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9721-t11580-kvm-ol8.zip
Smoke tests completed. 141 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

Pearl1594 · 2025-02-18T15:05:13Z

@blueorangutan test matrix

blueorangutan · 2025-02-18T15:06:04Z

@Pearl1594 a [SL] Trillian-Jenkins matrix job (EL8 mgmt + EL8 KVM, Ubuntu22 mgmt + Ubuntu22 KVM, EL8 mgmt + VMware 7.0u3, EL9 mgmt + XCP-ng 8.2 ) has been kicked to run smoke tests

blueorangutan · 2025-02-19T07:10:05Z

[SF] Trillian test result (tid-12465)
Environment: kvm-ubuntu22 (x2), Advanced Networking with Mgmt server u22
Total time taken: 55625 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9721-t12465-kvm-ubuntu22.zip
Smoke tests completed. 137 look OK, 4 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestClusterDRS>:setup	`Error`	0.00	test_cluster_drs.py
test_11_isolated_network_with_dynamic_routed_mode	`Error`	1.34	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	2.43	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	2.43	test_ipv4_routing.py
test_oobm_multiple_mgmt_server_ownership	`Failure`	31.89	test_outofbandmanagement.py
test_06_purge_expunged_vm_background_task	`Failure`	385.67	test_purge_expunged_vms.py

blueorangutan · 2025-02-19T08:20:16Z

[SF] Trillian test result (tid-12464)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 59355 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9721-t12464-kvm-ol8.zip
Smoke tests completed. 138 look OK, 3 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_11_isolated_network_with_dynamic_routed_mode	`Error`	2.28	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	2.48	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	2.48	test_ipv4_routing.py
test_06_purge_expunged_vm_background_task	`Failure`	393.56	test_purge_expunged_vms.py
test_01_redundant_vpc_site2site_vpn	`Failure`	481.92	test_vpc_vpn.py

blueorangutan · 2025-02-19T09:26:22Z

[SF] Trillian test result (tid-12466)
Environment: vmware-70u3 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 63561 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9721-t12466-vmware-70u3.zip
Smoke tests completed. 129 look OK, 12 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_events_resource	`Error`	361.73	test_events_resource.py
test_04_deploy_vm_with_extraconfig_throws_exception_vmware	`Error`	4.57	test_deploy_vm_extra_config_data.py
test_11_isolated_network_with_dynamic_routed_mode	`Error`	1.32	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	2.42	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	2.42	test_ipv4_routing.py
test_deploy_more_vms_than_limit_allows	`Error`	139.46	test_deploy_vms_in_parallel.py
test_04_list_domains_level_filter	`Failure`	0.04	test_list_domains.py
test_05_list_domains_no_filter	`Failure`	0.03	test_list_domains.py
test_05_list_volumes_isrecursive	`Failure`	0.04	test_list_volumes.py
test_07_list_volumes_listall	`Failure`	0.04	test_list_volumes.py
test_04_deploy_vm_for_other_user_and_test_vm_operations	`Error`	117.21	test_network_permissions.py
test_01_deployVMInSharedNetwork	`Error`	155.30	test_network.py
test_06_purge_expunged_vm_background_task	`Failure`	396.96	test_purge_expunged_vms.py
test_02_restore_vm_with_disk_offering	`Error`	60.18	test_restore_vm.py
test_03_restore_vm_with_disk_offering_custom_size	`Error`	52.89	test_restore_vm.py
test_14_destroy_vm_delete_protection	`Error`	7.62	test_vm_life_cycle.py
test_02_restore_vm_strict_tags_failure	`Error`	58.40	test_vm_strict_host_tags.py

blueorangutan · 2025-02-19T12:20:11Z

[SF] Trillian test result (tid-12467)
Environment: xcpng82 (x2), Advanced Networking with Mgmt server ol9
Total time taken: 74470 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9721-t12467-xcpng82.zip
Smoke tests completed. 133 look OK, 8 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File
test_01_condensed_drs_algorithm	`Failure`	164.19	test_cluster_drs.py
test_02_balanced_drs_algorithm	`Failure`	189.28	test_cluster_drs.py
test_11_isolated_network_with_dynamic_routed_mode	`Error`	1.24	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	2.75	test_ipv4_routing.py
test_12_vpc_and_tier_with_dynamic_routed_mode	`Error`	2.75	test_ipv4_routing.py
ContextSuite context=TestSharedNetworkWithConfigDrive>:setup	`Error`	10.49	test_network.py
test_01_non_strict_host_anti_affinity	`Error`	232.93	test_nonstrict_affinity_group.py
test_02_non_strict_host_affinity	`Error`	117.12	test_nonstrict_affinity_group.py
test_06_purge_expunged_vm_background_task	`Failure`	407.70	test_purge_expunged_vms.py
test_create_pvlan_network	`Error`	0.09	test_pvlan.py
test_02_create_volume	`Error`	3.21	test_resource_names.py
test_05_scale_vm_dont_allow_disk_offering_change	`Failure`	73.70	test_scale_vm.py

Pearl1594 · 2025-02-26T22:20:49Z

@blueorangutan package

blueorangutan · 2025-02-26T22:22:03Z

@Pearl1594 a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-02-26T23:30:07Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 12589

Pearl1594 · 2025-03-17T17:16:09Z

@blueorangutan test

blueorangutan · 2025-03-17T17:18:04Z

@Pearl1594 a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

blueorangutan · 2025-03-17T17:20:44Z

[SF] Trillian Build Failed (tid-12732)

shwstppr · 2025-04-16T10:02:42Z

@blueorangutan package

blueorangutan · 2025-04-16T10:04:04Z

@shwstppr a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-04-16T11:38:14Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 13071

github-actions · 2025-04-25T09:38:21Z

This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch.

rohityadavcloud · 2025-05-09T05:34:55Z

@BartJM pl address merge conflicts

sureshanaparti · 2025-06-05T09:48:49Z

@blueorangutan package

blueorangutan · 2025-06-05T09:50:04Z

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2025-06-05T11:57:50Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13627

blueorangutan · 2025-06-09T14:37:46Z

Packaging result [SF]: ✖️ el8 ✖️ el9 ✖️ debian ✖️ suse15. SL-JID 13672

boring-cyborg bot added component:networking component:secondary-storage component:ssvm labels Sep 23, 2024

weizhouapache reviewed Sep 23, 2024

View reviewed changes

engine/schema/src/main/java/com/cloud/vm/dao/ConsoleProxyDaoImpl.java Outdated Show resolved Hide resolved

DaanHoogland reviewed Sep 23, 2024

View reviewed changes

engine/schema/src/main/java/com/cloud/vm/dao/UserVmDetailsDaoImpl.java Show resolved Hide resolved

weizhouapache approved these changes Sep 24, 2024

View reviewed changes

engine/schema/src/main/java/com/cloud/vm/dao/UserVmDetailsDaoImpl.java Outdated Show resolved Hide resolved

engine/schema/src/main/java/com/cloud/vm/dao/UserVmDetailsDaoImpl.java Show resolved Hide resolved

BartJM force-pushed the systemvm-save-template-details branch from 9d3550d to e331976 Compare September 27, 2024 09:30

boring-cyborg bot added component:compute component:orchestration labels Sep 27, 2024

DaanHoogland approved these changes Sep 27, 2024

View reviewed changes

DaanHoogland added the status:needs-testing label Sep 27, 2024

Pearl1594 added this to ACS 4.20.1 Mar 17, 2025

Pearl1594 moved this to In Progress in ACS 4.20.1 Mar 17, 2025

github-actions bot added the status:has-conflicts label Apr 25, 2025

Save systemvm template details for cpvm, ssvm and vr's.

69d2347

BartJM force-pushed the systemvm-save-template-details branch from e331976 to 69d2347 Compare May 22, 2025 15:18

github-actions bot removed the status:has-conflicts label May 22, 2025

Pearl1594 removed this from ACS 4.20.1 Jun 3, 2025

Pearl1594 modified the milestones: 4.20.1, 4.20.2 Jun 3, 2025

Save systemvm template details for cpvm, ssvm and vr's. #9721

Are you sure you want to change the base?

Save systemvm template details for cpvm, ssvm and vr's. #9721

Uh oh!

Conversation

BartJM commented Sep 23, 2024

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

boring-cyborg bot commented Sep 23, 2024

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Sep 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

weizhouapache commented Sep 24, 2024

Uh oh!

blueorangutan commented Sep 24, 2024

Uh oh!

blueorangutan commented Sep 24, 2024

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

DaanHoogland commented Sep 27, 2024

Uh oh!

blueorangutan commented Sep 27, 2024

Uh oh!

BartJM commented Sep 27, 2024

Uh oh!

blueorangutan commented Sep 27, 2024

Uh oh!

DaanHoogland commented Sep 27, 2024

Uh oh!

blueorangutan commented Sep 27, 2024

Uh oh!

blueorangutan commented Sep 28, 2024

Uh oh!

DaanHoogland commented Sep 30, 2024

Uh oh!

DaanHoogland commented Sep 30, 2024

Uh oh!

blueorangutan commented Sep 30, 2024

Uh oh!

blueorangutan commented Sep 30, 2024

Uh oh!

DaanHoogland commented Sep 30, 2024

Uh oh!

blueorangutan commented Sep 30, 2024

Uh oh!

blueorangutan commented Oct 1, 2024

Uh oh!

Pearl1594 commented Feb 18, 2025

Uh oh!

blueorangutan commented Feb 18, 2025

Uh oh!

blueorangutan commented Feb 19, 2025

Uh oh!

blueorangutan commented Feb 19, 2025

Uh oh!

blueorangutan commented Feb 19, 2025

Uh oh!

blueorangutan commented Feb 19, 2025

Uh oh!

Pearl1594 commented Feb 26, 2025

Uh oh!

blueorangutan commented Feb 26, 2025

codecov bot commented Sep 23, 2024 •

edited

Loading