Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scale tests #916

Open
wants to merge 34 commits into
base: main
Choose a base branch
from
Open

Add scale tests #916

wants to merge 34 commits into from

Conversation

Zlaticanin
Copy link
Contributor

@Zlaticanin Zlaticanin commented Sep 12, 2024

This PR allows for testing a number of concurrently installed VDS, VSS and VPS CRs. You can specify the number by type with the env vars: VDS_CREATE_COUNT, VSS_CREATE_COUNT, and VPS_CREATE_COUNT. You can further specify different combos, such as VSS_KVV1_CREATE, VSS_KVV2_CREATE, VSS_BOTH_CREATE, etc..
If none specified, the default count will be used (previous behavior).

Prereq: Doormat AWS sandbox account

Before running any make commands, make sure to log in into doormat and export the AWS keys:
doormat login && eval $(doormat aws -a <AWS ACCOUNT NAME> export)

Make sure to export the path to your vault license:
export TF_VAR_vault_license_path=<path to license file>

make -f scale-testing.mk create-eks
make -f scale-testing.mk deploy-workload
make -f scale-testing.mk scale-tests

@Zlaticanin Zlaticanin marked this pull request as ready for review September 18, 2024 16:40
@Zlaticanin Zlaticanin requested a review from a team as a code owner September 18, 2024 16:40
Copy link
Contributor

@thyton thyton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good from the first pass. Appreciate your hard work on this!

test/integration/infra/scale-testing/eks-cluster/main.tf Outdated Show resolved Hide resolved
test/integration/integration_test.go Outdated Show resolved Hide resolved
test/integration/operator/providers/eks.tf Outdated Show resolved Hide resolved
test/integration/operator/providers/kind.tf Outdated Show resolved Hide resolved
test/integration/operator/terraform/variables.tf Outdated Show resolved Hide resolved
test/integration/vaultstaticsecret_integration_test.go Outdated Show resolved Hide resolved
test/integration/modules/operator/outputs.tf Outdated Show resolved Hide resolved
scale-testing.mk Outdated Show resolved Hide resolved
scale-testing.mk Outdated Show resolved Hide resolved
@lursu lursu requested a review from thyton October 4, 2024 16:51
Copy link
Collaborator

@benashz benashz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Add some questions/comments for your consideration.

Thanks!

@@ -180,9 +193,23 @@ func TestMain(m *testing.M) {
os.Exit(1)
}

var providerFile string
if isScaleTest {
providerFile = "eks.tf"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the future, we might move the scale testing TF to its own directory, then we could just copy over the TF from it into the target TF working directory. That would allow is to extend the TF for a specific cluster target. Not a blocker.

test/integration/vaultdynamicsecret_integration_test.go Outdated Show resolved Hide resolved
test/integration/integration_test.go Outdated Show resolved Hide resolved
test/integration/vaultpkisecret_integration_test.go Outdated Show resolved Hide resolved
test/integration/vaultstaticsecret_integration_test.go Outdated Show resolved Hide resolved
scale-testing.mk Show resolved Hide resolved
scale-testing.mk Show resolved Hide resolved
scale-testing.mk Outdated Show resolved Hide resolved
@benashz benashz changed the title Vault 28156/add scale tests Add scale tests Oct 8, 2024
test/integration/vaultdynamicsecret_integration_test.go Outdated Show resolved Hide resolved
test/integration/vaultdynamicsecret_integration_test.go Outdated Show resolved Hide resolved
test/integration/vaultpkisecret_integration_test.go Outdated Show resolved Hide resolved
test/integration/vaultdynamicsecret_integration_test.go Outdated Show resolved Hide resolved
test/integration/vaultpkisecret_integration_test.go Outdated Show resolved Hide resolved
test/integration/vaultstaticsecret_integration_test.go Outdated Show resolved Hide resolved
scale-testing.mk Outdated Show resolved Hide resolved
scale-testing.mk Outdated Show resolved Hide resolved
scale-testing.mk Outdated Show resolved Hide resolved
scale-testing.mk Outdated Show resolved Hide resolved
@Zlaticanin Zlaticanin requested a review from benashz October 10, 2024 21:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants