Test scenarios for validating Metric Profile API

[chandrams]

Define test scenarios for validating Metric profile API

• Create a perf profile with a valid yaml using kubectl, list the perf profile and validate the profile

• Create a perf profile with a valid json using kubectl, list the perf profile and validate the profile

• Create a perf profile with a valid json using the API, list the perf profile and validate the profile

• Create multiple perf profiles using the API and list the perf profiles

• Create the perf profile twice, validate for appropriate error message

• Create a perf profile with an invalid yaml using kubectl, validate for appropriate error message

• Create a perf profile with an invalid json using kubectl, validate for appropriate error message

• Create a perf profile with an invalid json using API, validate for appropriate error message

• Create a perf profile with a valid yaml using kubectl, use the same perf profile json equivalent and create a perf profile using the API - Will this create a perf profile again?

• List perf profiles without creating any profiles

• Is list perf profiles taking a profile name as input?

• Update perf profile by updating the below in each test scenario, ensure updates to only allowed fields work and appropriate error message is thrown. Validate the updated metric profile using listMetricProfiles and ensure recommendations are generated using the updated queries.

  • Update a new metric name
  • Update only the query of an existing metric name
  • Update a new aggregation to an existing metric name
  • Update version & query
  • Update a metric profile that is in use
  • Update a metric profile that is not in use
    (What is the behavior when we update a metric profile in use? It shouldn't affect the experiments using it. New queries should be used in the subsequent call to generate recommendations)

• Delete the specified metric profile and validate it using list. Is deleteMetricProfile taking the name as input?

• Delete a metric profile that is in use

• Delete a non-existing metric profile

• Validate the metric profiles are loaded again from the DB on kruize restart

[shreyabiradar07]

By extending the /listMetricProfiles API to support name and verbose query parameters below are the additional test scenarios to be considered

• List metric profiles without passing any query parameters
• List metric profiles passing profile name query parameter
• List metric profiles passing verbose query parameter
• List metric profiles passing profile name and verbose query parameters
• List metric profiles passing invalid query parameters other than name and verbose