System tests allow developers define scenarios and ensure datadog libraries produce consistent telemetry (that is, traces, metrics, profiles, etc...). This "edit" section addresses the following use-cases:
- Adding a new test (maybe to support a new or existing feature)
- Modifying an existing test, whether that's modifying the test client (test*.py files) or the weblog and/or parametric apps that serve the test client requests)
- Enabling/disabling tests for libraries under various conditions
Note: Anytime you make changes and open a PR, re-run the linter: format.md
To make changes, you must be able to run tests locally. Instructions for running end-to-end tests can be found here and for parametric, here.
Callout
You'll commonly need to run unmerged changes to your library against system tests (e.g. to ensure the feature is up to spec). Instructions for testing against unmerged changes can be found in enable-test.md.
- add-new-test.md: Add a new test
- scenarios.md: Add a new scenario
- format.md: Use the linter
- features.md: Mark tests for the feature parity dashboard
- enable-test.md: Enable a test
- skip-tests.md: Disable tests
- manifest.md: How tests are marked as enabled or disabled for libraries
- troubleshooting.md Tips for debugging
- iast-validations.md: Mark tests with vulnerabilities
- CI-and-scenarios.md: Understand how scenarios run in CI
- update-docker-images.md: Modify test app docker images
- remote-config.md: Write remote config tests