| Test name | Step instructions | Expected result | pass/fail |
|---|
| $Name | {what a tester should do} | {what a tester should see when they do that} | pass/fail |
@@ -53,20 +54,6 @@ Smoke tests are limited to core functionality and serve as a pre-release final r
3. forget password link prompts for email
4. valid credentials result in a successful login.
5. valid sso credentials result in a successful loginpass/fail |
-| Query flow | Create, edit, run, and delete queries. |
-
-1. permissions regarding creating/editing/deleting queries are up to date with documentation
-2. syntax errors result in error messaging
-3. queries can be run manually
- | pass/fail |
-| Host Flow | Verify a new host can be added and removed following modal instructions using your own device. |
-
-1. Host is added via command line
-2. Host serial number and date added are accurate
-3. Host is not visible after it is deleted
-4. Warning and informational modals show when expected and make sense
- | pass/fail |
-
| Packs flow | Verify management, operation, and logging of ["2017 packs"](https://fleetdm.com/handbook/company/why-this-way#why-does-fleet-support-query-packs). |
1. Packs successfully run on host machines after migrations
@@ -82,16 +69,17 @@ Smoke tests are limited to core functionality and serve as a pre-release final r
2. Software, query, policy, and packs logs are successfully sent to Filesystem log destinations
| pass/fail |
+| OS settings | Verify OS settings functionality |
-
- |
| My device page | Verify the end user's my device page loads successfully. |
-
-1. Clicking the Fleet desktop item, then "My device" successfully loads the my device page.
-2. The "My device" page is populated correctly and as expected.
-3. Styling and padding appears correct.
-
+1. Verify able to configure Disk encryption (macOS, Windows, & Linux).
+2. Verify host enrolled with Disk encryption enforced successfully encrypts.
| pass/fail |
+
+### MDM
+
+| Test name | Step instructions | Expected result | pass/fail |
|---|
+| $Name | {what a tester should do} | {what a tester should see when they do that} | pass/fail |
| MDM enrollment flow | Verify MDM enrollments, run MDM commands |
1. Erase an ADE-eligible macOS host and verify able to complete automated enrollment flow.
@@ -106,45 +94,22 @@ Smoke tests are limited to core functionality and serve as a pre-release final r
3. Turn off MDM on a non ADE-eligible macOS host.
4. On the My device page, follow the "Turn on MDM" instructions and verify that MDM is turned on.
| pass/fail |
-
-| Scripts | Verify script library and execution |
-
-1. Verify able to run a script on all host types from CLI.
-2. Verify scripts library upload/download/delete.
-3. From Host details (macOS, Windows, & Linux) run a script that should PASS, verify.
-4. From Host details (macOS, Windows, & Linux) run a script that should FAIL, verify.
-5. Verify UI loading state and statuses for scripts.
-6. Disable scripts globally and verify unable to run.
-7. Verify scripts display correctly in Activity feed.
- | pass/fail |
-
-| Software | Verify software library and install / download |
-
-1. Verify software library upload/download/delete.
-2. From Host details (macOS, Windows, & Linux) run an install that should PASS, verify.
-3. From My Device (macOS, Windows, & Linux) software tab should have self-service items available, verify.
-4. Verify UI loading state and statuses for installing software.
-6. Verify software installs display correctly in Activity feed.
- | pass/fail |
-
| OS settings | Verify OS settings functionality |
-1. Verify able to configure Disk encryption (macOS, Windows, & Linux).
-2. Verify host enrolled with Disk encryption enforced successfully encrypts.
-3. Verify Profiles upload/download/delete (macOS & Windows).
-4. Verify Profiles are delivered to host and applied.
+1. Verify Profiles upload/download/delete (macOS & Windows).
+2. Verify Profiles are delivered to host and applied.
| pass/fail |
| Setup experience | Verify macOS Setup experience |
1. Configure End user authentication.
-2. Upload a Bootstrap package.
-3. Add software (FMA, VPP, & Custom pkg)
-4. Add a script
-5. Enroll an ADE-eligible macOS host and verify successful authentication.
-6. Verify Bootstrap package is delivered.
-7. Verify SwiftDialogue window displays.
-8. Verify software installs and script runs.
+3. Upload a Bootstrap package.
+4. Add software (FMA, VPP, & Custom pkg)
+5. Add a script
+6. Enroll an ADE-eligible macOS host and verify successful authentication.
+7. Verify Bootstrap package is delivered.
+8. Verify SwiftDialogue window displays.
+9. Verify software installs and script runs.
| pass/fail |
| OS updates | Verify OS updates flow |
@@ -161,6 +126,7 @@ Smoke tests are limited to core functionality and serve as a pre-release final r
3. Verify Profiles are delivered to host and applied.
4. Verify VPP apps install & display correctly in Activity feed.
+ | pass/fail |
| Certificates Upload | APNs cert and ABM token renewal workflow |
1. Renew APNs Certificate.
@@ -168,18 +134,70 @@ Smoke tests are limited to core functionality and serve as a pre-release final r
3. Ensure ADE hosts can enroll.
| pass/fail |
+
+
+### Software
+
+| Test name | Step instructions | Expected result | pass/fail |
|---|
+| $Name | {what a tester should do} | {what a tester should see when they do that} | pass/fail |
+| Query flow | Create, edit, run, and delete queries. |
+
+1. permissions regarding creating/editing/deleting queries are up to date with documentation
+2. syntax errors result in error messaging
+3. queries can be run manually
+ | pass/fail |
+| Host Flow | Verify a new host can be added and removed following modal instructions using your own device. |
+
+1. Host is added via command line
+2. Host serial number and date added are accurate
+3. Host is not visible after it is deleted
+4. Warning and informational modals show when expected and make sense
+ | pass/fail |
+| My device page | Verify the end user's my device page loads successfully. |
+
+1. Clicking the Fleet desktop item, then "My device" successfully loads the my device page.
+2. The "My device" page is populated correctly and as expected.
+3. Styling and padding appears correct.
+
+ | pass/fail |
+| Scripts | Verify script library and execution |
+
+1. Verify able to run a script on all host types from CLI.
+2. Verify scripts library upload/download/delete.
+3. From Host details (macOS, Windows, & Linux) run a script that should PASS, verify.
+4. From Host details (macOS, Windows, & Linux) run a script that should FAIL, verify.
+5. Verify UI loading state and statuses for scripts.
+8. Disable scripts globally and verify unable to run.
+9. Verify scripts display correctly in Activity feed.
+ | pass/fail |
+
+| Software | Verify software library and install / download |
+
+1. Verify software library upload/download/delete.
+2. From Host details (macOS, Windows, & Linux) run an install that should PASS, verify.
+3. From My Device (macOS, Windows, & Linux) software tab should have self-service items available, verify.
+4. Verify UI loading state and statuses for installing software.
+7. Verify software installs display correctly in Activity feed.
+ | pass/fail |
+
+
| Migration Test | Verify Fleet can migrate to the next version with no issues. |
Using the migration scripts located in fleet/test/upgrade/
1. Run the upgrade_test.go script using the most recent stable version of Fleet and `main`.
2. Upgrade test returns an 'OK' response.
| pass/fail |
-
+
+
+### All Product Groups
+