diff --git a/.github/ISSUE_TEMPLATE/smoke-tests.md b/.github/ISSUE_TEMPLATE/smoke-tests.md new file mode 100644 index 0000000000..fb9d454adf --- /dev/null +++ b/.github/ISSUE_TEMPLATE/smoke-tests.md @@ -0,0 +1,67 @@ +--- +name: Release QA +about: Checklist of required tests prior to release +title: '' +labels: '' +assignees: '' + +--- + +# Goal: easy-to-follow test steps for sanity checking a release manually + +**Fleet version** (Head to the "My account" page in the Fleet UI or run `fleetctl version`): + +**Web browser** _(e.g. Chrome 88.0.4324)_: + +# Important reference data + +1. [fleetctl preview setup](https://fleetdm.com/get-started) +2. [permissions documentation](https://fleetdm.com/docs/using-fleet/permissions) + +# Smoke Tests +Smoke tests are limited to core functionality and serve as a sanity test. If smoke tests are failing, a release cannot proceed. + +## Prerequisites + +1. `fleetctrl preview` is set up and running the desired test version using `--tag` parameters. +2. Unless you are explicitly testing older browser versions, browser is up to date. +3. Certificate & flagfile are in place to create new host. + +## Instructions + +
| Test name | Step instructions | Expected result | pass/fail |
|---|---|---|---|
| $Name | {what a tester should do} | {what a tester should see when they do that} | pass/fail |
| Update flow | + +1. Prereq: previous version is set up with data. +2. Update fleet to testing version + | No data should be lost in update | pass/fail |
| Login flow | + +1. navigate to the login page and attempt to login with both valid and invalid credentials to verify some combination of expected results. +2. Login with SSO + | + +1. text fields prompt when blank +2. correct error message is "authentication failed" +3. forget password link prompts for email +4. valid credentials result in a successful login. | pass/fail |
| Query flow | Create, edit, run, and delete queries. | + +1. permissions regarding creating/editing/deleting queries are up to date with documentation +2. syntax errors result in error messaging +3. queries can be run manually + | pass/fail |
| Host Flow | Verify a new host can be added and removed following modal instructions using your own device. | + +1. Host is added via command line +2. Host serial number and date added are accurate +3. Host is not visible after it is deleted +4. Warning and informational modals show when expected and make sense + | pass/fail |