mirror of
https://github.com/fleetdm/fleet
synced 2026-05-23 08:58:41 +00:00
Update manual qa document (#2124)
* Pack location Future-proofing instructions for ease of onboards by adding location information. * Label queries are immutable (addition) Previous document made it seem users should be able to edit label queries. * removal of all non-preview tasks Non-preview tasks will be moved to /docs/3-Contributing/2-Testing.md. Additional changes include clarifications to the testing steps. * update link * Update manual-qa.md
This commit is contained in:
Renee Jackson
GitHub
parent
2505b68cb8
commit
e5452f4e4d
1 changed files with 10 additions and 116 deletions
|
|
@ -2,63 +2,31 @@
|
|||
|
||||
This living document outlines the manual quality assurance process conducted to ensure each release of Fleet meets organization standards.
|
||||
|
||||
All steps should be conducted during each QA pass.
|
||||
All steps should be conducted during each QA pass. All steps are possible with `fleetctrl preview`.
|
||||
|
||||
As new features are added to Fleet, new steps and flows will be added.
|
||||
|
||||
## Collecting bugs
|
||||
|
||||
The goal of manual QA is to catch unexpected behavior prior to release.
|
||||
The goal of manual QA is to catch unexpected behavior prior to release. All Manual QA steps should be possible using `fleetctl preview`. Please refer to [docs/03-Contributing/02-Testing.md](https://github.com/fleetdm/fleet/blob/main/docs/03-Contributing/02-Testing.md) for flows that cannot be completed using `fleetctl preview`.
|
||||
|
||||
Please start the manual QA process by creating a blank GitHub issue. As you complete each of the flows, record a list of the bugs you encounter in this new issue. Each item in this list should contain one sentence describing the bug and a screenshot if the item is a frontend bug.
|
||||
|
||||
## Fleet UI
|
||||
|
||||
### Clear your local MySQL database
|
||||
|
||||
Before you fire up your local Fleet server, wipe your local MySQL database by running the following command:
|
||||
|
||||
```
|
||||
docker volume rm fleet_mysql-persistent-volume
|
||||
```
|
||||
|
||||
If you receive an error that says "No such volume," double check that the MySQL volume doesn't have a different name by running this command:
|
||||
|
||||
```
|
||||
docker volume ls
|
||||
```
|
||||
|
||||
### Start your development server
|
||||
|
||||
Next, fire up your local Fleet server. Check out [this Loom video](https://www.loom.com/share/e7439f058eb44c45af872abe8f8de4a1) for instructions on starting up your local development environment.
|
||||
For all following flows, please refer to the [permissions documentation](https://fleetdm.com/docs/using-fleet/permissions) to ensure that actions are limited to the appropriate user type. Any users with access beyond what this document lists as availale should be considered a bug and reported for either documentation updates or investigation.
|
||||
|
||||
### Set up flow
|
||||
|
||||
Successfully set up Fleet.
|
||||
Successfully set up `fleetctl preview` using the preview steps outlined [here](https://fleetdm.com/get-started)
|
||||
|
||||
### Login and logout flow
|
||||
|
||||
Successfully logout and then login to your local Fleet.
|
||||
|
||||
### Enroll host flow
|
||||
|
||||
Enroll your local machine to Fleet. Check out the [Orbit for osquery documentation](https://github.com/fleetdm/orbit#orbit-osquery) for instructions on generating and installing an Orbit package.
|
||||
|
||||
### Host page
|
||||
|
||||
To populate the Fleet UI with more than just one host you'll need to use the [fleetdm/osquery-perf tool](https://github.com/fleetdm/osquery-perf/tree/629a7efb6097f9108f706ccd45828793ff73cf9c).
|
||||
|
||||
First, clone the fleetdm/osquery perf repo and then run the following commands from the top level of the cloned directory:
|
||||
|
||||
```
|
||||
go run agent.go --host_count 200 --enroll_secret <your enroll secret goes here>
|
||||
```
|
||||
|
||||
After about 10 seconds, the Fleet UI should be populated with 200 simulated hosts.
|
||||
|
||||
### Host details page
|
||||
|
||||
Select a host from the "Hosts" table as a global user with the Maintainer role.
|
||||
Select a host from the "Hosts" table as a global user with the Maintainer role. You may create a user with a fake email for this purpose.
|
||||
|
||||
You should be able to see and select the "Delete" button on this host's **Host details** page.
|
||||
|
||||
|
|
@ -70,7 +38,9 @@ You should be able to see and select the "Query" button on this host's **Host de
|
|||
|
||||
Create a new label by selecting "Add a new label" on the Hosts page. Make sure it correctly filters the host on the hosts page.
|
||||
|
||||
Edit this label and then delete this label.
|
||||
Edit this label. Confirm users can only edit the "Name" and "Description" fields for a label. Users cannot edit the "Query" field because label queries are immutable.
|
||||
|
||||
Delete this label.
|
||||
|
||||
### Query flow
|
||||
|
||||
|
|
@ -86,88 +56,12 @@ Edit this query and then delete this query.
|
|||
|
||||
`Flow is covered by e2e testing`
|
||||
|
||||
Create a new pack.
|
||||
Create a new pack (under Schedule/advanced).
|
||||
|
||||
Add a query as a saved query to the pack. Remove this query. Delete the pack.
|
||||
|
||||
### Organization settings flow
|
||||
|
||||
As an admin user, select the "Settings" tab in the top navigation and then select "Organization settings".
|
||||
|
||||
Follow [the instructions outlined in the Testing documentation](../docs/03-Contributing/02-Testing.md#email) to set up a local SMTP server.
|
||||
|
||||
Successfully edit your organization's name in Fleet.
|
||||
|
||||
### Manage users flow
|
||||
|
||||
Invite a new user. To be able to invite users, you must have your local SMTP server configured. Instructions for setting up a local SMTP server are outlined in [the Testing documentation](../docs/03-Contributing/02-Testing.md#email)
|
||||
|
||||
Logout of your current admin user and accept the invitation for the newly invited user. With your local SMTP server configured, head to https://localhost:8025 to view and select the invitation link.
|
||||
|
||||
### Agent options flow
|
||||
|
||||
Head to the global agent options page and set the `distributed_iterval` field to `5`.
|
||||
|
||||
Refresh the page to confirm that the agent options have been updated.
|
||||
|
||||
### My account flow
|
||||
|
||||
Head to the My account page by selecting the dropdown icon next to your avatar in the top navigation. Select "My account" and successfully update your password.
|
||||
Head to the My account page by selecting the dropdown icon next to your avatar in the top navigation. Select "My account" and successfully update your password. Please do this with an extra user created for this purpose to maintain accessibility of `fleetctl preview` admin user.
|
||||
|
||||
## `fleetctl` CLI
|
||||
|
||||
### Set up flow
|
||||
|
||||
Successfully set up Fleet by running the `fleetctl setup` command.
|
||||
|
||||
You may have to wipe your local MySQL database in order to successfully set up Fleet. Check out the [Clear your local MySQL database](#clear-your-local-mysql-database) section of this document for instructions.
|
||||
|
||||
### Login and logout flow
|
||||
|
||||
Successfully login by running the `fleetctl login` command.
|
||||
|
||||
Successfully logout by running the `fleetctl logout` command. Then, log in again.
|
||||
|
||||
### Hosts
|
||||
|
||||
Run the `fleetctl get hosts` command.
|
||||
|
||||
You should see your local machine returned. If your host isn't showing up, you may have to reenroll your local machine. Check out the [Orbit for osquery documentation](https://github.com/fleetdm/fleet/blob/main/orbit/README.md) for instructions on generating and installing an Orbit package.
|
||||
|
||||
### Query flow
|
||||
|
||||
Apply the standard query library by running the following command:
|
||||
|
||||
`fleetctl apply -f docs/1-Using-Fleet/standard-query-library/standard-query-library.yml`
|
||||
|
||||
Make sure all queries were successfully added by running the following command:
|
||||
|
||||
`fleetctl get queries`
|
||||
|
||||
Run the "Get the version of the resident operating system" query against your local machine by running the following command:
|
||||
|
||||
`fleetctl query --hosts <your-local-machine-here> --query-name "Get the version of the resident operating system"`
|
||||
|
||||
### Pack flow
|
||||
|
||||
Apply a pack by running the following commands:
|
||||
|
||||
`fleetctl apply -f docs/1-Using-Fleet/configuration-files/multi-file-configuration/queries.yml`
|
||||
|
||||
`fleetctl apply -f docs/1-Using-Fleet/configuration-files/multi-file-configuration/pack.yml`
|
||||
|
||||
Make sure the pack was successfully added by running the following command:
|
||||
|
||||
`fleetctl get packs`
|
||||
|
||||
### Organization settings flow
|
||||
|
||||
Apply organization settings by running the following command:
|
||||
|
||||
`fleetctl apply -f docs/1-Using-Fleet/configuration-files/multi-file-configuration/organization-settings.yml`
|
||||
|
||||
### Manage users flow
|
||||
|
||||
Create a new user by running the `fleetctl user create` command.
|
||||
|
||||
Logout of your current user and log in with the newly created user.
|
||||
|
|
|
|||
Loading…
Reference in a new issue