For #26218
- Added `GET /api/_version_/fleet/android_enterprise` andpoint and tests
- Set up some testing infrastructure for Android service tests -- see
new README.md
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #23750
# Overview
This PR adds a basic tool for creating and restoring Fleet dev
snapshots. In this first iteration a snapshot is just a folder
containing a MySQL db dump made using the existing backup/restore
scripts, and the tool allows you to easily save and restore snapshots
interactively.
## Usage
* `make snapshot` to create a new snapshot
* `make restore` to select and restore a snapshot
## Future plans
Future iterations can add metadata to snapshots to integrate things
like:
* node keys from osquery-perf, so you can easily reconnect to hosts
created in a previous session
* env vars from when the snapshot was made
* the branch from when the snapshot was made, to allow switching to that
branch and restarting the server as part of the restore process
*
## Demo
https://github.com/user-attachments/assets/1590c37a-3df9-4201-a42b-ccd1a36cb6cf
For #25748.
Manually tested by:
1. Building a `fleet` executable for Linux on a Ubuntu VM (with
`-extldflags "-static"`) for Alpine.
2. Placing the executable in `tools/fleet-docker`.
3. Building a local docker image using `docker build -t
fleetdm/fleet:v42.42.42 --platform=linux/amd64 .`.
4. Running the docker image (using `docker run`) and use Fleet on the
browser.
```
docker run -v $(pwd)/../osquery:/run -p 8412:8412 -e FLEET_MYSQL_ADDRESS=host.docker.internal:3306 -e FLEET_MYSQL_DATABASE=fleet -e FLEET_MYSQL_USERNAME=fleet -e FLEET_MYSQL_PASSWORD=insecure -e FLEET_REDIS_ADDRESS=host.docker.internal:6379 -e FLEET_SERVER_ADDRESS=0.0.0.0:8412 -e FLEET_SERVER_CERT=/run/fleet.crt -e FLEET_SERVER_KEY=/run/fleet.key -e FLEET_LOGGING_JSON='true' -e FLEET_VULNERABILITIES_CURRENT_INSTANCE_CHECKS="yes" -e FLEET_VULNERABILITIES_DATABASES_PATH=/vulndb -e FLEET_LOGGING_DEBUG='true' -it fleetdm/fleet:v42.42.42
```
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
# Checklist for submitter
- [X] Manual QA for all new/changed functionality
# Details
This PR adds two new user-facing `make` targets:
* `run-go-tests`: run Go tests for one or more packages, optionally
filtering to specific tests
* `debug-go-tests`: debug (using Delve) Go tests for one or more
packages, optionally filtering to specific tests
Example usage:
```
# Run all tests in the mysql and gdmf packages
make run-go-tests PKG_TO_TEST="server/mdm/apple/gdmf server/datastore/mysql"
```
```
# Run all the TestMDMApple tests in the mysql package
make run-go-tests PKG_TO_TEST=server/datastore/mysql TESTS_TO_RUN="^TestMDMApple\$$"
```
```
# Run only the TestMDMAppleProfileLabels test in the mysql package
make run-go-tests PKG_TO_TEST=server/datastore/mysql TESTS_TO_RUN="^TestMDMApple\$$/^TestMDMAppleProfileLabels\$$"
```
# Notes
Two new "private" targets `.run-go-tests` and `.debug-go-tests` were
created as base commands for both `test-go` (used in CI) and the new
user-facing commands.
#22740
Full QA is still a WIP but this is ready for review.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#22424, #22705
TODO: integration test updates
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
#22115, #22116
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
No changes file, as FE changes file covers the entire feature
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Tim Lee <timlee@fleetdm.com>
Also adds a line in the makefile help for generate-doc, as it took me
way too long to find that command this time.
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
#20571
## Summary of changes
We have a few moving parts in fleetctl land (`fleetdm/wix` is used to
build `msi`s and `fleetdm/bomutils` is used to build `pkg`s, and
`fleetdm/fleetctl` can be used to build packages using docker, no need
for fleetctl executable):
```mermaid
graph LR
fleetctl_exec[fleetctl<br>executable];
wix_image[fleetdm/wix<br>docker image];
bomutils_image[fleetdm/bomutils<br>docker image];
fleetctl_image[fleetdm/fleetctl<br>docker image];
fleetctl_exec -- uses --> wix_image;
fleetctl_image -- COPY dependencies<br>FROM --> wix_image;
fleetctl_exec -- uses --> bomutils_image;
fleetctl_image -- COPY dependencies<br>FROM --> bomutils_image;
```
So, we'll need to update the three images: `fleetdm/bomutils`,
`fleetdm/wix` & `fleetdm/fleetctl`.
- `tools/bomutils-docker/Dockerfile`, `tools/wix-docker/Dockerfile` and
`tools/fleetctl-docker/Dockerfile`: Updating the base image to fix the
CRITICAL vulnerabilities.
- Modified existing+unused
`.github/workflows/build-and-check-fleetctl-docker-and-deps.yml` to run
every day to check for CRITICAL vulnerabilities in `fleetdm/wix`,
`fleetdm/bomutils` and `fleetdm/fleetctl`.
- `.github/workflows/goreleaser-fleetctl-docker-deps.yaml`:
`fleetdm/bomutils` and `fleetdm/wix` were pushed manually a few years
ago (most likely by Zach), so I've added a new action to release them
when we have changes to release (like now). It will basically release
`fleetctl/bomutils` and `fleetdm/wix` when pushing a tag of the form
`fleetctl-docker-deps-*` (we'll need to protect such tag prefix).
- Changes in `.github/workflows/test-native-tooling-packaging.yml` to
build `fleetdm/bomutils` and `fleetdm/wix` for `fleetdm/fleetctl` to use
them instead of the ones in docker hub.
--
Build before upgrading `debian:stable-slim`:
https://github.com/fleetdm/fleet/actions/runs/10255391418/job/28372231837
Build after upgrading `debian:stable-slim`:
https://github.com/fleetdm/fleet/actions/runs/10255550034
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
back-end and agent part of #13157
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#18766
Fixed a bug where updating user via `/api/v1/fleet/users/:id` endpoint
sometimes did not update the activity feed and returned the un-updated
user object.
You must use a DB configuration with a replica to reproduce the issue.
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
Adds a minimum supported node and yarn version to the project.
Currently if you are on an unsupported version of node or yarn, there is
no messaging telling you that is the issue. The build just fails, and
you are left to figure out it's because of your node version. With this
change, it will be much clearer why any of the node required commands
(e.g. make deps, make generate-dev, make lint-js, make test-js) are not
working, and it will tell you exactly which minimum version of node or
yarn you need.
**After the console error is clear about using an unsupported node
version**
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Manual QA for all new/changed functionality
Should tackle #14026.
This will run a daily Github action and create a PR if there's a new
update in our TUF on `edge` or `stable`.
E.g. somebody releases 1.22.0 fleetd to `stable` on our TUF and the next
day this automation runs and will create a PR that updates the versions
in `orbit/TUF.md` (or they can run the workflow manually).
Am happy to amend the shape of `orbit/TUF.md` (or we can iterate later).
#16076
This change removes ineffective rate limit to `/api/fleet/device/ping`
and `api/fleet/orbit/ping`.
Currently these endpoints are not rate limited, because the rate
limiting used in these was the `errorLimiter` which only takes effect if
the request fails and the ping endpoints never fail. So... we were
making ineffective Redis accesses on every `/api/fleet/device/ping` and
`api/fleet/orbit/ping` requests (we use Redis as the limiter store).
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
#15560
Probably best to review commit by commit.
First commit adds the mockimpl files, second commit amends README.md and
third commit fixes golangci-lint issues.
- [X] Manual QA for all new/changed functionality
Tested by adding a dummy method to service.go and running `make
generate-mock`.
---------
Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
#15556
We will need to pay attention when releasing fleet (the github actions
were modified to use the local file now).
Should be reviewed by commits (first commit is the actual adding of the
`version.go` file)
- [X] Manual QA for all new/changed functionality
Manually tested the following:
- `Settings -> My account` on the UI and checked the `/version` endpoint
response. (Or also visiting https://localhost:8080/version on a
browser).
- Ran `make fleetctl fleet`, `./build/fleetctl --version` and
`./build/fleet version`.
#15555
Probably the best way to review this is commit by commit:
- First commit does the actual moving.
- Second commit fixes golangci-lint issues (in the least effort way to
avoid refactoring or rearrangement of some of the code).
- Third commit moves a printf to before the migration step is executed.
In the past some customers hitting migration issues (like migration
steps hanging or taking long to execute) and wanted to know which one
was it. The only way to know was to look at the repository and looking
for the next migration after the last one logged.
Checks:
- [X] Manual QA for all new/changed functionality
Manual tests:
- `make fleet && make db-reset`.
- Adding a new migration via `make migration name=Foobar` and then
running `./build/fleet prepare db`.
- Enrolling a new device to Fleet (smoke test).
Related to: #10209
Changes:
- Updated the go test in the Makefile to have the `--codepkg` flag.
- Added a newline to the `test-go` GH workflow to trigger a run for this
PR
> Note: I'm creating this as a draft PR to see the results of the "Test
Go" workflow
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
---------
Co-authored-by: George Karr <gkarr@xerithas-mac-16.local>
- Adds a `universal` method for building the fleetd tables for darwin
- Sets `.exe` for the windows extension
- Adds an `all` method for building all fleetd tables
Related to #11534 this is an extract from the code I used to build a
prototype to see if `swiftDialog` would work for us.
This is very similar to the work we did for Nudge previously.
Upgrades webpack and other JS dependencies. This was primarily motivated
by GitHub reporting a vulnerability in Webpack (which shouldn't actually
effect our use of Webpack) and wanting to clean up some tech debt.
Note that equivalent functionality for url-loader and file-loader is now
included in webpack itself.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Manual QA for all new/changed functionality
#8948
- Add more go:generate commands for MDM mocks
- Add unit and integration tests for MDM code
- Move interfaces from their PoC location to match existing patterns
