For #27284
This PR:
- Adds SCIM as a fallback for username during macOS end user
authentication during setup experience
- Adds SCIM/endUsers details to host details
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27287
This PR adds integration tests for SCIM API endpoints as well as some
bug fixes found by these tests.
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #27301
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated automated tests
- [X] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
# Details
This PR adds the ability to set/unset labels on policies via GitOps. It
builds on https://github.com/fleetdm/fleet/pull/27575 (back end for
policy labels) and updates the `PolicySpec` type and `ApplyPolicySpecs`
methods to update the `policy_labels` table where needed.
## Testing
1. Create a few labels in the UI
1. Create a global policy "foo" in the UI without labels
2. Create a global policy "bar" in the UI with labels
2. Create a global policy "baz" in the UI with labels
4. Use `fleetctl gitops` with a global .yml file, and under `policies:`
add "foo", "bar", "baz" and "boop".
* Add labels to "foo" with `labels_include_any:`
* Don't add `labels_include_any:` to "bar"
* Add labels to "baz" with `labels_include_any:`, but different labels
than what you added in the UI
* Add labels to "boop" with `labels_include_any:`
The expected outcome when viewing the queries in the UI (on the "edit
query" screen)
* Foo, Baz and Boop should have the labels specified in gitops
* Bar should have no labels
Repeat testing with _excluded_ labels.
---------
Co-authored-by: dantecatalfamo <dante.catalfamo@gmail.com>
Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com>
For #27287
Video explaining the PR: https://www.youtube.com/watch?v=ZHgFUAvrPEI
This PR adds SCIM Users support for Okta. The goal is to first add
Users/Groups support so that the remaining backend SCIM work can be done
in parallel.
This PR does not include the following, which will be added in later PRs
- Changes file
- Groups support for Okta
- Full support for Entra ID
- Integration tests
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26713
# Details
This PR updates Fleet and its related tools and binaries to use Go
version 1.24.1.
Scanning through the changelog, I didn't see anything relevant to Fleet
that requires action. The only possible breaking change I spotted was:
> As [announced](https://tip.golang.org/doc/go1.23#linux) in the Go 1.23
release notes, Go 1.24 requires Linux kernel version 3.2 or later.
Linux kernel 3.2 was released in January of 2012, so I think we can
commit to dropping support for earlier kernel versions.
The new [tools directive](https://tip.golang.org/doc/go1.24#tools) is
interesting as it means we can move away from using `tools.go` files,
but it's not a required update.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet
- [x] Orbit runs on macOS ✅ , Linux ✅ and Windows.
- [x] Manual QA must be performed in the three main OSs, macOS ✅,
Windows and Linux ✅.
> For issues found during QA
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
If the Fleet server is on a connection that can't pull ~150 Mbps down,
it'll time out before being able to add Microsoft Word for macOS, due to
the 100-second default timeout. This skips that behavior.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
> For #26662
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27235.
The added logging might help troubleshooting #27235 which seems to only
happen in our dogfood environment.
- [X] Manual QA for all new/changed functionality
> For #27548
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #26655
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
For #26082.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27035
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
## Details
This PR adds an `author_id` column to the `labels` table, and adds the
associated properties to the `Label` and `LabelSpec` types. When a new
label is created via the UI or API, an author ID is set on the label if
one can be inferred from the context. Otherwise, the author ID is set to
`null`.
## Authz and Automated testing
Additional backend authorization logic is introduced in a follow-on PR,
https://github.com/fleetdm/fleet/pull/27089, because rconciling all of
the test updates between this PR and
https://github.com/fleetdm/fleet/pull/27038 was getting complicated.
## Manual Testing
* Tested in the UI by creating a new label on the Hosts page
* Tested via Gitops by merging this branch with
https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops`
with a global config with `labels:` in it.
For #24473
This PR allows users to add / update / remove labels from queries via
Gitops.
## Testing
1. Create a few labels in the UI
1. Create a global query "foo" in the UI without labels
2. Create a global query "bar" in the UI with labels
2. Create a global query "baz" in the UI with labels
4. Use `fleetctl gitops` with a global .yml file, and under `queries:`
and "foo", "bar", "baz" and "boop".
* Add labels to "foo" with `labels_include_any:`
* Don't add `labels_include_any:` to "bar"
* Add labels to "baz" with `labels_include_any:`, but different labels
than what you added in the UI
* Add labels to "boop" with `labels_include_any:`
The expected outcome when viewing the queries in the UI (on the "edit
query" screen)
* Foo, Baz and Boop should have the labels specified in gitops
* Bar should have no labels
For #26622
This PR includes:
- Making DigiCert client a real service that can be overridden in tests
- GitOps support for DigiCert and Custom SCEP configs
# Checklist for submitter
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #27234.
- Improved logging in orbit to help us during troubleshooting.
- Added some documentation on how to grep for errors related to software
package installation in orbit.
- Added `took` to server request error logs (it was only present when
the request succeeds).
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For #24473
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
## Details
This PR adds the ability to manage labels via GitOps. Usage is as
follows:
* If a top-level `labels:` key is provided in the global YAML file
provided to GitOps, then any labels in this list will be created (if
using a new name) or updated (if using an existing name).
* If no top-level `labels:` key is provided, no changes will be made to
labels. This allows backwards-compatibility; customers won't blow away
all of their labels if they don't immediately use `labels:` in their
YAML
Additionally, some new validation has been added so that label usage is
checked prior to application. This means that when the gitops command is
run, it will verify that any labels referenced elsewhere in the YAML
(e.g. by software installers or mdm profiles) exist, and will bail with
an error message if they don't.
## Testing
**Test label deletion**
1. Add some labels via the UI
2. Run `fleetctl gitops --dry-run` with a default.yml file _without_
`labels:` in it, and verify that it doesn't say it will update or delete
any labels
2. Run `fleetctl gitops` with a default.yml file _without_ `labels:` in
it, and verify that it doesn't modify or remove your labels
4. Run `fleetctl gitops --dry-run` with a default.yml file with
`labels:` in it and nothing underneath, and verify that it says that it
will delete your labels
4. Run `fleetctl gitops` with a default.yml file with `labels:` in it
and nothing underneath, and verify that it removes all your labels
**Test label create/update**
1. Add a label "foo" via the UI
2. Run `fleetctl gitops --dry-run` with a default.yml file with two
`labels:` in it, one named "foo" and one named "bar". Verify that the
output says that one label will be created and one will be updated.
2. Run `fleetctl gitops` with a default.yml file with two `labels:` in
it, one named "foo" and one named "bar". Verify that the two labels now
exist in the UI with the configuration you specified.
**Test label usage**
1. Add a label "foo" in the UI.
1. Run `fleetctl gitops --dry-run` with a default.yml file _without_
`labels:` in it, where a software installer or mdm profile uses the
"foo" label via `labels_include_any`. Verify that the output doesn't
complain about unknown labels.
1. Run `fleetctl gitops --dry-run` with a default.yml file _with_
`labels:` in it with nothing underneath, and a software installer or mdm
profile uses the "foo" label via `labels_include_any`. Verify that the
output complains about unknown label "foo"
1. Run `fleetctl gitops --dry-run` with a default.yml file _with_
`labels:` in it with a "foo" label defined underneath, and a software
installer or mdm profile uses the "foo" label via `labels_include_any`.
Verify that the output doesn't complain about unknown labels.
For #26603
This PR includes:
- Refactoring of NDES/SCEP verify/timeout logic for easier testing (with
dependency injection)
- Custom SCEP configs
- saving/deleting/updating of encrypted custom SCEP challenges
- validation call to custom SCEP server to verify connection
- Custom SCEP activities
- unit and integration tests for all of the above
This PR does not include the following:
- Changes file (in later PR)
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #26083
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
https://github.com/fleetdm/fleet/issues/21300
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26977.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26743
This PR fixes an issue where using `fleetctl gitops` unsets the "gitops
mode" settings in the UI. The code which prepares the config spec to
send to the "modify config" endpoint deliberately copies over the
current app settings for gitops mode to facilitate this. I updated an
existing test to verify the new behavior.
For #26603
This PR includes:
- DigiCert configs
- Some config code for custom SCEP, but not fully functional
- Validation tests for DigiCert Configs
- DigiCert activities (manually tested)
This PR does not include the following:
- Encryption of DigiCert API tokens -- this requires DB migration (in
next PR)
- Making an HTTP call to DigiCert API for validation (in later PR)
- Integration tests (in later PR)
- Changes file (in later PR)
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
- Added `users_deleted` table to track user actions if the user was
actually deleted.
- Added enable/disable Android MDM activities
Note: I could not auto-generate fleet.Service mock because it has issues
with methods that don't return anything. I ended up using testify mock
instead.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
- Added `GET /api/_version_/fleet/android_enterprise/signup_sse`
endpoint and tests
- Fixed up handling of Android status reports with a deleted device. We
don't actually expect this to happen in production since the proxy
should delete the pubSub connection when the enterprise is deleted.
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26219
Video explaining the PR: https://www.youtube.com/watch?v=t2hmpE7N-50
This PR includes:
- applying a default/dummy policy to Android enterprise
- Setting up PubSub topic and subscription to point at Fleet Server
- Generating device enrollment token
- Enrolling a device
Although some tests were added, this PR is mostly manually tested.
Integration tests are an outstanding task.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> no issue. trying to make loadtests with vulnerable software more
accurate. these are the sw versions with most vulns from dogfood.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
## For #26230
- Add `gitops` settings to app config
- GET and PATCH endpoint functionality to retrieve and modify these
settings
- generate activities for enabling and disabling GitOps mode
- Premium only
- Update tests
<img width="1355" alt="Screenshot 2025-02-17 at 9 22 13 AM"
src="https://github.com/user-attachments/assets/e03d7fd6-8795-4df2-9c8f-ffbcabd2d212"
/>
<img width="1575" alt="Screenshot 2025-02-17 at 9 24 33 AM"
src="https://github.com/user-attachments/assets/5bf963d4-6fb7-4586-8f6f-9e605c25bf06"
/>
## Checklist for submitter
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
For #26218
Basic Android MDM on/off backend functionality. Manually tested.
The following env vars must be set:
```
FLEET_DEV_ANDROID_ENABLED=1
FLEET_DEV_ANDROID_SERVICE_CREDENTIALS=$(cat credentials.json)
FLEET_DEV_ANDROID_PUBSUB_TOPIC=projects/your-project/topics/your-topic
```
I picked https://github.com/go-json-experiment/json as the JSON library,
which seems like the safest option.
- will become json/v2 at some point
- currently used in production by other companies, like Tailscale
- well-maintained
- Some context here: https://github.com/fleetdm/fleet/issues/25512
Plan for next work:
- refactoring from 1st PR
- add pubsub with device enroll -> spec proxy for fleetdm.com
- come back to this sub-task to add tests and finish TODOs
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #24222
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
Android scaffold code and refactorings
- Android packages intended to be decoupled from other Fleet code
Video explaining the PR: https://www.youtube.com/watch?v=cza-35Z9Wxk
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #26177
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #25648
Fixed issue where `fleetctl gitops` was NOT deleting macOS setup
experience bootstrap package and enrollment profile. GitOps should clear
all settings that are not explicitly set in YAML config files.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #25770
We already unmarshal macOS/Windows settings (added by Martin), so we
replace the path with an absolute file path and keep them unmarshalled
so they don't have to be re-unmarshalled later. Note: the custom
UnmarshalJSON method on these structs checks for (and handles) legacy
format (before labels were added).
Also some refactorings:
- extracted `extractControlsForNoTeam`
- reorganized `TestGitOpsBasicGlobalAndNoTeam` with subtests -- I did
not actually change functionality of this test
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
for #24471
This PR adds missing support for updating a team's failing policy
webhook via GitOps:
* If `team_settings.webhook_settings.failing_policies_webhook` is not
set, the webhook will be disabled and all settings cleared.
* If `team_settings.webhook_settings.failing_policies_webhook` is set,
the webhook will be configured according to the specified settings.
A new test has been added to verify this functionality, and I tested it
locally using the `fleetctl gitops` command.
---------
Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com>
For #24870 subtask
API changes doc: #25293
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For #25160.
Measured improvement by splitting the MySQL query into two queries to
use the indexes more efficiently:
- ~8s vs ~100ms for ~30k entries in software_titles for ~1.7k incoming
software without bundle_identifier (linux software).
- ~1.64s vs ~2ms for 25k entries in software_titles and ~500 incoming
new software with bundle_identifier + ~200 new software without
bundle_identifier (macOS software).
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [X] Manual QA for all new/changed functionality
For #25533
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #23531.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
## For #25320
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
# Changes
- orbit >= 1.38.0, when configured to connect to
https://tuf.fleetctl.com (existing fleetd deployments) will now connect
to https://updates.fleetdm.com and start using the metadata in path
`/opt/orbit/updates-metadata.json`.
- orbit >= 1.38.0, when configured to connect to some custom TUF (not
Fleet's TUFs) will copy `/opt/orbit/tuf-metadata.json` to
`/opt/orbit/updates-metadata.json` (if it doesn't exist) and start using
the latter.
- fleetctl `4.63.0` will now generate artifacts using
https://updates.fleetdm.com by default (or a custom TUF if
`--update-url` is set) and generate two (same file) metadata files
`/opt/orbit/updates-metadata.json` and the legacy one to support
downgrades `/opt/orbit/tuf-metadata.json`.
- fleetctl `4.62.0` when configured to use custom TUF (not Fleet's TUF)
will generate just the legacy metadata file
`/opt/orbit/tuf-metadata.json`.
## User stories
See "User stories" in
https://github.com/fleetdm/confidential/issues/8488.
- [x] Update `update.defaultRootMetadata` and `update.DefaultURL` when
the new repository is ready.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For #24869
This subtask contains code to sign the CloudFront software installer and
bootstrap package URL using AWS SDK URL signer.
It works with the current bootstrap package delivery. For software
installers, fleetd will need to be modified to take advantage of this
URL in a future subtask (which will also include updated API contributor
docs).
My article on signed URLs, for context:
https://victoronsoftware.com/posts/cloudfront-signed-urls/
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
# Overview
This PR adds support for remote YARA queries to osquery-perf, so that
remote YARA queries can be load-tested.
# Details
The existing `runLiveQuery()` is updated to branch off into different
query running functions based on the content of the query. If the query
contains `from yara` and `sigurl`, then the new `runLiveYaraQuery()`
function is run which makes a request to the Fleet "get yara rules" API
before returning an appropriate response. Otherwise, the new
`RunLiveMockQuery()` function is run which includes the previous logic
for sending a mock response.
# Testing
I don't see any automated testing for osquery-perf, but I manually
tested in the following way:
1. Started osquery-perf with `go run agent.go`
2. Ran a live query on the new host using
```
SELECT * FROM yara where sigurl="https://localhost:8080/api/osquery/yara/rule1.yar"
```
and verified that the result was as-expected:
<img width="1642" alt="image"
src="https://github.com/user-attachments/assets/a2c9cacf-e28d-409c-8e83-1c82809b89c0"
/>
I also used a log in Fleet to verify that the "get yara rules" API was
really being called.
3. Ran another live query on the host using:
```
SELECT * FROM system_info"
```
and verified that the result was as expected:
<img width="1665" alt="image"
src="https://github.com/user-attachments/assets/a8d35389-f193-4902-badf-200d760fdf46"
/>
I also tested that sending a `sigurl` with the wrong host returns a
`live yara query failed because sigurl host did not match server
address` error
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
For #24868 (subtask)
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
#24900
This PR includes and depends on PR #25012, which should be
reviewed/merged before this one.
Windows profiles are not included in this PR due to issue #25030
This PR adds the following functionality: Apple config/DDM profile is
resent to the device when the profile contains secret variables, and the
values of those variables have changed. For example.
- Upload secret variables
- Upload profile
- Device gets profile
- Upload the same profile
- Nothing happens
- Upload a different secret variable value
- Upload the same profile
- Device gets updated profile
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Manual QA for all new/changed functionality
For #22875.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
For secrets subtask #24548
Fixed secret variables support in Apple configuration profiles.
# Checklist for submitter
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
for #24041
This PR addresses an issue that can cause Windows vulnerability checks
to fail (possibly causing false negatives). We determine whether a
vulnerability in an MSRC bulletin applies to any hosts in a Fleet
instance by attempting to matching the data in [each row of the
`operating_systems`
table](65e374c85c/cmd/fleet/cron.go (L297-L303))
with [at least one "product" in a
bulletin](e2d9a9016c/server/vulnerabilities/msrc/analyzer.go (L39)),
including [matching architecture and "display
version"](76f5baced9/server/vulnerabilities/msrc/parsed/product.go (L26-L39)).
However a subset of products listed in these bulletins do not include
the display version, so for example a host whose OS was listed as
`Microsoft Windows Server 2022 Datacenter 21H2` (21H2 being the "display
version") would match nothing in the bulletins because no listed Server
2022 products include "21H2" in their names.
The fix made here is to add relevant version info to the products list
when we do our ETL of the MSRC bulletins. The version info was gleaned
from https://en.wikipedia.org/wiki/List_of_Microsoft_Windows_versions.
We see logs related to this issue a lot, so cleaning this up will
alleviate some noise and infra costs as well.
## Addresses #21855 and all of its subtasks
**Frontend:**
- Update list queries API call to include pagination and filter-related
query params, including new `platform` param for filtering queries by
platforms they've been set to target
- Convert all filtering, sorting, and pagination functionality of the
Manage queries page from client-side to server-side
- Remove unneeded variable declarations / logic
- Various typing and naming improvements
**Server:**
- Add new `platform` `ListQueryOption`
- Update service and datastore level list queries logic to handle
filtering queries by targeted platform
- Update service and datastore level list queries logic to include
`meta` and `count` fields in addition to filtered/paginated queries
- [x] Changes file added for user-visible changes in `changes/`, `
- [x] Added/updated tests
- [x] update DB, integration
- [x] add integration (pagination)
- [x] add integration (platform filter)
- [x] add DB (pagination)
- [x] add DB (platform filter)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
#23816
This fix may not completely fix the customer's issue. However, I'd like
to see if there are improvements from this fix combined with the
previous query optimization fix.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
#22790
Changes file is on the FE PR.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
for #22824
This PR splits up the `newMDMProfileManager` function into two functions
`newAppleMDMProfileManager` and `newWindowsMDMProfileManager`, the
latter containing the code to start the windows-specific profile
management job. This allows scheduling and scaling the Apple and Windows
profile management jobs separately.
Tested locally by checking that the jobs were scheduled at startup, and
then triggering the `mdm_apple_profile_manager` and
`mdm_windows_profile_manager` schedules via the `/trigger` API; nothing
blew up.
For #24132
Drops retry wait a bit as there doesn't seem to be a point in waiting
that long (I pulled feeds with a 10s retry interval in curl earlier).
Total time to pull feeds and build the CPE DB was ~4 hours this evening
(~6pm to ~10pm CST).
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
Previously when passing labels to the query run endpoints that do not
exist, the labels would simply be ignored. Now the endpoint will return
an error indicating which labels are invalid. This change also affects
the `fleetctl query` command `--labels` flag.
https://github.com/fleetdm/fleet/issues/23015
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
## Addresses #22702, #23713, #23756, #23746, #23747, and #23876
_-Note that much of this code as is will render as expected only once
integrated with the backend or if manipulated manually for testing
purposes_
**Frontend**:
- Update banners on my device page, tests
- Build new logic for calling endpoint to trigger linux key escrow on
clicking `Create key`
- Add `CreateLinuxKeyModal` to inform user of next steps after clicking
`Create key`
- Update banners on host details page, tests
- Update the Controls > OS settings section with new logic related to
linux disk encryption
- Expect and include counts of Linux hosts in aggregate disk encryption
stats UI
- Add "Linux" column to the disk encryption table
- Show disk encryption related UI for supported Linux platforms
- TODO: confirm platform string matching functionality in manual e2e
testing
- Expand capabilities of `SectionHeader` component, apply to new UI
- Flash "missing private key" error, with clickable link, when trying to
update disk encryption enabled while no server private key is present.
- TODO: QA this once other endpoints on Controls > Disk encryption are
enabled even when MDM not turned on
- Update Disk encryption key modal copy
-Other TODO:
- Confirm when integrated with API:
- Aggregate disk encryption counts
- Disk encryption table Linux column
- Show disk encryption key action on host details page when expected
- Opens Disk encryption key modal, displays key as expected
**Backend**:
- For "No team" and teams, error when trying to update disk encryption
enabled while no server private key is present.
- Remove requirement of mdm being enabled for use of various endpoints
related to Linux disk encryption
- Update tests
_________
**Host details and my device page banners**
**Create key modal**
<img width="1799" alt="create-key-modal"
src="https://github.com/user-attachments/assets/81a55ccb-b6b9-4eb6-b2ff-a463c60724c0">
**Enabling disk encryption**
**Disk encryption: Fleet free**
<img width="1912" alt="free"
src="https://github.com/user-attachments/assets/9f9cace3-8955-47c2-87d9-24ff9387ac1a">
**Custom settings: turn on MDM**
<img width="1912" alt="turn on mdm"
src="https://github.com/user-attachments/assets/4d3ad47b-4035-4d93-86f0-dc2691b38bb4">
**Device status indicators**
**Encryption key action and modal**
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- [ ] Full e2e testing to do when integrated with backend
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
#22810
# Demo
[](http://www.youtube.com/watch?v=le71QQ92suc)
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#19696
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
> Related issue: #22359
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Implements the Fleet side of #14899
- Add new endpoints to update and retrieve yara rules
- Add support in fleetctl for applying the rules
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.
- [x] Added/updated tests
- [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
#22187
Similar fix to #22555: resolve paths at spec parsing time rather than
when trying to grab files
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#23258 (see
[comment](https://github.com/fleetdm/fleet/issues/23258#issuecomment-2443304838)
for rationale)
Validated by removing the two places that would create the directory
(early in scanVulnerabilities in cron.go, partway through download in
download.go) and ensuring the test failed (timeout after 10s).
Both dir creations happen early in the vulns cron so I was able to
drastically tighten the timing on the periodic check on this test, so
this tests completes way quicker than before as an added benefit
(automatic test parallelism notwithstanding).
The panic recovery here theoretically shouldn't be necessary, as on a
passed test the context will get cancelled while syncing the CPE sqlite,
but is included to ensure the test doesn't flake if the implementation
of the vulnerabilities cron changes such that we _would_ get a panic by
cancelling the context this early.
# Checklist for submitter
- [x] Added/updated tests
#22740
Full QA is still a WIP but this is ready for review.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#21715
This fix is needed for https://github.com/fleetdm/fleet-gitops CI to run
successfully against a non-premium Fleet server.
That CI test will cover this change.
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
#23050
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
makes a fix to the enroll byod handler. The hander use to be set once
when the server started but this lead to a user receiving a 404 page
after the fleet instance setup was complete but was not restarted.
now the `/enroll` handler will check for the setup requirement itself.
There is some error handling when the setup is required. if the fleet
instance has been setup, they will get the enroll byod page.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
For #21955 (the story has a video demo of core functionality)
Follow up for PR #22542
# Checklist for submitter
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
For #22787
Also revises the spec check to explain that scripts have to be defined
"controls" when used in policies for the same team, with an explicit
call-out for no-team.yml since this fix doesn't support pulling scripts
from the global file. This is because parsing and script-matching
happens early enough that we can't throw an error in the part of the
code where we bail when controls are defined in both no-team and default
files.
To minimize diff size, we're both "passing-by-ref" and returning the
maps-by-team of scripts and installers, though the former would be
sufficient on its own.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- N/A Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests (sorta)
- [x] Manual QA for all new/changed functionality
#22115, #22116
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
No changes file, as FE changes file covers the entire feature
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Co-authored-by: Tim Lee <timlee@fleetdm.com>
#22473
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
---------
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
> Related issue: #22198
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
See #22492
As of today, MSRC's API will 401 if you pass it a JWT it doesn't expect,
and by reusing the GitHub API HTTP client for MSRC pulls we were passing
the API an unexpected JWT. Wasn't able to reproduce this locally because
I didn't need a GitHub token locally to pull release details, while the
token is populated in Actions. Was able to repro both the issue and this
fix inside Actions on my fork.
This also updates to call v3.0 of the API directly, which v2.0 has been
redirecting to for awhile.
Finally, adds slightly better logging so we know which part of the feed
generation process we're in when we're running this in Actions.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
Same as #22412, for #21370, but against `main` rather than 4.57.0.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
This override only happens in testing, so this isn't release-blocking,
but this is the quickest way to clean up a test that will otherwise be
flaky due to data races, at the cost of performance (vs. setting up a
more complex solution with mutexes).
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality (via manually running
test using the KV store)
#22069
API changes: https://github.com/fleetdm/fleet/pull/22259
QAd by applying 10 pieces of software on a team, which took 3+ minutes
in total (which, before these changes was timing out at 100s.)
With this approach, a GitOps CI run timing out might leave the
background process running (which will eventually be applied to the
database). The team discussed and agreed that we can fix this edge case
later.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
`fleetctl gitops` subtask for #20320
# Checklist for submitter
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#21467
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
> Related issue: #17558
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
for #21019
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#20828
This is a fix specific to the VPP App ID field in gitops. It's a field
where users are likely to get the type wrong, so I've addressed this
separately before a larger effort.
#20895
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
> Related issue: #21246
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
relates to #19448
Adds the ability for a user to enroll a their device into fleet MDM.
> NOTE: this is the PR for the feature branch to go into main so all
code has already been approved.
relates to #21559
This adds the ability for end users to enrol their own device in fleet
mdm.
> NOTE: this new byod HTML page is a separate HTML asset that contains
all styles and scripts needed for the page to work. We do not send the
fleet UI assets and this drastically cuts down the response time to the
users who will be visiting this page on mobile devices
There are two sides included in this:
**Adding a new add host modal ios and iPad section for IT admins**
**delivering a new byod HTML page to end users that will allow end users
to download the config profile to enrol into fleet mdm**
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
#20757
API endpoint `/api/v1/fleet/mdm/profiles/batch` will now not log an
activity for profile types that did not change in the database (Apple
configuration profiles, Windows configuration profiles, or Apple
declarations).
Demo video: https://www.loom.com/share/8b75cbd8e7394c12ac6b56746b72c244
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
#21428
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
> Related issue: #9956
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
#20934
This is tied to https://github.com/fleetdm/vulnerabilities/pull/14; for
supported OS versions (currently Amazon Linux 1/2/2022/2023) we'll pull
XZ'd sqlite files from the vulnerabilities repo and query them to
determine what's vulnerable. See the associated issue for how I
self-QA'd this.
This replaced OVAL parsing for Amazon Linux 2, as we were using the
wrong data source there (Amazon has backported a bunch of fixes to their
own-named releases, so any RHEL fixes don't match).
Some checklist items are missing here; getting this set up in draft to
get code feedback now, and I'll push updates with e.g. docs changes, as
well ass an addition to the changes file.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Add tests to oval_platform
- [x] Add sync_test
- [x] Add database_test
- [x] Manual QA for all new/changed functionality
- [x] Update vulnerability management docs
#21073
- Deleted iOS/iPadOS host will continue to report to Fleet as long as
host is in Apple Business Manager (ABM).
- Refetching an offline iOS/iPadOS host will not add new MDM commands to
the queue if previous refetch has not completed yet.
Video demo:
https://www.loom.com/share/2f7ecb22e1924d4cbbbdd7dd297439ef?sid=dbfe1939-cb46-47ca-a7a8-84965ed68a7e
I considered `nano_command_results` but could not think of an efficient
way to use `nano_command_results` to determine if refetch was already
done. The problem is that we're overloading `command_uuid` to include
the `REFETCH` identifier. So we can't simply add an index on `(uuid,
command_uuid, updated_at)`. We need to do a partial text match on
`command_uuid` which then requires a full scan of the matching
`updated_at` timestamps. It feels like a search like this could blow up
for a large `nano_enrollment_queue` and `nano_command_results` table.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
Additional statistics as described in
https://github.com/fleetdm/fleet/pull/20091 :
`aiFeaturesDisabled`:
Whether server_settings.ai_features_disabled is set to true in the
config.
`maintenanceWindowsEnabled`:
Whether at least one team has
integrations.google_calendar.enable_calendar_events set to true
`maintenanceWindowsConfigured`:
Maintenance windows are considered "configured" if:
configuration has value set for integrations.google_calendar[0].domain
configuration has value set for
integrations.google_calendar[0].api_key_json
`numHostsFleetDesktopEnabled`:
The number of hosts with Fleet desktop installed.
#21038
Fleet server now accepts arguments via stdin. This is useful for passing
secrets that you don't want to expose as env vars, in the command line,
or in the config file.
Demo: https://www.loom.com/share/c8b4dc6ae6ef4182bc812d7f43423f4d
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#20464
Adding gitops support for a top level `software` key to be used to
manage installable software into "no team".
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
---------
Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
#19447
iOS and iPadOS apps can be installed using Apple's VPP (Volume Purchase
Program)
VPP apps are now using a composite primary key (Adam ID and platform)
because we want to keep iOS/iPadOS/macOS separate. It is possible for
one app to be installable on all Apple platforms.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
Part 2 of https://github.com/fleetdm/fleet/issues/19447
- iOS and iPadOS user-installed apps are loaded into Fleet
- Added an additional identifier into software_titles table to
differentiate between iOS/iPadOS apps
- Updated nano queue timestamp precision
Note: TestIntegrationsMDM/TestVPPApps fails when run as part of the
suite, but passes standalone. I'd like to proceed with merging this PR,
and figure out the issue next week.
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
Part 1 of #19447
- iOS and iPadOS device details refetch can now be triggered with the
existing `POST /api/latest/fleet/hosts/:id/refetch` endpoint
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
