Migrates CAs out of the App Config JSON and into their own table
including their secrets. Does not implement CRUD endpoints or update
clients of existing app config to use them yet.
I think there are good arguments both for keeping secrets in separate
tables and keeping them in this table but on balance I felt it was best
to move everything CA to this new table
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
Fixes#30879
Demo video: https://www.youtube.com/watch?v=jVyh5x8EMnc
I added a `FleetVarName` type, which should improve
safety/maintainability, but that resulted in a lot of files touched.
I also added the following. However, these are not strictly needed for
this feature (only useful for debug right now). But we are following the
pattern created by MDM team.
1. Add the migration to insert HOST_UUID into fleet_variables
2. Update the Windows profile save logic to populate
mdm_configuration_profile_variables
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host isolation]
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
* **New Features**
* Added support for the `$FLEET_VAR_HOST_UUID` variable in Windows MDM
configuration profiles, enabling per-host customization during profile
deployment.
* Enhanced profile delivery by substituting Fleet variables with actual
host data in Windows profiles.
* Introduced a database migration to register the new Fleet variable for
host UUID.
* **Bug Fixes**
* Improved validation and error handling to reject unsupported Fleet
variables in Windows MDM profiles with detailed messages.
* Ensured robust handling of errors during profile command insertion
without aborting the entire reconciliation process.
* **Tests**
* Added extensive tests covering validation, substitution, error
handling, and reconciliation workflows for Windows MDM profiles using
Fleet variables.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#31697 flaky TestCacheEviction test.
Seen here:
https://github.com/fleetdm/fleet/actions/runs/16802837212/job/47588005349
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved cache stability by adding safety checks to prevent invalid
updates when cache entries are evicted or removed.
* Enhanced eviction logic to prevent out-of-bounds errors during cache
management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
#29554
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com>
for #31555
# Details
This PR adds a new cron schedule "batch_activity_completion_checker"
that runs every 5 minutes and checks whether any batch activities marked
as "started" have completed their runs. In general this is done by
determining whether the sum of the "ran", "incompatible", "errored" and
"canceled" hosts equals the number of "targeted" hosts for the activity.
How that is computed will vary by batch activity type (currently we just
have batch scripts).
When an activity is marked as finished, we cache the final tally of host
statuses (ran, incompatible, errored, canceled) on the record. This is
important so that future queries on activity records don't have to do
the expensive query to compute the host counts on activities where those
counts will never change.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
Started a new batch script run using the update run modal (see
https://github.com/fleetdm/fleet/pull/31604) and then triggered the new
job using `fleetctl trigger --name batch_activity_completion_checker`,
and verified that the `batch_activities` record status was `finished`
and the expected fields were populated.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced an automated process that regularly marks completed batch
activities, ensuring more accurate and up-to-date activity statuses.
* **Bug Fixes**
* Improved reliability in updating the status of batch activities when
all targeted hosts have finished their tasks.
* **Tests**
* Added comprehensive tests to verify correct marking of completed batch
activities.
* **Chores**
* Enhanced internal scheduling and datastore interfaces to support the
new completion-checking process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
for #31623
for #31526
# Details
This PR adds two new endpoints:
* `GET /scripts/batch/:batch_execution_id` returns the status of a
single batch script execution
* `GET /scripts/batch` returns a paginated list of batch script
executions, filtered by team and status
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
Added new batch script runs in UI, used Postman to list them and get
details.
#31592
There's still some QA to be done for edge cases and re-connects, but
this is ready for review.
<img width="341" height="103" alt="Screenshot 2025-08-07 at 11 19 33 AM"
src="https://github.com/user-attachments/assets/01e48ca2-8ab1-412c-be01-8e806a5a8b1c"
/>
Changes:
- To improve UX I'm now using `HEAD /api/fleet/device/ping` API every 10
seconds for connectivity/offline check (instead of the expensive
DesktopSummary one every 5 minutes). This is to address feedback from a
customer:
> "If the internet is not connected and we reconnect with an ethernet
connection for example, it would be good to try to see if we can refresh
it text from the offline indicator given that's not the case anymore.
- It might take up to 1m for Fleet Desktop to show the offline indicator
(we check every 10s with ping and now we are adding 6 more requests in 1
minute to make sure just one bad request doesn't unnecessarily display
the offline indicator).
- Requests without proper public IP were being incorrectly rate limited
(all under the same bucket). So we will now not make these requests and
instead log a WARNING. This is a-ok as the recommended approach to
deploy Fleet is with a TLS terminator that will add the public IP of the
request before sending it to Fleet.
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [ ] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved accuracy in identifying client public IP addresses, reducing
incorrect rate limiting for Fleet Desktop users.
* Offline indicator is now less sensitive to brief network
interruptions, reducing false offline signals and allowing faster
recovery when connectivity is restored.
* Updated offline message for clearer status communication.
* **New Features**
* Enhanced error messages and logging for rate limiting events,
providing clearer feedback when limits are reached.
* **Tests**
* Expanded test coverage for rate limiting, including scenarios with
missing public IPs and improved assertions for error handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #31180.
Added new detail query, only executed if TPM PIN enforcement is
required, for determining whether a BitLocker PIN is set. The result of
the new detail query is used for setting the tpm_pin_set column on the
host_disks table.
Fixes#31690
No functional changes: extra logging and refactoring
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Enhanced error messages for Windows MDM discovery, providing more
detailed information about unsupported request versions.
* **Bug Fixes**
* Improved logging for errors encountered during the Windows MDM
discovery process, aiding in issue diagnosis.
* **Refactor**
* Streamlined the Windows MDM discovery endpoint to centralize
validation and response logic for better maintainability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #31518
# Details
This PR updates the jobs package with a new `GetFilteredQueuedJobs`
method which accepts a list of job names to return from the `jobs`
table, rather than returning all jobs matching the other criteria. It
also updates the `worker` package to use this new method instead of
`GetQueuedJobs`.
The purpose of this update is to allow us to add multiple workers which
process different kinds of jobs, while sharing the same `jobs` table and
framework.
I chose to add the new `GetFilteredQueuedJobs` method rather than
updating `GetQueuedJobs` because the latter is used in a bunch of tests
that would need to be updated. `GetQueuedJobs` now calls
`GetFilteredQueuedJobs` with an empty list, which is interpreted as
"return all jobs".
# Checklist for submitter
## Testing
- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
There shouldn't be any functional change here, but if someone has a Jira
or Zendesk integration set up to easily test with, that'd be good.
fixes#30682
If some of the following don't apply, delete the relevant line.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Putting this up for comments
On certain errors(like a network error, perhaps even Apple ratelimiting)
we previously would drop assignments during the DEP sync and leave the
host_dep_assignments row null and the assignment unset on the Apple
side. Because of how the sync works it is entirely possible when this
happens that we would happily go along, update the cursor and never
return to resync these devices unless and until the admin did something
that forced a resync like changing something about the cloud config
profile.
Now any devices that for any reason don't get returned by the response
get marked as failed so that our logic for retrying and processing
cooldowns picks them up for later retry.
Explanation here as far as what I think is going wrong:
https://github.com/fleetdm/fleet/issues/31385#issuecomment-3145117080
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
fixes: #30857
This PR also adds the canceled check for Lock and Wipe scripts, even
though they can not be canceled as it stands today.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
for #31505
# Checklist for submitter
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
no changed functionality
## Database migrations
- [X] Checked table schema to confirm autoupdate
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
collate for both these tables is previously set to `utf8mb4_0900_ai_ci`,
but they don't contain user-settable strings
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded batch activity tracking with new fields for job ID, status,
and activity type.
* Enhanced database structure to support more flexible batch activity
and host result management.
* **Bug Fixes**
* Updated references throughout the application to use the new table
names for batch activities and host results, ensuring consistency.
* **Tests**
* Added and updated tests to validate the new batch activity schema and
migration behavior.
* **Chores**
* Applied database migration to rename and restructure relevant tables
for improved clarity and extensibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #31182
# Details
This PR implements the "Action Required" state for Windows host disk
encryption. This includes updates to reporting for:
* disk encryption summary (`GET /fleet/disk_encryption`)
* config profiles summary (`GET /configuration_profiles/summary`)
* config profile status ( `GET
/configuration_profiles/{profile_uuid}/status`)
For disk encryption summary, the statuses are now determined according
to [the rules in the
Figma](https://www.figma.com/design/XbhlPuEJxQtOgTZW9EOJZp/-28133-Enforce-BitLocker-PIN?node-id=5484-928&t=JB13g8zQ2QDVEmPB-0).
TL;DR if the criteria for "verified" or "verifying" are set, but a
required PIN is not set, we report a host as "action required".
For profiles, I followed what seems to be the existing pattern and set
the profile status to "pending" if the disk encryption status is "action
required". This is what we do for hosts with the "enforcing" or
"removing enforcement" statuses.
A lot of the changes in these files are due to the creation of the
`fleet.DiskEncryptionConfig` struct to hold info about disk encryption
config, and passing variables of that type to various functions instead
of passing a `bool` to indicate whether encryption is enabled. Other
than that, the functional changes are constrained to a few files.
> Note: to get the "require bitlocker pin" UI, compile the front end
with:
```
SHOW_BITLOCKER_PIN_OPTION=true NODE_ENV=development yarn run webpack --progress --watch
```
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
Changelog will be added when feature is complete.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
Could use some help testing this end-to-end. I was able to test the
banners showing up correctly, but testing the Disk Encryption table
requires some Windows-MDM-fu (I just get all zeroes).
## Database migrations
- [X] Checked table schema to confirm autoupdate
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
fixes#29833
Checks if the error is a not found error and then return a notFoundError
that does not get logged as an error but as an info log instead.
`level=info ts=2025-08-05T10:46:06.237581Z component=http
path=/api/latest/fleet/setup_experience/eula/metadata took=1.939958ms
uuid=0ab0c579-07c5-48be-b6bd-5e4ebd81212d err="not found"`
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes#31545
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Resolved issues with Single Sign-On (SSO) and Mobile Device Management
(MDM) SSO authentication failures caused by trailing slashes in URLs,
ensuring proper URL formatting and preventing authentication errors.
* **Tests**
* Added tests to verify correct handling of trailing slashes in SSO URLs
and to ensure errors are properly returned for invalid SSO URL
configurations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#31444
The changes are primarily in tests. The only changes in production code
are a couple validations/checks for invalid values in:
- mysql/apple_mdm.go
- mysql/hosts.go
- mysql/queries.go
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved handling of timestamp and default values across various
features to prevent database errors and warnings.
* Enhanced validation and data consistency for Apple Business Manager
tokens and MDM profiles.
* Updated test data and logic to comply with stricter database
constraints and realistic scenarios, including date handling and field
lengths.
* **Chores**
* Updated test setups to reflect schema changes, improve data integrity,
and avoid future compatibility issues.
* Standardized SQL mode and timestamp usage in test environments.
* Refined test data for VPP apps, software installers, and device
enrollments for better reliability.
* **Tests**
* Expanded and updated tests to cover new fields, stricter validation,
and more accurate simulation of real-world conditions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This change allows configuring a separate URL for SSO callbacks, which
is useful when organizations have different URLs for admin access vs
agent/API access.
Fixes#31480 the SSO issue where organizations with dual URL setups were
getting 'Destination does not match requested URL' errors after
upgrading to v4.71.0 with the new SAML library.
Video demo: https://www.youtube.com/watch?v=dFzNpUY3XKI
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- Same PR since this is going to be a 4.71.1 patch
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Summary by CodeRabbit
* **New Features**
* Added support for configuring a dedicated SSO URL, allowing
organizations to restrict SSO authentication to a specific URL.
* The new SSO URL option is available in both the UI and API
configuration settings.
* **Documentation**
* Updated configuration and API documentation to include the new SSO URL
option with usage examples.
* **Bug Fixes**
* Resolved authentication issues for organizations using separate URLs
for admin and agent/API access.
* **Tests**
* Added new unit and integration tests to verify SSO behavior with and
without the dedicated SSO URL.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #31193.
Added a new detail query used for determining whether the user is able to set up a TPM PIN protector, if not able, an MDM command is queued up to apply the proper policy on the host.
Fixes#31450
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
Done by not allowing any deferrals as before one deferral was allowed
fixes: #29250
_There is no doc change as we nowhere state that we allow one deferral,
let me know if we want to write somewhere that this is now the standard
behaviour._
_I also investigated trying to force it directly when the profile
arrived but without any luck, so still need a logout/login to get
filevault enabled, but it's no longer possible to cancel/defer it._
**Do verify when testing in automatic enrollment that FileVault is still
enforced as I can't test Automatic enrollment yet.**
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] If database migrations are included, checked table schema to
confirm autoupdate **not relevant as it does not update any schema just
modifies existing entries.**
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#30390
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded support for certificate country codes up to 32 characters,
allowing non-standard country code values.
* **Bug Fixes**
* Improved certificate ingestion to handle and log fields that exceed
maximum allowed lengths instead of causing failures.
* **Tests**
* Added tests to verify handling of long country codes and truncation of
overly long certificate fields.
* **Chores**
* Updated database schema to increase the allowed length for country
code fields in certificates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This PR addresses the concern of potentially being able to release a
device before any profile is sent, and the check thinking there is no
pending. It addresses both the release worker, but also the orbit setup
experience endpoint, even though that is less likely.
_Checked the query against my host on dogfood where it took 0.1 seconds,
with the single host._
fixes: #31143
_I also ended up putting my main test in a new file
`integration_mdm_release_worker_test.go` and decided not to do fancy
setup, as there is only one test so no recurring things, and based on
our retro talk also moved the setup experience related tests inside of
`integration_mdm_dep_test.go` into their separate file
`integration_mdm_setup_experience_test.go`_
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually (No, since this one is
hard to reproduce, but instead wrote an integration test before doing
the change to verify the behaviour.)
Fixes#28573Fixes#31141
I don't know of a good way to add automated tests for this unfortunately
- there is no easy way that I can see in our tests to really reproduce
either of these issues
Manual testing can be done for 31141 by creating a different user,
recreating the nano_view_queue as that user using the old query(i.e.
without SQL SECURITY INVOKER), and then deleting the user, seeing that
the nano_view_queue gets busted(i.e. no queries can be run against it),
then doing this migration or even just running the new query and
verifying that the nano_view_queue can be queried again.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
## Testing
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
Fixes#30853
Install and uninstall scripts that contain fleet secrets do not need to
be validated in the `batchSetSoftwareInstallersEndpoint` during gitops
dry runs. These secrets are already validated on the gitops side.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes#31232
The Microsoft documentation does not directly address
RequestVersion/EnrollmentVersion 7.0 in the allowed versions lists but
grepping the MS-MDE2 protocol docs for EnrollmentVersion finds a few
references related to Federated Authentication and this specifically
seems to be a changeset introduced for AzureAD/Intune related features
and does not impact how we do enrollments, as best I can tell.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
For #31065
Added new orbit config flag 'EnableBitLockerPINProtectorConfig' set iff Disk encryption is enforced and the RequireBitLockerPIN server config flag is set.
fix#27689
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (added in the
comment)
- [x] Manual QA for all new/changed functionality
output ss
Noticed that editing a global policy doesn't generate an edited_policy
activity log in my local environment, while it does in others.
Investigated the gap and shared findings in the issue thread for further
clarification(in https://github.com/fleetdm/fleet/issues/27689)
---------
Co-authored-by: Scott Gress <scottmgress@gmail.com>
Co-authored-by: Scott Gress <scott@fleetdm.com>
Fixes#26404.
This means that for long vulns runs vulns will stick around longer, so
we don't wind up nuking vulns that were added earlier in the run, and in
cases where the vulns run takes less than 2h we'll see vulns clear
cleanly more quickly.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
---------
Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
Add the capability to build a request body with `fleetctl api`,
including uploading files.
Example command to upload a software package:
```sh
fleetctl api --debug -X POST -F team_id=0 -F 'software=@./server/service/testdata/software-installers/ruby.deb' software/package
```
Unit tests are included for both simple POST requests and file uploads.
Closes#21754.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#30565. Applies to FMA-only extensions (DMG, ZIP), EXEs, and
tarballs. This means that MSI/PKG FMAs will still have package IDs
populated a day after server start if they aren't filled in, on the off
chance that admins use $PACKAGE_ID on uninstall scripts on either of
those, replicating existing behavior.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
# Checklist for submitter
Commenting out test so that CI runs don't fail. This will be fixed in
https://github.com/fleetdm/fleet/issues/31303.
If some of the following don't apply, delete the relevant line.
## Testing
- [x] Added/updated automated tests
We'll want to pull this into a feed so fixes don't take a Fleet release
to propagate, and some fixes currently in the vulns mutations list
should probably move over here (as they're also dealing with non-semver
versions), but that's out of scope for this particular fix.
Fixes#31123.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #31063
# Details
This PR adds the `RequireBitLockerPIN` config to app-wide and team
configs. This maps to a new `windows_require_bitlocker_pin` JSON field
for gitops and `fleetctl apply`.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
* Will add changelog when feature is complete
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
* Will add to docs when feature is complete
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
* No UI yet
- [X] Manual QA for all new/changed functionality
* Tested No Team and team config via Postman API calls
* Tested Gitops for no-team and team YML files using `fleetctl`
* Tested `fleetctl generate-gitops`
Fixes#30082
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Microsoft Office applications for Mac now display a simplified short
version identifier, improving consistency with inventory systems.
* Added support for version transformation during ingestion of Homebrew
apps using external reference functions.
* **Bug Fixes**
* Enhanced uninstall process for Microsoft Word on Mac to remove a
broader set of user data and configuration files.
* **Tests**
* Added tests to ensure correct extraction of build numbers and short
version formats from Microsoft Office release notes.
* **Documentation**
* Updated changelog to reflect the addition of short version identifiers
for Microsoft Office Fleet maintained apps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [ ] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [ ] Manual QA for all new/changed functionality
Attempt at fix: #24652
I used randokiller with up to 100 and 150 retries, where sometimes it
failed after 89 attempts, to verify the flakiness.
I tried a couple of approaches, with one being just upping the
time.sleep to 200ms still without luck.
I then opted for a _sync_ channel that blocks and waits for the timed
flush to execute which allows it to go further and with a timeout that
fails the test if a deadlock is caused.
I looked into [synctest](https://pkg.go.dev/testing/synctest), but that
is still experimental and expected to become non-experimental in 1.26,
so not really usable for now, but sounds like it will help in time/timer
based tests that are a bit flaky.
Successful randokiller run:
https://github.com/fleetdm/fleet/actions/runs/16446175657/job/46478695973
where retries is set to 100 using the channel approach here.
_I do feel it's a bit weird that the last test `cap and timed flush`
does not have the same timer issues, might be due to the setup of only
having one item before each sleep, where if it had multiple it might be
suspect to the same time issues seen in this test `timed flush`_
Fixes: #29824
This PR fixes a situtation where Apple Declarations could be lingering
around for hosts, if they were offline when the decl. was added and
removed, and no further declaration config is pushed to force a status
update.
It tackles it by deleting the pending and failed installs from the
table, before setting the remaining (verified and verifying) to be
remove operation, as those have hit the host.
I couldn't come up with a way that would auto-fix the hosts we see in
dogfood, as those have the same declaration identifier and token for
both an install row and pending remove row. Those needs to manually be
adjusted and then it should be good.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes https://github.com/fleetdm/fleet/issues/30747
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [ ] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
This was required to test https://github.com/fleetdm/fleet/pull/30864 on
Apple Silicon.
I've created https://github.com/fleetdm/fleet/issues/31092 for tracking
purposes.
Fixes:
- Build univeral binary extension on macOS to test on VMs without
Rosetta.
- Add support for linux and Windows arm64. Which is also needed to test
Linux and Windows on UTM on Apple Silicon.
- Add Linux arm64 & Windows arm64 to the test scripts.
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
> Closes#31060
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Added/updated automated tests
- [x] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [x] Manual QA for all new/changed functionality
#30461
This PR contains the changes for the happy path.
On a separate PR we will be adding tests and further fixes for edge
cases.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for using a TPM-backed key and SCEP-issued certificate
to sign HTTP requests, enhancing security through hardware-based key
management.
* Introduced new CLI and environment flags to enable TPM-backed client
certificates for Linux packages and Orbit.
* Added a local HTTPS proxy that automatically signs requests using the
TPM-backed key.
* **Bug Fixes**
* Improved cleanup and restart behavior when authentication fails with a
host identity certificate.
* **Tests**
* Added comprehensive tests for SCEP client functionality and TPM
integration.
* **Chores**
* Updated scripts and documentation to support TPM-backed client
certificate packaging and configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#30475
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced support for HTTP message signing in agent-server
communications, enhancing request authentication.
* Added a configurable option to control the probability of agents using
HTTP message signatures via a new command-line flag.
* **Bug Fixes**
* Improved error logging for issues encountered during HTTP signature
key retrieval, providing better visibility into failures.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Closes#28490
This has been fixed on the NVD side. No changes needed other than
re-enabling the test.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#30989
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added rate limiting to SCEP certificate enrollment, returning a "Too
Many Requests" (HTTP 429) response if hosts request certificates too
frequently.
* **Bug Fixes**
* Improved error handling for rate-limited SCEP requests, providing
clear feedback when rate limits are exceeded.
* **Tests**
* Introduced integration tests to verify SCEP rate limiting behavior.
* **Chores**
* Enhanced internal configuration handling for SCEP certificate
management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Closes#31020
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
## For #30749, #31013
This PR implements changes to the UI and back end to accommodate
Software automations and Failing policy Ticket and Webhook automations
when Fleet is in Primo mode. Follow-up to
https://github.com/fleetdm/fleet/pull/30291
### Software automations
- When on the `/software` page and in Primo mode, the UI is, under the
hood, on "No team," though any reference to "team"s is hidden as much as
possible. In "normal" Fleet, Software automations can only be accessed
when on "All teams." This PR implements a special case in Primo mode:
when on No team and Primo mode is enabled, the user can now access the
"Software automations" modal to configure automation settings, which are
global.
- Simplified some conditions
- Moved logic living in the parent Software page that was specific to
the `SoftwareAutomations` modal into the modal for better encapsulation.
### Policy automations
The calendar, software, and scripts failing policy automations are
currently only configurable on a team (including No team) and not for
All teams. Ticket and webhook automations, accessible via the "Other
workflows" modal, by contrast, are only configurable for All teams and
teams other than No team, but not for No team. This PR updates the
Policies page, when in Primo mode (and therefore forced to be on "No
team") to:
- Continue providing "No team" data to the first 3 mentioned policy
automations modals.
- Include an enabled Other workflows option in the automations dropdown
- Update the submission handler of the Other workflows modal to update
the relevant _global_ config values
- The backend is updated to recognize this case (Failing policy webhook
/ ticket destination, policy on No team, in Primo mode) and handle it
using the global config, making the above logic sound
_Product should consider if any of these changes should be implemented
for "normal" Fleet_
### Listing and deleting policies
- Primo mode presents a pseudo-team-less UX. However, it is still
possible for earlier clients to have policies on "All teams." This
implements the ability to both see and delete "teamless" (No team under
the hood) policies and any such inherited global policies
### Other UI considerations
- Remove teams-related functionality in a couple more places - see
#31013
### Demos
- [Deleting policies, including any potentially inherited from All teams
(possible from before Primo
mode)](https://drive.google.com/file/d/1ZI4MNM3bkiOtD5MInAU32htQw8kDEupK/view?usp=drive_link)
- [x] Changes file added for user-visible changes in `changes/
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Fixes#27580.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> Closes#27447
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
Fixes#27758.
<img width="807" height="303" alt="image"
src="https://github.com/user-attachments/assets/58e5b9bc-42d6-4195-868e-bf6206ec9cd5"
/>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#30474
# Checklist for submitter
- [x] Added/updated automated tests
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Added new test cases to verify that re-enrollment with a deleted
host's certificate is correctly rejected.
* **Bug Fixes**
* Improved host deletion to ensure related certificate records are
properly cleared.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #30947
# Checklist for submitter
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for requiring HTTP message signatures for fleetd
requests, configurable via a new setting.
* Enhanced middleware to enforce HTTP message signature requirements
when enabled.
* **Tests**
* Introduced integration tests to verify host identity endpoints enforce
HTTP message signature requirements.
* Updated test utilities and suite setup to support configurable
signature enforcement.
* **Chores**
* Refactored configuration and test server options to support the new
signature enforcement feature.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#30473
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for TPM-backed host identity certificates enabling
hardware-backed HTTP signature authentication for hosts.
* Introduced HTTP signature verification middleware for API requests,
applied conditionally for premium licenses.
* Hosts presenting identity certificates must authenticate with matching
HTTP message signatures during enrollment and authentication.
* Added SCEP-based certificate issuance for secure host identity
management.
* Updated enrollment endpoints to use standardized request/response
contract types.
* **Bug Fixes**
* Enhanced authentication logic to verify consistency between host
identity certificates and host records, preventing duplicate or
mismatched identities.
* **Chores**
* Updated dependencies and test infrastructure to support HTTP signature
verification and host identity certificate workflows.
* Added comprehensive integration and datastore tests for host identity
certificate issuance, storage, and authentication.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Profile status respect remove operation for pending and failed status,
where the declarations did not, meaning the host would show up with a
wrong status if only a declaration was pending or failed removal.
This was also affecting the `os_setting` api filter option for list
hosts (maybe elsewhere), which is also fixed by this change.
A part of #29824
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
Fixes#29849.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#30473
Refactore Datastore.EnrollHost and Datastore.EnrollOrbit methods to use
functional options. Doing this refactor before adding new options to
those methods. This should make the code more maintainable and easier to
understand.
No functional changes here. Just refactoring.
# Checklist for submitter
- [x] Added/updated automated tests
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Streamlined host and Orbit enrollment methods to use a flexible
options-based pattern instead of fixed parameter lists.
* Updated related tests and service logic to use the new options
approach, improving clarity and extensibility for enrollment operations.
* **New Features**
* Introduced configuration options for host and Orbit enrollment,
allowing more explicit and customizable parameter setting during
enrollment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#27061.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes 30636
I am adding a handful of additional unit tests but this is ready for
review now. Integrates changes from Victor's PoC for Account Driven User
Enrollment including a nice end to end integration test including the
SAML portion
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> Closes https://github.com/fleetdm/fleet/issues/30678
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
Fixes https://github.com/fleetdm/fleet/issues/29315
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* The "created_at" label no longer affects manual label scoping for
software packages, ensuring more accurate filtering.
* Device authentication tokens are now validated solely by their value,
not by their expiration time.
* **Tests**
* Added new tests to verify label scoping logic, ensuring correct
handling of dynamic and manual labels based on timestamps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#30458
Contributor docs PR: https://github.com/fleetdm/fleet/pull/30651
# Checklist for submitter
- We will add changes file later.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- Did not do manual QA since the SCEP client I have doesn't support ECC.
Will rely on next subtasks for manual QA.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced Host Identity SCEP (Simple Certificate Enrollment Protocol)
support, enabling secure host identity certificate enrollment and
management.
* Added new API endpoints for Host Identity SCEP, including certificate
issuance and retrieval.
* Implemented MySQL-backed storage and management for host identity SCEP
certificates and serials.
* Added new database tables for storing host identity SCEP certificates
and serial numbers.
* Provided utilities for encoding certificates and keys, and handling
ECDSA public keys.
* **Bug Fixes**
* None.
* **Tests**
* Added comprehensive integration and unit tests for Host Identity SCEP
functionality, including certificate issuance, validation, and error
scenarios.
* **Chores**
* Updated test utilities to support unique test names and new SCEP
storage options.
* Extended mock datastore and interfaces for new host identity
certificate methods.
* **Documentation**
* Added comments and documentation for new SCEP-related interfaces,
methods, and database schema changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Closes#26618
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#29988.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fix unreleased bug #30693.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Documentation**
* Updated testing documentation to include a missing command for
creating the Firehose delivery stream for "status" logs.
* **Refactor**
* Centralized AWS STS Assume Role credential configuration across
multiple AWS integrations (S3, Firehose, Kinesis, Lambda, SES) to use a
shared helper, improving maintainability and consistency.
* Removed deprecated inline credential configuration logic in favor of
the new centralized approach.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Get tests passing for now; #30670 tracks a proper fix.
- [x] Added/updated automated tests
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Disabled two test cases related to specific Microsoft Python extension
versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Mocking the manifest server to ensure that an app with a no_check hash
is always returned (in the event that chrome no longer has this property
about it)
Mocking the file content in the integration test. There is no reason to
really download the real 1password or chrome binaries for this
particular case.
For: https://github.com/fleetdm/fleet/issues/30325
- [x] Added/updated automated tests
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Tests**
* Enhanced test coverage for software installer payloads with simulated
manifest endpoints and SHA256 hash validations.
* Added mock servers to verify installer downloads and hash computations
for maintained apps, improving batch installer processing tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fix for #23899.
I found this while preparing the demo (I had a user with department and
no groups).
Assigning to @getvictor because Scott is OOO.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved host filtering for Identity Provider (IdP) group labels to
include hosts without corresponding group entries.
* **Tests**
* Enhanced label tests to cover IdP department labels, including
scenarios where users have no groups.
* Added new subtests to verify correct host inclusion and label counts
for department-based labels.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For https://github.com/fleetdm/confidential/issues/9931.
[Here](ec3e8edbdc/docs/Contributing/Testing-and-local-development.md (L339))'s
how to test SAML locally with SimpleSAML.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved SSO and SAML integration with enhanced session management
using secure cookies.
* Added support for IdP-initiated login flows.
* Introduced new tests covering SSO login flows, metadata handling, and
error scenarios.
* **Bug Fixes**
* Enhanced validation and error handling for invalid or tampered SAML
responses.
* Fixed session cookie handling during SSO and Apple MDM SSO flows.
* **Refactor**
* Replaced custom SAML implementation with the crewjam/saml library for
improved reliability.
* Simplified SAML metadata parsing and session store management.
* Streamlined SSO authorization request and response processing.
* Removed deprecated fields and redundant code related to SSO.
* **Documentation**
* Updated testing and local development docs with clearer instructions
for SSO and IdP-initiated login.
* **Chores**
* Upgraded dependencies including crewjam/saml and related packages.
* Cleaned up tests and configuration by removing deprecated fields and
unused imports.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
relates to [#28691](https://github.com/fleetdm/fleet/issues/28691)
This fixes an issue where `end_user_license_agreeement` is defined but
is an empty string or nil.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved handling of the end user license agreement (EULA) field in
app configuration to ensure it is always set correctly and removed from
the configuration map after processing.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fixes: #30325
Related to incorrect behavior introduced at
https://github.com/fleetdm/fleet/pull/28945
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* When uploading software batches, if the installer SHA is set to
"no_check," the system will now automatically generate and use the
SHA256 checksum of the installer file.
* **Bug Fixes**
* Fixed an issue ensuring the latest Google Chrome version is pulled
during Fleet-maintained app updates.
* Corrected the display of the SHA256 hash in the UI and API to show
valid values.
* Improved handling of installer uploads to ensure a valid SHA256
checksum is always applied, even when "no_check" is specified.
* **Tests**
* Added a test to verify correct SHA256 hash calculation for installer
files.
* Extended integration tests to validate batch software installer
operations for maintained apps with SHA256 hash checks.
* Added tests covering behavior when SHA256 checksum is marked as
"no_check" for maintained apps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
Found one during testing of Entra Conditional Access and then ran
https://github.com/crate-ci/typos to fix a few more.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected various typographical errors in user-facing text, tooltips,
table headers, and messages across the application to improve clarity
and professionalism.
* Fixed spelling mistakes in success and error messages, tooltips, and
descriptive text on multiple pages.
* Updated test setup to use the correct sort type value for CSV
conversion tests.
* Improved consistency in UI labels and documentation comments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#30574.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Resolved issues with recording host certificate sources when multiple
hosts share the same certificate but have different usernames, improving
accuracy and performance.
* Addressed related performance and database load problems for these
scenarios.
* **Tests**
* Added new tests to ensure certificate source records remain properly
isolated per host, even when certificates are shared across hosts.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Fixes#29980
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved support for iOS VPP app installations, including enhanced
verification and activity logging.
* Hosts now automatically request a software data update after
successful app installs.
* Installation status now includes "Installing" state for better
tracking.
* **Tests**
* Expanded integration tests to cover iOS VPP app installations
alongside macOS.
* Added checks to verify refetch requests and correct MDM command
behavior after app installs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
For #29400.
Added test fails without the change to retry upon deadlocks.
How to reproduce in UI:
1. Create 10 policies on a team.
2. Refetch host to have results for the policies.
3. Add (could be the same) or update the installer associated to the 10
policies in "Manage automations" > "Software".
4. Hit `Save`.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved the reliability of policy updates on the "Manage automations"
page by automatically retrying requests in case of deadlock errors.
* **Tests**
* Added a test to verify that concurrent policy updates handle deadlocks
correctly and complete without errors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This was added to support the "All Software" when listing software on
the host.
Fixes#30188
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added an option to explicitly exclude software available for install
from host software listings.
* **Bug Fixes**
* Improved accuracy of software inventory results when filtering by
availability for install.
* **Tests**
* Added a test to verify exclusion of available-for-install software
when the relevant option is set.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
Fixes#29916.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Host vitals data now refreshes automatically after successful software
installation or uninstallation, ensuring up-to-date status information.
* **Tests**
* Enhanced tests to verify that host vitals are only refreshed after
successful software changes, improving reliability and accuracy of the
system’s behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Maps are reference types in Go, so the called function can modify the
map even when it is passed by value (because the pointer is passed by
value, pointing to the same underlying map). This simplifies the code.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
For #30385.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
> Fixes ##28738
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#30522.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Resolved an issue where host software counts were not updated if the
database contained rows with a zero software ID.
* **Tests**
* Enhanced tests to verify correct handling of host software records
with a zero software ID.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #29867 . Includes latest copy requested by product.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
for #30502
# Details
This PR fixes an issue where `fleetctl generate-gitops` would not always
add a `macos_setup` setting to a .yml file even if the team had a setup
experience configured. This was due to relying on the `MacOSSetup`
config returned by app/team config APIs to have this data populated,
which turned out to be an incorrect assumption. Instead, we now utilize
various APIs to check for the presence of setup software, scripts,
bootstrap packages and profiles.
Note that for now, `generate-gitops` will only output a `TODO` line if
setup experience is detected;
https://github.com/fleetdm/fleet/issues/30210 is open to flesh this out.
In the meantime `fleetctl gitops` will fail if this TODO is inserted, so
that the user must go and fix it manually.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
# Testing
I set up MDM on a local instance and tried the following both on No Team
and a regular team:
* Turned "End user authentication on", verified that `fleetctl
generate-gitops` output a `macos_setup` setting for the team. Turned it
back off and verified that `macos_setup` was no longer exported by
`fleetctl generate-gitops`.
* Did the same for bootstrap package.
* Did the same for install software, and additionally verified that
having software available but _not_ selected did not cause `macos_setup`
to be exported. Same for teams with no software available at all.
* Did the same for setup assistant.
I also tested that changes to No Team didn't affect the output when
exporting a regular team.
---------
Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
relates to [#28691](https://github.com/fleetdm/fleet/issues/28691)
This adds the ability to upload the EULA users see during the setup
experience via gitops. It follows patterns used for uploading the
bootstrap package via gitops.
I've also added a sha256 column to the `eulas` table in order to easily
compare the existing eula with a new one to see if we need to perform an
upload.
Finally I added the support to generate this new gitops setting with the
`generate-gitops` command
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [x] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
- For database migrations:
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> Fixes#29851
> Fixes#29902
> Mainly followups from https://github.com/fleetdm/fleet/pull/30295,
plus improved integration testing
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
#29482
[Migrate to the AWS SDK for Go
v2](https://docs.aws.amazon.com/sdk-for-go/v2/developer-guide/migrate-gosdk.html)
documents how to migrate codebases.
QA on features that use AWS SDK Go:
- Bootstrap package:
- upload: ✅
- download: ✅
- cleanup: ✅
- Software (upload, download, installation, etc.) ✅
- Cloudfront: Luckly, this feature was already using aws-sdk-go-v2.
- Carves ✅
- Logging:
- Firehose ✅
- Kinesis ✅
- Lambda ✅ (tested result logs to a lambda function on our AWS Dogfood
account)
- Email:
- Amazon SES TODO ⚠️ (this is what Dogfood uses and a few customers)
- We cannot easily test locally, we can use dogfood or load testing
(AWS) environments.
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Manual QA for all new/changed functionality
# Details
This PR adds on to the https://github.com/fleetdm/fleet/pull/30278 which
added support for host vitals labels, by adding a cron job which updates
host vitals label membership every 5 minutes.
Unlike "dynamic" label types, where the hosts determine membership
themselves and report their decision to Fleet when they check in, "host
vitals" label membership is determine by Fleet. This means they can be
applied to hosts which don't check in at the `/distributed/write`
endpoint (like mobile devices).
The mechanism in the cron job is pretty naïve, it just lists all the
labels, post-filters for "host vitals" labels and updates membership for
each. Since the # of labels on an instance tends not to be excessive,
and since updating membership consists of one `DELETE` query and one
`INSERT...SELECT` query, this is not expected to contribute significant
load, but load testing should verify this.
# Checklist for submitter
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
# Testing
Tested by manually adding scim groups, users etc. in the db and adding a
couple of new host vitals labels using the API. I've uploaded a folder
containing a db snapshot and creds to [Google Drive](
https://drive.google.com/drive/folders/1pDlg2XtS139d3sxq9iFqFs6vez8LeUgg?usp=sharing).
To use it, create a new folder
`~/.fleet/snapshots/test_host_vitals_labels`, download the `db.sql.gz`
file into it, then do `fdm restore --prep` and select
"test_host_vitals_labels". After starting the server you can trigger the
new job using `fleetctl trigger --name host_vitals_label_membership` or
wait five minutes.
New automated tests were added for a small change to the `GetLabels()`
method, and for the new cron job. Tests for other functionality were
added in https://github.com/fleetdm/fleet/pull/30278.
For #29609.
From @marko-lisica:
> In #23900 we decided to error if $FLEET_VAR_HOST_END_USER_IDP_GROUPS
or $FLEET_VAR_HOST_END_USER_IDP_USERNAME is added to a profile and the
host doesn't have a username or groups assigned to it. We decided that
because we think that if user deploys profile with variables above then
user would expected that host have some value for those. I think we
should be consistent with departments.
Context:
https://fleetdm.slack.com/archives/C084F4MKYSJ/p1751281745736999?thread_ts=1751025745.430259&cid=C084F4MKYSJ.
# Details
This PR adds support for a new label membership type, `host_vitals`.
Membership for these labels is based on a database query created from
user-supplied criteria. In this first iteration, the allowed criteria
are very simple: a label can specify either an IdP group or IdP
department, and hosts with linked users with a matching group or
department.
Groundwork is laid here for more complex host vitals queries, including
`and` and `or` logic, different data types and different kinds of vitals
(rather than just the "foreign" vitals of which IdP is an example).
Note that this PR does _not_ include the cron job that will trigger
membership updating, and it doesn't include ; for sake of simplicity in
review that will be done in a follow-on PR.
## Basic flow
### Creating a host vitals label
1. A new label is created via the API / GitOps with membership type
`host_vitals` and a `criteria` property that's a JSON blob. Currently
the JSON can only contain `vital` and `value` keys (and must contain
those keys)
2. The server validates that the specified `vital` exists in our [set of
known host
vitals](https://github.com/fleetdm/fleet/pull/30278/files#diff-b6d4c48f2624b82c2567b2b88db1de51c6b152eeb261d40acfd5b63a890839b7R418-R436).
3. The server validates that the [criteria can be parsed into a
query](https://github.com/fleetdm/fleet/pull/30278/files?diff=unified&w=1#diff-4ac4cfba8bed490e8ef125a0556f5417156f805017bfe93c6e2c61aa94ba8a8cR81-R86).
This also happens during GitOps dry run.
4. The label is saved (criteria is saved as JSON in the db)
### Updating membership for a host vitals label
1. The label's criteria is used to generate a query to run on the
_Fleet_ db.
1. For each vital criteria, check the vital type. Currently only foreign
vitals are supported.
2. For foreign vitals, add its group to a set we keep track of.
3. Add a `WHERE` clause section for the vital and value, e.g.
`end_user_idp_groups = ?`
4. Once we have all the `WHERE` clauses, create the query as `SELECT %s
FROM %s` + any joins contributed by foreign vitals groups + `WHERE ` +
all the `WHERE` clauses we just calculated. The `%s` provide some
flexibility if we want to use these queries in other contexts.
2. Delete all existing label members
3. Do an `INSERT...SELECT` using the query we calculated from the label
criteria. The query will be `SELECT <label id> as label_id, hosts.id
FROM hosts JOIN ...`
## Future work
### Domestic vitals
These can be anything that we already store in the `hosts` table.
Domestic vitals won't add any `JOIN`s to the calculated label query, and
will simply be e.g. `hosts.hostname = ?`
### Custom vitals
We currently support an `additional_queries` config that will cause
other queries to run on hosts. The data returned from these queries is
stored in a `hosts_additional` table as a JSON blob. We can use MySQL
JSON functions to match values in this data, e.g.
`JSON_EXTRACT(host_additional, `$.some_custom_vital`) = ?`
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
> I'll add the changelog item when I add the cron job PR
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
#29609
Verified the changes with [Entra ID's
validator](https://scimvalidator.microsoft.com/) and adding the
department attribute to the tester:
<img width="1312" alt="Screenshot 2025-06-27 at 8 54 32 AM"
src="https://github.com/user-attachments/assets/45a5deb8-7c65-49df-b3e8-eb05bea11f6b"
/>
<img width="1312" alt="Screenshot 2025-06-27 at 8 54 21 AM"
src="https://github.com/user-attachments/assets/91b554b5-b0b9-4bb6-a0cf-4e3b40e6ce21"
/>
- Tested with Okta
- TODO: Test with Entra ID and Google Workspace.
- I decided to not fail profile deployment if a user has no department
because it's not a required attribute, instead the
`FLEET_VAR_HOST_END_USER_IDP_DEPARTMENT` will be replaced with the empty
string.
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] If database migrations are included, checked table schema to
confirm autoupdate
(https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
Fixes#30229. No changes file since this is a vulns feed fix not tied to
a release.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#29618, #30282.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#28207.
No changes file since this will go live once merged, and it affects the
vulns feed. Tests will fail for vulns on-branch since this requires a
vulns feed rebuild.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> No issue, fixes broken test
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
Fixes#28499
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
## Issue
Closes#26258
## Description
Returns 0 hosts instead of some random VPP error when software_status is
valid but software_title_id doesn't exist on that team
## Screenshot of fix
<img width="1186" alt="Screenshot 2025-06-23 at 2 04 52 PM"
src="https://github.com/user-attachments/assets/577cc05a-c8e4-4aaf-85c4-38ab9403018b"
/>
## Screenshot of before
<img width="1176" alt="Screenshot 2025-06-23 at 1 50 40 PM"
src="https://github.com/user-attachments/assets/cb0b6ccd-79dd-4309-ae5d-c1c1b938292d"
/>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
For #30199
The hottest path for these changes is the Orbit config getter, which
runs every 30 seconds for each host. That means that for 10k enrolled
hosts this will save ~333 prepares per second...which adds up.
There are a few other places that use this query, but not on as hot of a
path.
Safe despite not using prepared statements because you can't SQL-inject
a number. Existing tests cover this path well, hence no new ones.
Needs manual MDM testing.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
For #30199. This is one of a few approaches to mitigate the issue the
customer is seeing.
This is SQLi-safe because we're dealing with an unsigned int parameter,
sprintf'd %d. Existing tests fully cover this path.
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Manual QA for all new/changed functionality
Confirmed that both CVEs apply to this version by checking against NVD.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
#28621
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
> Closes#29506
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes [#30059](https://github.com/fleetdm/fleet/issues/30059)
Fixes an issue where you couldn't delete a bootstrap package. The issue
was an unused json struct tag for DryRun on the
`deleteBootstrapPackageRequest` struct.
I also updated the UI to use the current endpoint to delete a bootstrap
package
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
For #30118. Not sure why this worked before; CVE feeds must have been
inaccurate given the CVE description.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
Fixed#21947.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Adds support for the Apple MDM user channel however we are waiting on
stories around verification among other things for this and we are not
shipping as part of 4.70 so this can be reviewed but should not be
merged yet
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
For #26519
This PR allows Fleet server to use Android with either fleetdm.com proxy
or locally. It also removes the Android feature flag from the backend.
The frontend changes and proxy API documentation will be in separate
PRs.
Updated contributor docs:
https://github.com/fleetdm/fleet/pull/29880/files
Integration tests are missing and tracked as a separate issue:
https://github.com/fleetdm/fleet/issues/27080
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #29958
Found during 4.69.0 load testing.
We need to call disablePoliciesDurignSetupExperience after calling
GetHostPolicyReportedAt and running the time-based logic which often
results in us not contacting the DB for policies at all(since the time
based logic just hits redis)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
Re-ordering migrations for cherry-picking #27042 into v4.69.0.
Migrations in `main`:
```
[...]
20250519161614_AddExecutableSha256HostSoftwareInstalledPaths.go
20250520153848_AddCancelledStatusToSetupExperienceStatusResults.go
20250528115932_AddInviteIDToUsersTable.go
20250529102706_SetMediumTextColumns.go
20250603105558_AlterLegacyEnrollRefsUUIDSize.go
20250609102714_AddHostCertificateSourcesTable.go
20250609120000_MicrosoftCompliancePartner.go
````
Migrations in rc-minor-fleet-v4.69.0:
```
[...]
20250519161614_AddExecutableSha256HostSoftwareInstalledPaths.go
```
So we have to move this one between
`20250519161614_AddExecutableSha256HostSoftwareInstalledPaths.go` and
`20250520153848_AddCancelledStatusToSetupExperienceStatusResults.go`.
For #21979
Extended POST /api/v1/fleet/hosts/:id/wipe end-point to allow users to
specify an optional payload for specifying what type of remote wipe to
perform on Win hosts.
---------
Co-authored-by: Rachael Shaw <r@rachael.wtf>
For #27042.
Ready for review, just missing integration tests that I will be writing
today.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
---------
Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com>
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
For #29614.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- For new Fleet configuration settings
- [x] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. - Excluded
from GitOps (env var)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #28846.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
For #26375.
When listing software titles for 'All teams', do not join against
software installers nor vpps to avoid duplicates.
Since filters related to software installers/VPP apps are no longer used
when viewing titles for 'All teams', the filter dropdown is disabled if
'All teams' is selected.
Tests (automated and manual) and contributor API docs will arrive in
another PR
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
for #29443
# Checklist for submitter
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
## Details
This PR adds two new `HostListOptions` for the `ListHosts` method:
* `BatchScriptExecutionIDFilter`: if set, only returns hosts that were
targeted by the specified batch script run
* `BatchScriptExecutionStatus`: if set, further filters hosts by the
status of their script execution in the specified batch
## Testing
I added new automated tests that check:
* Starting a batch script run on 5 hosts and calling `ListHosts` with
only `BatchScriptExecutionIDFilter` set returns all 5 targeted hosts
* Calling `ListHosts` with `BatchScriptExecutionStatus` set to
`pending`, `run`, `errored` or `cancelled` returns the expected set of
hosts
* Simulating various script run outcomes (success, failure, cancel) and
calling `ListHosts` with each status again returns the expected set of
hosts
I also tested manually by executing a batch script in the UI and calling
the API via Postman with each status.
Fixes (partially) #29720.
Split from website changes since this has different reviewers. Assigned
to MDM for review since the non-comment change here touches setup
experience.
- [x] Changes file added to `orbit/changes`
Fixes part of unreleased for #28846.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
Also adds some more rate limiter tests to make sure separate rate limit
buckets interact as expected.
Fixes#29614.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- For new Fleet configuration settings
- [x] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. (excluded;
env var or YAML)
- [x] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
---------
Co-authored-by: George Karr <georgekarrv@users.noreply.github.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
fixes#29434
This fixes an issue where a dry run on gitops would not show a failure
if a user was trying to upload a non distributed bootstrap package.
- [x] Manual QA for all new/changed functionality
Fixes#28589
Previously "include any" was not being properly honored when we queried
profiles for verification and so we never would actually verify profiles
where the user had an include any rule with multiple labels and the host
included a subset of the labels. Updated the query that returns profiles
to verify for Windows and Apple to return those profiles with "include
any" labels and a nonzero number of the targeted labels applied
Also rearranged and refactored the associated tests slightly and added a
single test that does the various permutations of "include all" "include
any" and "exclude any"
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Env var: `FLEET_MDM_SSO_RATE_LIMIT_PER_MINUTE`. **Not** managed via
GitOps.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- For new Fleet configuration settings
- [x] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps.
- [ ] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
