<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#44723
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Strengthened validation of sorting/order parameters across many list
and cursor-based endpoints — unsupported sort keys now return explicit
errors and prevent unsafe queries.
* Labels listing: label-list pagination query name changed; ordering by
host_count is rejected when host counts are disabled (validated at
request parsing).
* **Tests**
* Added/expanded tests covering allowed order keys, rejection of unknown
keys, and pagination behavior for multiple listing APIs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed filtering in the /api/v1/fleet/labels/:id/hosts endpoint and
tightened validation to reject invalid sort/order keys with HTTP 422
responses.
* Enforced ordering restrictions tied to feature flags (issues and
device-mapping), rejecting unsupported order_key values.
* **Tests**
* Added extensive integration tests for order_key validation,
deterministic sorting across allowed keys, and cursor pagination.
* **Documentation**
* Added a changelog entry noting the hosts-in-label filtering fix.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43943
# Checklist for submitter
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
See
https://github.com/fleetdm/fleet/issues/43943#issuecomment-4329658412
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
(see https://github.com/fleetdm/fleet/pull/43877/changes)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled (should be done by
https://github.com/fleetdm/fleet/issues/43947)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a configuration option to preserve host activities during host
re-enrollment, letting admins choose whether activity history is
retained when hosts re-enroll.
* **Chores**
* Updated defaults and database migration state so the new setting is
present in stored and generated configs and in GitOps outputs.
* **Tests**
* Added unit, integration, migration, and GitOps fixtures to validate
behavior, serialization, and upgrade semantics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #36087
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added script execution API supporting asynchronous and synchronous
operations with timeout handling.
* Introduced batch script execution capabilities including batch run
creation, status querying, and execution cancellation.
* Added host management API endpoints for locking, unlocking, and wiping
devices.
* Enhanced script management with create, update, delete, list, and
retrieval operations.
* Improved file download responses with proper content headers and
attachment handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#42885
Added new middleware (APIOnlyEndpointCheck) that enforces 403 for
API-only users whose request either isn't in the API endpoint catalog or
falls outside their configured per-user endpoint restrictions.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43047
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
See
https://github.com/fleetdm/fleet/issues/42960#issuecomment-4244206563
and subsequent comments.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Apple DDM declarations support a vetted subset of Fleet variables with
per-host substitution; premium license required. Declaration tokens and
resend behavior now reflect variable changes; unresolved host
substitutions mark that host’s declaration as failed.
* **Bug Fixes**
* Clearer errors for unsupported or license-restricted Fleet variables
and more consistent DDM resend/update semantics when variables change.
* **Tests**
* Added extensive unit and integration tests covering Fleet variable
validation, substitution, token changes, resends, and failure states.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #36087
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Refactor**
* Consolidated and centralized request/response type definitions for
query and scheduled query API operations
* Updated internal service handlers and client code to use unified type
structures
* Improved code consistency and reduced duplication across query-related
endpoints
* **Tests**
* Updated integration tests to align with new API type organization
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#42883
Added a new premium GET /api/_version_/fleet/rest_api endpoint that
returns the contents of the embedded `api_endpoints.yml` artifact.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40809
**Orbit agent: key rotation replaces decrypt-then-re-encrypt:**
- When the disk is already encrypted, orbit now adds a new Fleet-managed
recovery key protector, removes old ones, and escrows the new key. The
disk is never decrypted.
- If key escrow fails, the rotated key is cached in memory and retried
on subsequent ticks without rotating again.
- Removes `DecryptVolume` and `decrypt()` (no longer called from
production code).
**Server: osquery query returns both protection_status and
conversion_status:**
- The `disk_encryption_windows` query now returns both columns instead
of just checking `protection_status = 1`. This lets the server correctly
identify a disk as encrypted via `conversion_status = 1` even when
`protection_status = 0`.
- New `directIngestDiskEncryptionWindows` function parses both values,
handles parse errors, and normalizes `protection_status = 2` (unknown)
to NULL.
**Server: new `bitlocker_protection_status` column and status logic:**
- Adds `bitlocker_protection_status` column to `host_disks` (DB
migration).
- When a disk is encrypted and key is escrowed but protection is off,
the host shows "Action required" with a detail message explaining the
issue, instead of misleadingly showing "Verified."
- `protection_status = 2` (unknown) and `NULL` (older orbit hosts) are
treated as protection on for backward compatibility.
- The `profiles_verified` and `profiles_verifying` branches in the
combined profiles+BitLocker summary now handle
`bitlocker_action_required`, counting those hosts as "pending".
Contributor docs updates: https://github.com/fleetdm/fleet/pull/43241
Public docs updates: https://github.com/fleetdm/fleet/pull/43243/changes
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Bug Fixes**
* Fixed Windows BitLocker encryption/decryption request loop on systems
with secondary drives and auto-unlock.
* **New Features**
* Added BitLocker recovery key rotation capability, allowing safe key
updates without full disk re-encryption.
* Enhanced BitLocker protection status tracking to correctly display
"Action required" when protection is disabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42836
This is another hot path optimization.
## Before
When a host submits policy results via `SubmitDistributedQueryResults`,
the system needed to determine which policies "flipped" (changed from
passing to failing or vice versa). Each consumer computed this
independently:
```
SubmitDistributedQueryResults(policyResults)
|
+-- processScriptsForNewlyFailingPolicies
| filter to failing policies with scripts
| BUILD SUBSET of results
| CALL FlippingPoliciesForHost(subset) <-- DB query #1
| convert result to set, filter, queue scripts
|
+-- processSoftwareForNewlyFailingPolicies
| filter to failing policies with installers
| BUILD SUBSET of results
| CALL FlippingPoliciesForHost(subset) <-- DB query #2
| convert result to set, filter, queue installs
|
+-- processVPPForNewlyFailingPolicies
| filter to failing policies with VPP apps
| BUILD SUBSET of results
| CALL FlippingPoliciesForHost(subset) <-- DB query #3
| convert result to set, filter, queue VPP
|
+-- webhook filtering
| filter to webhook-enabled policies
| CALL FlippingPoliciesForHost(subset) <-- DB query #4
| register flipped policies in Redis
|
+-- RecordPolicyQueryExecutions
CALL FlippingPoliciesForHost(all results) <-- DB query #5
reset attempt counters for newly passing
INSERT/UPDATE policy_membership
```
Each `FlippingPoliciesForHost` call runs `SELECT policy_id, passes FROM
policy_membership WHERE host_id = ? AND policy_id IN (?)`. All 5 queries
hit the same table for the same host before `policy_membership` is
updated, so they all see identical state.
Each consumer also built intermediate maps to narrow down to its subset
before calling `FlippingPoliciesForHost`, then converted the result into
yet another set for filtering. This meant 3-4 temporary maps per
consumer.
## After
```
SubmitDistributedQueryResults(policyResults)
|
CALL FlippingPoliciesForHost(all results) <-- single DB query
build newFailingSet, normalize newPassing
|
+-- processScriptsForNewlyFailingPolicies
| filter to failing policies with scripts
| CHECK newFailingSet (in-memory map lookup)
| queue scripts
|
+-- processSoftwareForNewlyFailingPolicies
| filter to failing policies with installers
| CHECK newFailingSet (in-memory map lookup)
| queue installs
|
+-- processVPPForNewlyFailingPolicies
| filter to failing policies with VPP apps
| CHECK newFailingSet (in-memory map lookup)
| queue VPP
|
+-- webhook filtering
| filter to webhook-enabled policies
| FILTER newFailing/newPassing by policy IDs (in-memory)
| register flipped policies in Redis
|
+-- RecordPolicyQueryExecutions
USE pre-computed newPassing (skip DB query)
reset attempt counters for newly passing
INSERT/UPDATE policy_membership
```
The intermediate subset maps and per-consumer set conversions are
removed. Each process function goes directly from "policies with
associated automation" to "is this policy in newFailingSet?" in a single
map lookup.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance Improvements**
* Reduced redundant database queries during policy result submissions by
computing flipping policies once per host check-in instead of multiple
times.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** The entire ACME feature branch merge
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Jordan Montgomery <elijah.jordan.montgomery@gmail.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Resolves#40396.
No changes file because there should be no user visible changes.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] Verified that fleetd runs on macOS, Linux and Windows
- [X] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41533
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [ ] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
https://github.com/user-attachments/assets/64a5f726-1e9f-4508-8726-6227813dcc77
Below I show the `Report clipped` and the `X additional results not
shown` states. For that, I manually inserted records in my DB:
```sql
-- make "clipped"
INSERT INTO query_results (query_id, host_id, last_fetched, data)
SELECT 1, t.n + 1000, NOW(), '{"fake_key": "fake_value"}'
FROM (
SELECT a.N + b.N * 10 + c.N * 100 AS n
FROM (SELECT 0 AS N UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION
SELECT 9) a,
(SELECT 0 AS N UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION
SELECT 9) b,
(SELECT 0 AS N UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION
SELECT 9) c
) t
WHERE t.n BETWEEN 1 AND 999;
-- populate extra query results
INSERT INTO query_results (query_id, host_id, last_fetched, data)
VALUES
(1, 2, NOW(), '{"pid": "9999", "version": "5.21.0"}'),
(1, 2, NOW(), '{"pid": "8888", "version": "5.20.0"}');
```
https://github.com/user-attachments/assets/8056ea4c-b042-47cf-a05f-ee9d8621252a
Pagination (manually changed to 3 items per page for testing purposes)
https://github.com/user-attachments/assets/87a97259-0821-4659-a612-c952e98a158c
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42008
# Details
Step one in https://github.com/fleetdm/fleet/issues/40171.
This PR adds a new `exceptions` subsection to the current GitOps config,
with boolean keys for software, secrets and labels. For existing
instances a migration is included to set labels and secrets to `true`.
For new instances, only `secrets` will be `true`.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
n/a, will put changelog in when more functionality is implemented.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
(https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] ran migration and verified that app config had `gitops.exceptions`
with `software: false, secrets: true, labels: true`
- [X] created a new instance and verified that that app config had
`gitops.exceptions` with `software: false, secrets: true, labels: false`
- [X] verified that the PATCH /config API works and can update
exceptions independently of other config
## Database migrations
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
n/a
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
n/a
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
n/a
## New Fleet configuration settings
- [X] Setting(s) is/are explicitly excluded from GitOps
these will not be set in GitOps, since they're _about_ how GitOps works.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* GitOps configuration now supports exception settings for granular
resource control. Administrators can configure which specific resource
types (labels, software, and secrets) are included in or excluded from
GitOps mode operations.
* **Improvements**
* Improved GitOps configuration handling to preserve exception settings
during partial updates and system migrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41672
# Details
Updates GitOps label functionality so that omitting the `hosts:` key
under a manual label will _not_ clear hosts from that label, but will
instead preserve the existing membership. This allows users to manage
manual hosts with an external system (via the labels API), while still
managing the labels themselves in GitOps.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- [X] verified that you can still add a manual label with `hosts:`
- [X] verified that leaving `hosts:` off a manual label doesn't change
the host assignment
- [X] verified that putting `hosts:` with no value on a manual label
clears the hosts
- [X] verified that you can still add a dynamic label
- [X] verified that generate-gitops still exports manual hosts
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41603
# Details
Fixes an issue where the nested `query` key in the get, create and
update query API responses, which is the literal SQL query, was getting
duplicated into a `report` key with the SQL. This was happening because
our JSON field duplicator which adds the renamed version of deprecated
keys to responses is intentionally naive; it doesn't account for cases
where the parent and child structs have the same key with different
meanings because that is... not ideal. In Fleet 5 we won't have this
problem since it'll just be `report.query`, but for now the solution is
to just hard-code a `Report` field onto the response structs, rather
than over-complicate the duplicator code to account for different
nesting levels.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- [X] verified no `query.report` or `report.report` present in get query
response
- [X] verified no `query.report` or `report.report` present in create
query response
- [X] verified no `query.report` or `report.report` present in modify
query response
Resolves#41534
Added GET /api/v1/fleet/hosts/{id}/reports endpoint (also accessible as
/hosts/{id}/queries) that lists the query reports associated with a
specific host.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41091
# Details
Implements the following config key aliases:
- [x] Add a second name for `bootstrap_package`:
`macos_bootstrap_package`
- Support `bootstrap_package` for backwards compatibility
- [x] Add a second name for `manual_agent_install`:
`macos_manual_agent_install`
- Support `manual_agent_install` for backwards compatibility
- [x] Add a second name for `enable_release_device_manually `: `apple_
enable_release_device_manually `
- Support `enable_release_device_manually` for backwards compatibility
- [x] Add a second name for `script`: `macos_script`
- Support `script` for backwards compatibility
Also cleans up some error messages missed in previous alias PRs.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
ran gitops successfully with new keys
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#41074
# Details
Fixes an issue where CSV export still has `team_name` and `team_id`
columns, but not `fleet_name` or `fleet_id`.
Unlike the API param and other renames, I took a manual approach here
since it's just the two fields and isn't likely to expand. I added
cleaning them up to my Fleet 5 punchlist.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
- [X] exported report from UI, saw both team_name and fleet_name
- [X] exported report via API with no columns requested (so all columns
returned), saw team_id, team_name, fleet_id and fleet_name
Resolves#33045
When creating or updating queries either via the API or Gitops, make
sure that the license is premium if the query is targeting a label.
Resolves#40813
* Added configurable body size limits for the `/api/osquery/log`,
`/api/osquery/distributed/write` and `/api/osquery/config` endpoints.
* Fixed false positive `PayloadTooLargeError` errors.
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38536
This PR moves all logic to create new activities to activity bounded
context.
The old service and ActivityModule methods are not facades that route to
the new activity bounded context. The facades will be removed in a
subsequent PR.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added webhook support for activity events with configurable endpoint
and enable/disable settings.
* Enhanced automation-initiated activity creation without requiring a
user context.
* Improved activity service architecture with centralized creation and
management.
* **Improvements**
* Refactored activity creation to use a dedicated service layer for
better separation of concerns.
* Added support for host-specific and automation-originated activities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #40348
# Details
This PR updates a number of error message on the server to use `fleet`
and `report` instead of `team` or `query` where applicable.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
this is all internal, i don't think it warrants a changelog
## Testing
- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
I did not go trying to trigger all these errors. It's text changes.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #39344
# Details
This PR builds on the previous PR
(https://github.com/fleetdm/fleet/pull/39847) which added `renameto`
tags to certain API parameters to mark them as deprecated. How this is
used:
### In requests
* When decoding requests, log a warning if a `json` or `query` param is
used that has a `renameto` tag, e.g. if a `team_id` param is sent but
the related struct has `renameto:"fleet_id"` in it.
* If the `renamedto` version (e.g. `fleet_id`) is sent in the request,
rewrite it to the deprecated name so that it can be unmarshalled into
the struct
* If both versions are sent (e.g. `team_id` AND `fleet_id`), throw an
error and quit
* URLs with deprecated terms have new aliases using `WithAltPaths` --
warning on using old URLSs a TODO that will be handled in a subsequent
PR.
### In responses
* Output _both_ the deprecated and new names for fields that have
`renameto` tags, so that we don't break existing workflows expecting the
old keys. Uses a shared `DuplicateJSONKeys` to do the duplication.
* Most API responses are handled in `EncodeCommonResponse`. Exceptions
are activities, failing policy webhooks and the streaming "list hosts"
endpoints which call the function directly.
### In fleetctl
* Similar to requests, log warnings when deprecated keys are used and
rewrite the new keys internally so that they can be unmarshalled.
* For `fleetctl get` and `fleetctl generate-gitops`, _only_ output the
new names
* The set of keys to replace is hardcoded in `fleetctl` rather than
being dynamically generated as it is for API endpoints. Given the
mixture of typed and untyped data and the level of nesting, dynamic map
generation was very fragile and error-prone.
### Performance considerations
* The biggest performance hit is the addition of the JSON key rewriter
to the request pipeline. The rewriter buffers the entire request into
memory before eventually passing it to the decoder than unmarshals the
data into structs. I tried implementing this as a true streaming
rewriter but encountered issues where the request would hang if the
downstream reader (the decoder) encountered any errors. It's possible we
could implement this in a streaming fashion if we replace our [current
request
decoder](da43bf8371/server/service/endpoint_utils.go (L108))
with the v2 version, which is a bigger change requiring more thoughtful
discussion in the engineering team. As it stands, memory usage for
requests with deprecated fields will double while the request is being
decoded.
* The "alias rules" used to determine the old and new key names are
cached per struct type and for most endpoints are generated on server
start, so no performance impact is expected.
* Some `fleetctl` commands may have an extra unmarshal/marshal step but
as these are user-initiated and not performed in tight loops, the impact
should be minimal.
### TODO
* Log deprecation warnings when old URLs like "/fleet/teams" are used
* Update API fields that the front-end uses to avoid deprecation
warnings
* Update `fleetctl apply` to accept/return `kind: fleet` rather than
`kind: team`
* Find/update any fleet server config vars with old language
* Update all error messages that use old language
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
* Clicking around the front-end, no broken pages due to request
ingestion errors or bad responses
* Looking in network tab to verify that responses have both the old and
new keys
* Running `fleetctl generate-gitops` and verifying that the output looks
correct and can be ingested by `fleetctl gitops`
* Running `fleetctl get` and `fleetctl apply`
---------
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38889
Plan was to convert `osquery_utils` package to slog. Picked up some
additional code that was related.
# Checklist for submitter
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- Already have changes
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
# Release Notes
## Refactor
* Updated internal logging infrastructure to use improved system-level
logging utilities
## Tests
* Updated test suite to align with internal logging changes
---
**Note:** This release contains internal infrastructure improvements
with no user-facing changes or new features.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolves#38621, #38627, and #38623.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added a new premium-tier Technician role with read/write permissions
across teams, hosts, policies, queries, and configurations.
* License validation now prevents assigning premium roles on Fleet Free
editions.
* **Bug Fixes**
* Updated role-based access controls across team management pages to
properly restrict technician access.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39304 (part of #25080)
Implemented similar approach to
https://github.com/fleetdm/fleet/pull/39302, with the difference that
the list policies endpoint does not include a count, and there is a
separate endpoint. I extended the count policies endpoint to include an
`inherited_policy_count`.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39303 (child of #25080).
- Added `inherited_query_count` to `ListQueriesResponse` (thought of
adding a brand new endpoint just for counting, but felt like extending
the current one was good enough). In the parent task, [it was
suggested](https://github.com/fleetdm/fleet/issues/25080#issuecomment-3326071574)
to `"Depend on team list entity endpoint's count field / team entity
count endpoint for whether or not to disable the manage automations
button"`, which Rachael approved, so I went for this approach.
- The `ManageQueryAutomationsModal` now fetches its own data with
`merge_inherited = false` (meaning it only fetches non-inherited queries
only). Previously, queries were passed down as props to it, which would
not show the queries available to automate if the first page of queries
were all inherited and the second page contained queries for that team
(the user would have to navigate to the second page for the button to be
enabled).
^ The fact that the modal fetches its own data is similar behavior to
what is currently done in `Policies`. For queries, I noticed that we
would need to add pagination within the `Manage Automations` modal, but
that can be a follow-up.
<img width="2480" height="1309" alt="Screenshot 2026-02-04 at 11 48
42 AM"
src="https://github.com/user-attachments/assets/ebac79a5-a793-4708-9313-d9a697dfd7de"
/>
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
https://github.com/user-attachments/assets/119f03b9-dde1-4bb9-9fee-6204b1a58879
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38878 and #38879
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
---------
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- After doing a "forgot password", then logging in and changing password
manually, the "reset password" link I received no longer allows
resetting password
- Changing my password via the UI clears the my session and forces
re-login
- Changing another user's password via the UI clears their session and
forces re-login
- Upon first login for a new user, after forced password change, user's
session is _not_ cleared (they continue to home screen as expected)
- After clicking "require password reset" for a user, logging in as that
user, and changing password, user's session is _not_ cleared (they
continue to home screen as expected)
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35603
# Details
This PR aims to optimize the system for recording scheduled query
results in the database. Previously, each time a result set was received
from a host, the Fleet server would count all of the current result rows
in the db for that query before deciding whether to save more. This
count becomes more expensive as the DB size grows, until it becomes the
"long" pole in the recording process. With this PR, the system changes
in the following ways:
* When result rows are received from the host, no count is immediately
taken. Instead, a Redis key is checked which holds a current approximate
count of rows in the table. If the count is over the configured row
limit, no rows are saved. Otherwise, rows are saved and the count is
adjusted accordingly (it can go down, e.g. if a host previously returned
5 rows for a query and now returns 3). Keep in mind that we only store
one set of results per host for a scheduled query; when a host reports
results for a query, we delete that hosts previous results and write the
new ones if there's room.
* As an additional failsafe against runaway queries, if a result set
contains more than 1000 rows, it is rejected.
* Once a minute, a cron job runs which deletes all rows over the limit
for each query and resets the counter for all queries to the actual # of
rows in the table.
The end result is:
* No more expensive counts on every distributed write request for
scheduled queries
* Results for a single query can burst to over the limit for a short
time, but will get cleaned up after a minute
* Because of concurrency and race issues where multiple hosts might get
the same count from Redis before inserting rows, the actual # of results
in the db can burst higher than the limit. In testing w/ osquery-perf
with 1000 hosts started simultaneously, sending 500 rows at a time, a
50,000 row limit and a query running every 10 seconds, I saw the table
get up to 60,000 rows at times before being cleaned up. This is a very
bad case; in the real world we'd have a lot more jitter in the
reporting, and queries would not typically return this many rows.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
Added a new test to verify that results are still discarded if table
size is > limit, updated existing tests.
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
Ran osquery-perf with 1000 hosts and a 50,000 row limit per query, using
queries that returned 1, 500 and 1000 rows at a time. Verified that the
limits were respected (subject to the amount of flex discussed above).
I'm doing some A/B tests now using local MySQL metrics and will report
back.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Automated periodic cleanup of excess query results to retain recent
data and free storage
* Redis-backed query result counting to track per-query result volumes
* **Performance Improvements**
* Optimized recording of scheduled query results for reduced overhead
* Cleanup runs in configurable batches to lower database contention and
balance storage use
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enhanced authorization validation for certificate template batch
deletion operations, ensuring all templates belong to the specified team
before allowing deletion.
* **Tests**
* Added authorization verification tests for certificate template
deletion to prevent cross-team unauthorized access.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#37806
Removed `ds.ListActivities` from the legacy datastore and updated
code/tests to use the new activity bounded context instead.
The changes to `cron.go` and most changes to `mysql/activities_test.go`
will eventually be migrated to the activity bounded context. The current
changes are an intermediate step.
The issues tracked by https://github.com/fleetdm/fleet/issues/38234 will
be addressed in additional/parallel PRs shortly.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- Done in the previous PR
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Migrated activity retrieval from direct datastore calls to a
service-based architecture for improved maintainability and consistency.
* Enhanced system context handling for background automation tasks to
ensure proper authorization during scheduled operations.
* Streamlined activity recording for automated processes with dedicated
system identity tracking.
* **Tests**
* Updated test infrastructure with new helpers for activity service
integration across test suites.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
