<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#44330, Resolves#44331
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests. (I'd defer integration tests to a
separate PR since this one is pretty large already.)
- [x] QA'd all new/changed functionality manually. I've tested this on
both the setup flow and the organization settings page. I haven't had
the time to test this on other places where we render the logo (macOS
setup experience / MDM migration dialog).
https://github.com/user-attachments/assets/95d4eae5-3da6-40f4-98a1-8575b97d96b3
## New Fleet configuration settings
- [x] Setting(s) is/are explicitly excluded from GitOps.
Will handle GitOps in a separate PR.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Organizations can upload custom logos for light and dark modes.
* Registration and Org Settings support logo file upload, preview,
per-mode replace/delete, and validation (size & image formats).
* Activity feed records logo changes/deletions; site nav displays
uploaded logos per theme.
* File uploader/preview adds a Fleet logo graphic option and improved
logo validation.
* Config/GitOps outputs now include separate dark/light logo fields.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Version bumped to v4.84.2 across Helm charts, container images,
deployment infrastructure, and npm package.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Added Docker Compose configuration to deploy Percona PMM v2
monitoring: includes server and client services, persistent storage for
monitoring data, network connectivity across environments, and secure
agent-to-server communication setup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43928
This PR adds a Redis-backed cache in front of the two host-by-key
lookups on the agent auth paths.
Docs: https://github.com/fleetdm/fleet/pull/44504
## What changes
**Read path (osquery/orbit auth):**
- `LoadHostByNodeKey` and `LoadHostByOrbitNodeKey` now check Redis
before falling through to MySQL.
- Successful lookups are cached for 60s ± 10% jitter (configurable via
`FLEET_REDIS_HOST_CACHE_TTL`).
- `NotFound` results are cached for 5s as a negative entry, dampening
repeated probes for keys that
do not exist (deleted hosts whose agents are still polling, attacker
scans, retry storms).
- Concurrent lookups for the same key collapse into one DB query via
`singleflight`. The shared
query runs under a context detached from any one caller's deadline so
the leader giving up does
not abort the work for joiners. The shared query is itself bounded by a
30s timeout so a wedged
DB call cannot pin the singleflight slot indefinitely.
**Write path (invalidations):**
- These methods now invalidate the cache after a successful inner call:
`UpdateHost`, `SerialUpdateHost`, `UpdateHostOsqueryIntervals`,
`UpdateHostRefetchRequested`,
`UpdateHostRefetchCriticalQueriesUntil`,
`UpdateHostIdentityCertHostIDBySerial`, `EnrollOsquery`,
`EnrollOrbit`, `NewHost`, `DeleteHost`, `DeleteHosts`,
`CleanupExpiredHosts`,
`CleanupIncomingHosts`, `AddHostsToTeam`.
- `AddHostsToTeam`, `DeleteHosts`, `CleanupExpiredHosts`, and
`CleanupIncomingHosts` use a pipelined
batch invalidator so 10k-host operations stay in the millisecond range
instead of taking minutes
of sequential round-trips.
- Inner-call errors are not invalidations: a failing write leaves cached
state intact.
**Configuration:**
- New flags `FLEET_REDIS_HOST_CACHE_ENABLED` (default `true`) and
`FLEET_REDIS_HOST_CACHE_TTL`
(default `60s`).
- Server refuses to start if the cache is enabled with `TTL <= 0`.
**Observability:**
- Three new OTEL counters under the `fleet` meter:
- `fleet.host_cache.lookups{result=hit|negative_hit|miss}`
- `fleet.host_cache.errors{op=get|set|del}`
-
`fleet.host_cache.invalidations{reason=update|enroll|team|delete|cert}`
- A pre-built SigNoz dashboard ships in
`tools/signoz/host_cache_dashboard.json`.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Optional Redis-backed host lookup cache for osquery and orbit auth,
with automatic invalidation and metrics/monitoring dashboard.
* **Bug Fixes**
* Fixed host-removal batching so cache-related removals use correct
chunks.
* **Tests**
* Added comprehensive host-cache unit tests covering hits, negative
cache, invalidation, concurrency, and JSON round-trips.
* **Chores**
* New config flags to enable the cache and set TTL (default 60s ±10%
jitter).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Added app/kilo-code-bot exclude
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Updated internal tooling to recognize an additional bot account in
pull request filtering processes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Fleet version from v4.84.0 to v4.84.1 across deployment
configurations (Helm values, container images, Terraform for AWS/GCP,
and npm package) and bumped Helm chart package version v6.9.1 → v6.9.2.
* **Documentation**
* Updated CLI help/example text to reference the v4.84.1 milestone.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43928
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Redis image version to 6.2 across Docker Compose
configurations for improved stability and compatibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#35173
# macOS 26 Tahoe CIS benchmark v1.0.0 (new benchmark)
Adds a brand-new policy set covering the **CIS Apple macOS 26 Tahoe
Benchmark, v1.0.0** under `ee/cis/macos-26/`. Follows the same layout as
`macos-13`/`-14`/`-15` (`cis-policy-queries.yml`, `README.md`,
`test/scripts/`, `test/profiles/`).
## Coverage
| Section | Title | Status |
|---|---|---|
| 1 | Install Updates, Patches and Additional Security Software |
complete (6/6 automated) |
| 2 | System Settings | complete (all automated across §2.1–§2.18) |
| 3 | Logging and Auditing | complete (5/5 automated) |
| 4 | Network Configurations | complete (3/3 automated) |
| 5 | System Access, Authentication and Authorization | complete (19/19
automated) |
| 6 | Applications | complete (7/7 automated) |
| 7 | Supplemental | skipped (per Fleet convention) |
Total automated policies shipped: **89**. Manual-assessment
recommendations are documented in `ee/cis/macos-26/README.md` under
**Limitations**.
## Notable query/format choices
- **Combined-key profiles per CIS instructions.** §2.2.1+§2.2.2
(Firewall + Stealth Mode) are shipped as a single
`2.2.1-and-2.2.2.mobileconfig` because CIS explicitly requires both keys
in the same profile. §2.6.5 (Gatekeeper) and §2.11.2 (screensaver
wake-password + delay) follow the same pattern.
- **§2.5.2.1 (Siri)** uses the new `allowAssistant=false` key on
`com.apple.applicationaccess`, replacing the deprecated
`com.apple.ironwood.support` payload from earlier benchmarks.
- **§2.6.3.2** uses the spaced literal key `Siri Data Sharing Opt-In
Status` (integer 2) on `com.apple.assistant.support` — the v1.0.0
PayloadType move from `com.apple.applicationaccess`.
- **§5.1.6, §5.1.7, §3.1, §5.7** use fleetd-only osquery tables
(`find_cmd`, `authdb`, `pwd_policy`, `dscl`, etc.) and are flagged
`(Fleetd Required)` in the policy descriptions.
- **§2.10.1.2** (Apple Silicon sleep ≤15 min) default-passes on Intel
hosts via a `system_info.cpu_type` check.
## Test artifacts added
| Type | Count | Location |
|---|---|---|
| Pass scripts | 48 | `ee/cis/macos-26/test/scripts/CIS_*_pass.sh` |
| Fail scripts | 46 | `ee/cis/macos-26/test/scripts/CIS_*_fail.sh` |
| Pass-only scripts | 2 | `CIS_1.1.sh`, `CIS_5.1.6.sh` |
| MDM profiles | 37 | `ee/cis/macos-26/test/profiles/*.mobileconfig` |
Profile-only recommendations (§2.3.1.x AirDrop/AirPlay, §2.5.x Apple
Intelligence, §2.6.3.x Analytics, §6.x Safari/Terminal) ship with a
`.mobileconfig` only and no script counterpart, since CIS marks them as
configurable solely via profile.
## Documentation updates
| File | Change |
|---|---|
| `ee/cis/macos-26/README.md` | New file — coverage table, limitations,
per-section notes (query patterns, fleetd dependencies, FDA
requirements). |
| `ee/cis/CIS-BENCHMARKS.md` | Added `macos-26/` to the directory
layout; updated **Query patterns** doc to include the `EXISTS`/`NOT
EXISTS` user-vs-system-scope guidance and `username = ''` notes. |
| `ee/cis/prompt.md` | Refreshed authoring prompts with macOS-26
conventions (combined-key profiles, fleetd-table flagging). |
| `tools/cis/cis-test-runner.py` | Minor adjustments to support the new
benchmark directory. |
| `changes/35173-cis-macos-26-v1` | User-visible change note. |
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added macOS 26 CIS Benchmark v1.0.0 with comprehensive configuration
profiles to enforce recommended system and app settings (updates,
firewall/stealth, privacy, backups, FileVault, Safari, Terminal, etc.).
* **Tests**
* Added extensive pass/fail remediation and validation scripts for CIS
controls across macOS subsystems; test runner updated to include macOS
26 support and mark an SSH-related control as manual.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Removing some unmaintained/old docker images (Amazon Linux and CentOS),
and keeping most used/updated (Debian, Ubuntu, and Fedora).
Use cases for this:
- I used this to test wiping a linux host (without needing to wipe my
VMs).
- Test fleetd on linux amd64 on Apple Silicon (VMs usually are arm64).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for Fedora 43 and Debian 13.4 container images.
* **Bug Fixes**
* Improved environment variable validation and configuration handling in
the fleetd initialization process.
* Enhanced build process reliability with stricter error handling.
* **Chores**
* Removed support for older container images (Amazon Linux 2023, CentOS
Stream 10, Fedora 41, Debian 12.8).
* Streamlined Docker Compose configuration with improved security
defaults.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #43769
# Details
Adds methods to collect data for the `cve` dataset. As with all sets
this is collected at hourly granularity, but unlike the `uptime` set,
the `cve` set uses the "snapshot" strategy so that we record at most one
change (the most recent) per hour.
For this first iteration, we are _recording_ data for all CVEs (i.e.,
which hosts were exposed to which CVEs at a given time), but we are only
_reporting_ a subset of CVEs for the dashboard chart. See [this
comment](https://github.com/fleetdm/fleet/pull/44124#discussion_r3155554405)
for more info.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
- [X] Spot-checked the CVEs chosen by the `trackedCVESoftwareMatchers`
and didn't find any outside of the expected
- [X] With [front-end PR](https://github.com/fleetdm/fleet/pull/44261),
generated chart:
<img width="706" height="421" alt="image"
src="https://github.com/user-attachments/assets/539d9877-6573-4406-a159-1d2a711a045f"
/>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Host vulnerability (CVE) chart added to the dashboard; CVE chart data
collection is now active.
* Critical CVE tracking surfaces high-severity vulnerabilities.
* **Improvements**
* CVE chart refreshes every 3 hours (was daily) for more timely
insights.
* Snapshot collection reconciles and closes prior data during empty runs
to keep charts accurate.
* CVE queries may produce zero datapoints when no tracked CVEs exist,
without affecting other metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated osqueryd version to 5.23.0 across macOS, Linux, and Windows
build processes.
* Adjusted release tooling behavior so the confirmation prompt only
appears when the corresponding branch/PR creation step will actually
run.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#43943
# Checklist for submitter
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
See
https://github.com/fleetdm/fleet/issues/43943#issuecomment-4329658412
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
(see https://github.com/fleetdm/fleet/pull/43877/changes)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled (should be done by
https://github.com/fleetdm/fleet/issues/43947)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a configuration option to preserve host activities during host
re-enrollment, letting admins choose whether activity history is
retained when hosts re-enroll.
* **Chores**
* Updated defaults and database migration state so the new setting is
present in stored and generated configs and in GitOps outputs.
* **Tests**
* Added unit, integration, migration, and GitOps fixtures to validate
behavior, serialization, and upgrade semantics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Quality of life improvements when testing unmerged changes in osquery
(bundled in fleetd using local TUF).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a build target that fetches a Linux osqueryd executable for a
specified pull request and architecture, validates the downloaded
artifact, extracts and verifies the binary, and installs a runnable
osqueryd into a user-specified output directory. The target reports
clear errors when inputs are missing or when retrieval/validation fails.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Ease testing locally built osquery on macOS (bundled with fleetd).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for packaging locally built osqueryd binaries into
osquery.app tarballs on macOS with automatic validation and cleanup.
* **Improvements**
* Enhanced PR artifact download workflow for improved artifact
retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolves#43671.
Bumps the Alpine base image from 3.23.3 to 3.23.4 in the Dockerfiles
that produce published images, picking up patched openssl, musl, and
zlib packages. Follows the same pattern as #38977.
### CVEs resolved
- HIGH: CVE-2026-28388, CVE-2026-28389, CVE-2026-28390, CVE-2026-31790,
CVE-2026-2673, CVE-2026-40200
- MEDIUM: CVE-2026-27171, CVE-2026-6042, CVE-2026-22184
### Test plan
- CI image build passes.
- Trivy/ECR scan on the resulting fleetdm/fleet image confirms the nine
listed CVEs are gone.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Docker base images to Alpine 3.23.4 across infrastructure and
deployment components for improved stability and security.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42948
- Updated `(mos *MacOSSetup) Validate()` and `(mos *MacOSSetup)
SetDefaultsIfNeeded()` to account for new fields
- Updated default creation and editing for team edit/creation paths
- Updated `generate-gitops` warning message from `macos_setup` to
`setup_experience`
- Updated fields types to optjson and updated test files
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [ ] Timeouts are implemented and retries are limited to avoid infinite
loops
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
- Team edit and team creation through GitOps, validated config with `
curl -k -X GET 'https://localhost:8080/api/v1/fleet/fleets/:id'`
- New error message says `setup_experience` instead of `macos_setup`
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added macOS MDM settings to control local account behavior: enable
managed local accounts (default false) and specify end-user local
account type (default "admin") for fleet and team configs. GitOps output
now highlights unsupported setup-experience cases.
* **Tests**
* Updated fixtures and integration tests to assert and persist the new
macOS local-account settings across config, team, and GitOps scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #42812
# Details
This PR implements a new bounded context, `chart`, with a single
endpoint `/charts`. The context encompasses a framework for recording
and querying and aggregating historical data for Fleet hosts, and
returning that data via the API for the purpose of charting.
This initial iteration has a full implementation of a dataset called
"uptime" which captures which hosts were online hour-by-hour (online
meaning, having been "seen" at some point during that hour). It has a
partial implementation of a "cve" dataset which will capture which hosts
were vulnerable to which CVEs during a given day.
### Data storage
Data is stored in an SCD (slowly-changing dimension) format in the
`host_scd_data` table, where the main "value" in a row is stored in the
`host_bitmap` column, which is a `mediumblob` where each bit encodes a
host ID (bit one represents host ID 1, bit 1444 represents host ID 1444,
etc.). The set of bits set on a row represents that hosts for which that
dataset is "on" during a given time period represented by the
`valid_from` (inclusive) and `valid_to` (exclusive) dates, where a
`valid_to` can have the special "sentinel" value 9999-12-31T00:00:00.000
meaning that the row is still "open" (the value represents everything
from `valid_from` to the present). Additionally an `entity_id` column
can be used for datasets with multiple dimensions, e.g. CVE exposure or
software usage which would have entity IDs representing CVEs or software
items respectively.
### Data collection
Data is collected via a cron job that runs every 10 minutes. Each
dataset has its own `Collect` method which will sample the data for the
given moment. For example the "uptime" dataset gathers the set of hosts
that are online at the moment, and the "cve" dataset will gather the set
of hosts that are vulnerable to each CVE at that moment. The sample can
then be recorded using one of two strategies:
* `accumulate`: bitwise OR the sample with any data already recorded for
the current hour, or add a new pre-closed row for that hour.
* `snapshot`: if there is no open row, create one with the sample and
`valid_to set` to the sentinel. Otherwise:
* If the sample has the same value as the current open row, do nothing
* If the sample has a different value and the current open row's
`valid_from` is within the same hour, update the current row's value
* If the sample has a different value and the current open row's
`valid_from` is not within the same hour, close the current open row and
start a new one with `valid_from` = the start of the current hour
### Data retrieval
1. Gets the set of host IDs to retrieve data for. This starts with the
set of host IDs in the requested fleet (or all the hosts a user has
access to if no `fleet_id` param was passed to the `/charts` endpoint),
and further whittled down by any filter options supplied with the
request (labels, platforms, etc.).
2. Finds all `host_scd_data` rows for the requested dataset and date
range (i.e. all rows whose `valid_from` is < the date range end and
`valid_to` is > the date range start).
3. Calculates the date ranges of the "buckets" to return datapoints for.
For the uptime chart we default to 3-hour buckets, so we want 8 buckets
per day.
4. Iterates over each bucket and finds the row or rows from
host_scd_data that cover that bucket range. For datasets using the
"accumulate" strategy, the values for those rows are ORed together. For
"snapshot"s, we take the one active at the bucket end time to represent
the bucket (e.g. "which hosts had a given CVE at the end of the day")
### Tools
This PR includes two dev tools that don't require deep review:
* **chart-backfill** - used to backfill data to various datasets for
testing
* **charts-collect** - used to collect data from a live server via the
API and put into a local hosts_scd_data table
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
- With [front-end branch](https://github.com/fleetdm/fleet/pull/43878)
<img width="712" height="434" alt="image"
src="https://github.com/user-attachments/assets/b2ccce49-b5fd-4076-b47f-0eea6a53260c"
/>
## Database migrations
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added charting bounded context: HTTP API for metrics (uptime, CVE),
dataset registry, hosted dataset collection, background
collection/cleanup with opt-out env.
* New utilities: host bitmap operations and string-list/uint-list
parsers.
* New CLI tools to collect and backfill chart data.
* **Database**
* Migration and schema to store host time-series SCD chart data.
* **Tests**
* Extensive unit and integration tests for service, storage, caching,
cron, and utilities.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#35172
Updates the macOS 14 (Sonoma) CIS policy set to benchmark v3.0.0, adds a
`cis_id` field to every policy, fixes several broken test scripts,
introduces an automated test runner, and ships `CIS-BENCHMARKS.md` as a
central guide for authoring and maintaining CIS benchmarks.
## Summary of changes
- `ee/cis/macos-14/cis-policy-queries.yml`: v3.0.0 updates + `cis_id`
added to every entry
- `ee/cis/CIS-BENCHMARKS.md`: new authoring/testing/automation guide for
all macOS CIS benchmarks (and the pattern other OS dirs follow)
- `tools/cis/cis-test-runner.py`: new 2150-line Python runner that
drives end-to-end validation against a real tart VM + Fleet server
- `ee/cis/macos-14/test/scripts/`: 10 new pass/fail script pairs, 8
existing scripts fixed (several were silently broken)
# How the automated testing works
The runner (`tools/cis/cis-test-runner.py`) exercises the full policy
lifecycle against a real macOS VM:
**Phases**
1. **Parse** `cis-policy-queries.yml` and filter by `--all`,
`--cis-ids`, `--match`, and type flags (`--only-scripts`, `--only-mdm`,
`--only-manual`).
2. **Classify** each policy into a test type based on available
artifacts:
| Priority | Type | Artifacts | Behavior |
|---|---|---|---|
| 1 | `PASS_FAIL` | `CIS_{id}_pass.sh` + `_fail.sh` | Run fail → verify
query fails → run pass → verify passes |
| 2 | `PASS_ONLY` | `CIS_{id}.sh` | Run script → verify passes |
| 3 | `PROFILE` | `.mobileconfig` only | Verify query fails before
profile → push profile → verify passes |
| 4 | `ORG_DECISION` | paired `-enable`/`-disable` profiles | Toggle
between variants |
| 5 | `MANUAL` | none | Prompt operator, or skip with `--skip-manual` |
3. **Provision**: create a fresh Fleet team with a unique enroll secret,
build a fleetd pkg bound to it, create+boot a tart VM, install the
agent, and enroll.
4. **MDM**: prompt the operator for MDM enrollment if any tests need it.
Clear team profiles, baseline the VM, push all required profiles in one
batch, wait for delivery.
5. **Execute**: for each plan, SCP the script, run it over SSH, then run
the policy via `fleetctl query --hosts <hostname>`. A query that returns
rows = pass.
6. **Report** summary with PASS/FAIL/SKIP/ERROR counts.
7. **Cleanup** (with `--cleanup`) deletes the team, host record, and VM.
**Special-case handling** (keyed by OS version because CIS IDs aren't
stable across releases):
- `SSH_BREAKING_CIS_IDS`: tests that disable sshd (2.3.3.4, 2.3.3.5) are
forced to `MANUAL` so the runner doesn't lock itself out.
- `PASSWORD_POLICY_CIS_IDS`: 5.2.x profiles invalidate the VM's
`admin`/`admin` login — forced to `MANUAL`.
- `NON_AUTOMATABLE_CIS_IDS`: tests that can't run reliably in a VM
(Location Services, Touch ID, shared Siri profile state) forced to
`MANUAL` with a per-entry reason.
- `--keep-vm`: reuses the VM across runs, skipping agent
install/enrollment if the host is already in Fleet. Falls back to fresh
creation if SSH is unreachable.
**Credential resolution order**: CLI flag →
`FLEET_URL`/`FLEET_API_TOKEN` env → `~/.fleet/config` (from `fleetctl
login`).
## How to use `CIS-BENCHMARKS.md` going forward
The doc is the single reference for authoring and maintaining CIS
benchmark policies across all macOS (and Windows) versions. For each new
benchmark release, the workflow is:
1. **Read "Updating benchmarks when a new CIS version is released"** —
directs you to the PDF's *Appendix: Change History* to enumerate
Added/Modified/Removed recommendations.
2. **Use the field reference and query patterns** to write or update
policies: direct table check, `managed_policies` EXISTS/NOT EXISTS, or
plist negation check. Name qualifiers `(MDM Required)` / `(Fleetd
Required)` / `(FDA Required)` are documented.
3. **Create matching test artifacts** — pass/fail scripts for togglable
settings, `.mobileconfig` profiles for MDM-only settings. Script
conventions (full paths, sudo pattern, `not_always_working_` prefix) are
standardized.
4. **Update the per-OS README** with limitations, org-decision policies,
and optional policies.
5. **Run the test runner** to validate.
The doc also contains an **end-to-end AI agent prompt** (section at the
bottom) designed to be handed a new CIS PDF plus the previous version's
PDF, to automatically generate the diff, write policies, produce test
artifacts, update docs, and run validation. This lets future benchmark
updates start from a consistent, repeatable baseline rather than being
hand-authored from scratch.
## Query changes
All entries in `cis-policy-queries.yml` received a `cis_id` field so the
runner (and humans) can map policies → scripts → profiles → the
benchmark document without parsing the display name.
| CIS ID | Change |
|---|---|
| 1.1 | Renamed "Ensure All Apple-provided Software Is Current" →
"Ensure Apple-provided Software Updates Are Installed"; added terminal
remediation to `resolution` |
| 1.6 | Expanded description with v3.0.0 language about rapid security
response updates |
| 1.x (deferment) | **Removed** — "Ensure Software Update Deferment Is
Less Than or Equal to 30 Days" dropped from v3.0.0 |
| 2.3.1.1 | Renamed "Ensure AirDrop Is Disabled" → "Ensure AirDrop Is
Disabled When Not Actively Transferring Files"; expanded description |
| 2.3.3.1 (DVD/CD Sharing) | **Removed** — dropped from v3.0.0 |
| 2.3.3.4 (Remote Login) | Query now checks BOTH `disabled.plist` AND
that `com.openssh.sshd` is not in the `launchd` table; resolution
updated to terminal method |
| 5.1.7 | Query rewritten: sticky-bit dirs now properly excluded
(first-char-of-mode check instead of bit-AND on full mode string);
SIP-protected dirs excluded via `com.apple.rootless` xattr check |
| 6.3.1, 6.3.2, 6.3.3, 6.3.4, 6.3.7 | Dropped `username = ''` filter —
Safari profiles deliver at user scope, so the system-scope filter
guaranteed zero rows |
| 6.3.3 | Fixed `NOT EXISTS` domain typo: `com.apple.loginwindow` →
`com.apple.Safari` (the check was previously meaningless) |
| Wi-Fi/Bluetooth menu bar | **Removed** — "Show Wi-Fi status in Menu
Bar" and "Show Bluetooth status in Menu Bar" dropped from v3.0.0 |
| Show All Filename Extensions | **Removed** — dropped from v3.0.0 |
## Script changes
### New scripts
| Script | Purpose |
|---|---|
| `CIS_1.1_pass.sh` / `_fail.sh` | Install updates (pass); clear
`LastFullSuccessfulDate` (fail — caveat: only works when real updates
are pending) |
| `CIS_1.6_pass.sh` / `_fail.sh` | Open/remove the `1.6.mobileconfig`
profile |
| `CIS_2.3.1.1_pass.sh` / `_fail.sh` | Open/remove the AirDrop profile |
| `CIS_2.3.2.2_pass.sh` / `_fail.sh` | `launchctl load/unload -w` of
`com.apple.timed.plist` |
| `CIS_2.3.3.4_pass.sh` / `_fail.sh` | `systemsetup -setremotelogin
off/on` (runs as MANUAL via the runner's SSH-breaking safeguard) |
### Existing scripts fixed
| Script | Bug | Fix |
|---|---|---|
| `CIS_2.3.3.1.sh` | Disabled `com.apple.ODSAgent` (DVD/CD sharing), not
Screen Sharing | Now disables `com.apple.screensharing` |
| `CIS_2.9.2.sh` | `pmset -a womp 0` sets Wake-on-Network, not Power Nap
| Now `pmset -a/-b/-c powernap 0` |
| `CIS_3.2.sh` | `sed` pipeline into root-owned file via user
redirection silently failed; did nothing if the flags line was missing |
`awk` with `tee`/`mv`; appends a flags line when absent; enforces 0400
root:wheel |
| `CIS_3.3.sh` | Only stripped `all_max=`; never added `ttl=365` when
missing, so the query could never pass from a fresh system | `awk` now
both strips `all_max=` and inserts/updates `ttl=365` on the `* file`
line |
| `CIS_3.4.sh` | `sudo sed … > /etc/security/audit_control` — redirect
runs as caller, not root; write silently failed | Rewrites via
`tee`/`mv` with proper perms; appends when line is absent |
| `CIS_3.5.sh` | `chmod -R o-rw` doesn't produce the exact `0400`/`0440`
modes the query requires | Explicit `chmod 0400` on `audit_control`,
`find … -exec chmod 0440 {}` under `/var/audit/` |
| `CIS_5.1.7.sh` | `sudo IFS=$'\n'` runs IFS in a subshell that exits
immediately; searched `/System/Volumes/Data/Library` but the query looks
at `/Library/%` | IFS set in parent shell; searches `/Library`; skips
SIP-protected dirs via xattr |
| `CIS_5.7.sh` | Wrote `use-login-window-ui` which the query doesn't
accept | Writes `authenticate-session-owner` |
| `CIS_6.3.6.sh` | Contained literal `<username>` placeholders that were
never substituted | Iterates non-system users (`UniqueID >= 500`) and
runs `defaults write` as each |
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* macOS 14 (Sonoma) CIS policies updated to v3.0.0 with refreshed policy
names and CIS IDs.
* New CLI test runner to automate CIS validation against macOS VMs.
* **Bug Fixes / Improvements**
* Updated remediations and audit/query logic; safer, atomic config
updates; several policies revised or removed.
* **Tests**
* Many new and improved pass/fail helper scripts for validating CIS
checks and profiles.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Extra colon in the "Products:" section auto-generated file:
<img width="228" height="59" alt="Screenshot 2026-04-20 at 3 07 26 PM"
src="https://github.com/user-attachments/assets/687be6ea-71ae-45c7-a1e9-641994ee86ba"
/>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Corrected formatting in product list display by removing redundant
punctuation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Golang 1.26.2 has been released. It fixes some CVEs:
https://github.com/golang/go/issues?q=milestone%3AGo1.26.2+label%3ACherryPickApproved
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Updated Go toolchain to 1.26.2 across the repository and build
configs.
* Updated Docker build images to use Go 1.26.2.
* Expanded the set of tracked modules for the Go version update so
additional module files are included in automated updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This is a way to test osquery PRs as part of local fleetd TUF builds.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Chores**
* Enhanced macOS build process to support creating application bundles
from pull request workflow artifacts in addition to released versions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Released patch version v4.83.2 with updated Helm chart and application
metadata.
* Updated deployment configurations to use the latest container image
version across cloud providers.
* Updated published package version to v4.83.2.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
**Related issue:**
Ref #34797
Ref #42675
## Problem
When a software installer spec has no `hash_sha256`, Fleet re-downloads
the package, re-extracts metadata, and re-upserts the DB on every GitOps
run, even if the upstream file hasn't changed. For deployments with 50+
URL-only packages across multiple teams, this wastes bandwidth and
processing time on every run.
## Solution
By default, use etags to avoid unnecessary downloads:
1. First run: Fleet downloads the package normally and stores the
server's ETag header
2. Subsequent runs: Fleet sends a conditional GET with `If-None-Match`.
If the server returns 304 Not Modified, Fleet skips the download,
metadata extraction, S3 upload, and DB upsert entirely
Opt-out with `always_download:true`, meaning packages continue to be
downloaded and re-processed on every run, same as today. No UI changes
needed.
```yaml
url: https://nvidia.gpcloudservice.com/global-protect/getmsi.esp?version=64&platform=windows
always_download: true
install_script:
path: install.ps1
```
### Why conditional GET instead of HEAD
Fleet team [analysis of 276 maintained
apps](https://github.com/fleetdm/fleet/pull/42216#issuecomment-4105430061)
showed 7 apps where HEAD requests fail (405, 403, timeout) but GET works
for all. Conditional GET eliminates that failure class: if the server
doesn't support conditional requests, it returns 200 with the full body,
same as today.
### Why opt-in
5 of 276 apps (1.8%) have stale ETags (content changes but ETag stays
the same), caused by CDN caching artifacts (CloudFront, Cloudflare,
nginx inode-based ETags). The `cache` key lets users opt in per package
for URLs where they've verified ETag behavior is correct.
Validation rejects `always_download: true` when hash_sha256` is set
## Changes
- New YAML field: `cache` (bool, package-level)
- New migration: `http_etag` VARCHAR(512) column (explicit
`utf8mb4_unicode_ci` collation) + composite index `(global_or_team_id,
url(255))` on `software_installers`
- New datastore method: `GetInstallerByTeamAndURL`
- `downloadURLFn` accepts optional `If-None-Match` header, returns 304
as `(resp, nil, nil)` with `http.NoBody`
- ETag validated per RFC 7232 (ASCII printable only, no control chars,
max 512 bytes) at both write and read time
- Cache skipped for `.ipa` packages (multi-platform extraInstallers)
- TempFileReader and HTTP response leak prevention on download retry
- Docs updated in `yaml-files.md`
## What doesn't change
- Packages with `hash_sha256`: existing hash-based skip, untouched
- FMA packages: FMA version cache, untouched
- Packages with `always_download: true`: identical to current behavior
- Fleet UI: no changes
## Test plan
Automated testing:
- [x] 16 unit tests for `validETag`
- [x] 8 unit tests for conditional GET behavior (304, 200, 403, 500,
weak ETag, S3 multipart, no ETag)
- [x] MySQL integration test for `GetInstallerByTeamAndURL`
- [x] All 23 existing `TestSoftwareInstallers` datastore tests pass
- [x] All existing service tests pass
Manual testing:
- [x] E2E: 86 packages across 6 CDN patterns, second apply shows 51
conditional hits (304)
- [x] @sgress454 used a local fileserver tool to test w/ a new instance
and dummy packages
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* ETag-based conditional downloads to skip unchanged remote installer
files.
* New always_download flag to force full re-downloads.
* **Tests**
* Added integration and unit tests covering conditional GETs, ETag
validation, retries, edge cases, and payload behavior.
* **Chores**
* Persist HTTP ETag and related metadata; DB migration and index to
speed installer lookups.
* Added installer lookup by team+URL to support conditional download
flow.
* **Bug Fix**
* Rejects using always_download together with an explicit SHA256 in
uploads.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Scott Gress <scott@fleetdm.com>
Co-authored-by: Scott Gress <scott@pigandcow.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Released patch v4.83.1: updated chart and app metadata, container
image tags, Terraform deployment defaults, and npm package version to
v4.83.1.
* **Documentation**
* Updated CLI help/example to reference the v4.83.1 milestone.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42512
---------
Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42853
This PR simply adds the `require_all_software_windows` config option. It
doesn't use it. The logic to use it will be hooked up in subsequent PRs.
The fleetctl TestIntegrationsPreview test is expected to fail since it
builds the server against main and doesn't know about our new config
option.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- Not exported. generate-gitops does not export
require_all_software_windows (or require_all_software_macos either). The
generateControls function (generate_gitops.go) outputs a "TODO: update
with your setup_experience configuration" placeholder when any setup
experience config exists, rather than exporting individual field values.
This is a pre-existing limitation that applies equally to both fields -
not something introduced by our PR.
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- Yes. PR #42046 adds require_all_software_windows to both docs/REST
API/rest-api.md and docs/Configuration/yaml-files.md.
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- Yes, it gets cleared to false - both when setup_experience: is present
without the field, and when setup_experience: is omitted entirely. This
is the same behavior as the existing require_all_software_macos field
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
- Covered by #42854 (frontend subtask). The existing macOS checkbox in
InstallSoftwareForm.tsx:271 already checks gitOpsModeEnabled to disable
itself. The Windows checkbox to be added in #42854 will follow the same
pattern.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a Windows setup experience software requirement setting. When
enabled, Windows devices will cancel the Autopilot setup if any required
software installation fails.
* **Tests**
* Added test coverage for the new Windows software requirement
configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42691
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
n/a
## Testing
- [ ] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- I ran the updated snapshot action on this branch and verified that it
pushed the branch-tagged image, but not the SHA-tagged one.
- I ran the cleanup script in dry-run mode and verified that it didn't
expect to delete any non-sha-tagged images
- I wasn't able to test the delete-image-on-branch-delete action for
obvious reasons.
- I haven't tested the cleanup script in non-dry-run mode... I could do
on my personal dockerhub...
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Automated cleanup of Docker images when development branches are
deleted to maintain registry hygiene.
* New utility for managing and cleaning up legacy Docker image tags.
* **Chores**
* Enhanced Docker image tagging in snapshot builds with improved branch
name handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** The entire ACME feature branch merge
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Jordan Montgomery <elijah.jordan.montgomery@gmail.com>
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
