fixes: #32164
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Speculative fix for #30915
For why this is needed, see
https://github.com/fleetdm/fleet/issues/30915#issuecomment-3259641371
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved reliability of Apple device profile installation and removal
by performing coordinated, read-only transactional reads. Reduces race
conditions and intermittent discrepancies during profile syncs, leading
to more consistent outcomes across fleets.
* **Tests**
* Added tests to verify the combined install/remove results remain
consistent with the individual lists, ensuring accurate and stable
behavior under various state changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32280
- Removed osqueryd.tar.gz from macOS package and desktop.tar.gz from
macOS and Linux packages and replaced them with .sha512 hash caches.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Eliminated duplicate osqueryd and Fleet Desktop binaries in Linux and
macOS packages, preventing duplicate entries in .deb/.pkg and ensuring
cleaner installs.
* **Chores**
* Added packaging cleanup to remove leftover tar.gz artifacts, reducing
package size and avoiding accidental inclusion in builds.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Adds support for reading server `private_key` from AWS Secrets Manager.
Combined with #31075, this should allow removing all common sensitive
secrets from the environment/config (if I missed any let me know). This
works with localstack for local development (set
`AWS_ENDPOINT_URL=$LOCALSTACK_URL`, `AWS_ACCESS_KEY_ID=test`, and
`AWS_SECRET_ACCESS_KEY=test`).
I did not include config options for `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` because they are a bad practice vs role
credentials and defeat the purpose of this feature which is to remove
secrets from the environment/config.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Scott Gress <scott@fleetdm.com>
Fixes: #32067
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes#29909
- Do not update DB if rules haven't changed
- Cache Yara rules when retrieved by hosts. This should reduce DB
accesses with large number of hosts retrieving large numbers of rules
I manually QA'd using OpenTelemetry (APM would also work) and monitoring
the DB accesses when updating or retrieving yara rules.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance Improvements**
* Faster config saves when many YARA rules are present (incremental
updates, reduced work).
* Lower latency and load when many hosts fetch YARA rules (caching and
smarter retrieval).
* More efficient handling of unchanged, added, modified, and removed
YARA rules.
* **Documentation**
* Changelog entry noting YARA rules performance and fetch improvements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## PR 2/2 for #32037
- Implements update for the Linux setup experience from the end-user's
point of view (the "My device" page).
- Works in concert with the new endpoints implemented in
https://github.com/fleetdm/fleet/pull/32493
- My device page calls a new endpoint to get in-progress setup
experience software installations. If there are any, the page is
replaced with a "Setting up your device" page
- The UI polls this endpoint until all such installations are either
successful or failed (including canceled)
- Setting up your device page includes a table displaying the name and
status of each software installation
- Once all installations are finished (succeed/fail), renders the
regular My device page
- Add a handler for the new API call for relevant tests
## Testing
Can use [this branch with fake
data](https://github.com/fleetdm/fleet/tree/32037-end-user-fake-data) to
help test this PR
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated automated tests - additional tests coming in
follow-up
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
resolves#31166
This adds a user icon and tooltip if the custom profile is user scoped
<img width="236" height="114" alt="image"
src="https://github.com/user-attachments/assets/be8e241d-6f70-40dc-808d-625245eb771f"
/>
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] QA'd all new/changed functionality manually
For #30849.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [n/a] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
for #31202
# Details
This PR updates the filename generation code in `generate-gitops` to be
more permissive. It will still replace spaces with dashes, but otherwise
all characters (including emojis).
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- generate-gitops now preserves emojis and other special characters in
generated filenames and outputs.
- Team names with emojis are fully supported, including corresponding
resource paths.
- Bug Fixes
- Replaced escaped Unicode sequences in YAML output with actual
characters, improving readability and parity with source data.
- Tests
- Updated test data and scenarios to include emoji-containing team names
and paths to ensure coverage for special character handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32295
The issue was identified/fixed using this performance test:
https://gist.github.com/getvictor/b289b7b14981fb7bf77e57c80af117d1
With the fix:
- 100 records: 2.6ms (similar)
- 1,000 records: ~7ms (32x faster)
- 5,000 records: ~10ms (530x faster)
- 10,000 records: 13ms (1,430x faster)
- 20,000 records: ~25ms (2,960x faster)
- 40,000 records: 50ms (6,000x faster)
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Script details now surface the most relevant “latest” status per
script, prioritizing upcoming executions when present.
- Performance Improvements
- Significantly faster loading of host script results and script
details, especially at large scale.
- Improved responsiveness when filtering/sorting script results.
- Documentation
- Added changelog entry describing the optimization to script results
handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
There are still some TODOs particularly within Gitops test code which
will be worked on in a followup PR
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [x] Alerted the release DRI if additional load testing is needed
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
---------
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
For #32040.
---
Backend changes to unblock the development of the orbit and frontend
changes.
New GET and PUT APIs for setting/getting software for Linux Setup
Experience:
```
curl -k -X GET -H "Authorization: Bearer $TEST_TOKEN" https://localhost:8080/api/latest/fleet/setup_experience/linux/software?team_id=8&per_page=3000
curl -k -X PUT -H "Authorization: Bearer $TEST_TOKEN" https://localhost:8080/api/latest/fleet/setup_experience/linux/software -d '{"team_id":8,"software_title_ids":[3000, 3001, 3007]}'
```
New setup_experience/init API called by orbit to trigger the Linux setup
experience on the device:
```
curl -v -k -X POST -H "Content-Type: application/json" "https://localhost:8080/api/fleet/orbit/setup_experience/init" -d '{"orbit_node_key": "ynYEtFsvv9xZ7rX619UE8of1I28H+GCj"}'
```
Get status API to call on "My device":
```
curl -v -k -X POST "https://localhost:8080/api/latest/fleet/device/7d940b6e-130a-493b-b58a-2b6e9f9f8bfc/setup_experience/status"
```
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added Linux support for Setup Experience alongside macOS.
- Introduced platform-specific admin APIs to configure and retrieve
Setup Experience software (macOS/Linux).
- Added device API to report Setup Experience status and an Orbit API to
initialize Setup Experience on non-macOS devices.
- Setup Experience now gates policy queries on Linux until setup is
complete.
- New activity log entry when Setup Experience software is edited
(includes platform and team).
- Documentation
- Updated audit logs reference to include the new “edited setup
experience software” event.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32160
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Dedicated automations for “No team,” including failing policy webhooks
and Jira/Zendesk ticketing, separate from global settings.
- Policies without a team now use the default team automation
configuration instead of global.
- Chores
- Database migration splits global vs “No team” policy IDs and copies
applicable automation settings to the default team.
- To enable this behavior, set FLEET_PARTNERSHIPS_ENABLE_PRIMO=1 before
running migrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #1817
# Details
This PR gives Fleet servers the ability to connect to RDS MySQL and
Elasticache Redis via AWS [Identity and Access Management
(IAM)](https://aws.amazon.com/iam/). It is based almost entirely on the
work of @titanous, branched from his [original pull
request](https://github.com/fleetdm/fleet/pull/31075). The main
differences between his branch and this are:
1. Removal of auto-detection of AWS region (and cache name for
Elasticache) in favor of specifying these values in configuration. The
auto-detection is admittedly handy but parsing AWS host URLs is not
considered a best practice.
2. Relying on the existence of these new configs to determine whether or
not to connect via IAM. This sidesteps a thorny issue of whether to try
an IAM-based Elasticache connection when a password is not supplied,
since this is technically a valid setup.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually - besides using
@titanous's excellent test tool, I verified the following end-to-end:
- [X] regular (non RDS) MySQL connection
- [X] RDS MySQL connection using username/password
- [X] RDS MySQL connection using IAM (no role)
- [X] RDS MySQL connection using IAM (assuming role)
- [X] regular (non Elasticache) Redis connection
- [X] Elasticache Redis connection using username/password
- [X] Elasticache Redis connection using NO password (without IAM)
- [X] Elasticache Redis connection using IAM (no role)
- [X] Elasticache Redis connection using IAM (assuming role)
---------
Co-authored-by: Jonathan Rudenberg <jonathan@titanous.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes#32550
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Restores ticket integration options in Policies > Other workflows
modal for team policies, ensuring available integrations are correctly
shown.
* Improves permission handling for managing policies, providing
consistent access to add or delete policies across related workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #30095.
#32482 is additional cleanup. Merging this to unblock orchestration
Linux setup experience work. Code has already been reviewed prior to
merging into the feature branch.
---------
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
Co-authored-by: Anthony Maxwell <133805840+Illbjorn@users.noreply.github.com>
- Added Jira and Zendesk integrations for "No team". (These are not
supported by GitOps for teams)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [x] Setting(s) is/are explicitly excluded from GitOps
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Default (No Team) responses now include limited integrations (Jira,
Zendesk).
- You can configure or clear Jira/Zendesk integrations for the Default
(No Team) settings.
- Bug Fixes
- More consistent handling of the Default (No Team) when fetching team
details.
- Improved validation to prevent conflicting automation settings between
webhooks and integrations.
- Documentation
- Clarified that Jira/Zendesk integrations aren’t supported via GitOps
or at the team level (including No Team).
- Noted that certain options (e.g., Google Calendar, Conditional Access)
aren’t supported for the Default (No Team).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes: #30403
Keys for deletedTitles map were generated differently, causing the same
software title to be marked removed even when a new version of the same
title was inserted.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
For #28713
Refactored the PATH fleet/config end-point to use the primary DB node
for both persisting changes and fetching modified App Config to avoid
stale UI due to read replica delay.
Fixes#31580
Fixes issues
- When updating a script to exactly match the content of another script,
we fail
- When updating one script which happens to match content of another
script, both get updated and not just the one being edited
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Resolved error when updating a script to exactly match another
script’s contents.
* Improved handling of script content updates: identical contents are
deduplicated and unused versions are cleaned up.
* Scheduled/pending runs are canceled on content updates with clearer
cancellation messaging.
* **Documentation**
* Added changelog entry describing the fix.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## For #31226
New features:
- Dynamic header for each possible state of a batch script run: Started,
Scheduled, and Finished (corresponds to tabs at
`/controls/scripts/progress`
- Unique tabs for each possible status of hosts targeted by a batch
script run: Ran, Errored, Pending, Incompatible, Canceled.
- Within each tab, sortable, paginated host results with output preview
and execution time.
- View script/run details, cancel a batch, view manage hosts page
filtered for the script batch run and a status.
- Global script batch runs activities and and Scripts progress rows now
navigate to this details page.
Cleanups and improvements:
- Expand tab count badge options using “alert”/“pending” variants across
hosts, policies, and query results.
- Misc cleanups and improvements
- [x] Changes file added for user-visible changes in `changes/`,
- [x] Updated automated tests - new tests tracked for follow-up work
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Fixes#32313
OpenTelemetry Tracing
- Added tracing to async task collectors: FlushHostsLastSeen,
collectHostsLastSeen, collectLabelQueryExecutions,
collectPolicyQueryExecutions, collectScheduledQueryStats
- Updated HTTP middleware to use OTEL semantic convention for span names
({method} {route})
- Added OTELEnabled() helper to FleetConfig
Optimizations
- Reduced OTEL batch size from 512 to 256 spans to prevent gRPC message
size errors
- Enabled gzip compression for trace exports
NOTE: I tried to improve OTEL instrumentation for cron jobs, but it got
too complicated due to goroutines in `schedule.go` so that effort should
be separate. We do have SQL instrumentation for cron jobs, but we are
missing root spans for cron jobs as a whole.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded OpenTelemetry tracing for async tasks (host last seen, label
membership, policy membership, scheduled query stats) to provide richer
observability.
* More descriptive HTTP span names using “METHOD /route” for clearer
trace analysis.
* **Bug Fixes**
* Improved OTLP gRPC exporter reliability by enabling gzip compression
and reducing export batch size, mitigating intermittent gRPC errors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32060
This PR adds:
- new default_team_config_json table
- caching of config from that table, including deep copy methods -- all
of this is not absolutely needed for this change since we are only using
`webhook_settings.failing_policies_webhook` here but added for
completeness/future
- teams/0 API updates
- GitOps updates
- generate gitops updates
Future PRs will add:
- ticket automation
- primo mode migration
- frontend changes
- documentation
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Configure failing-policy webhooks for “No team” via GitOps
(no-team.yml) and API, including enable/disable, destination URL, policy
IDs, and batch size; settings clear when omitted.
- GitOps and CLI now read/apply the real “No team” settings with dry-run
support.
- Policy automation evaluates hosts without a team and triggers “No
team” webhooks when applicable.
- GET/PATCH team 0 returns/accepts a minimal, webhook-focused config.
- Chores
- Added persistence and caching for the default “No team” configuration.
- Introduced a database table to store the default configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes#31432
- Added campaign target cleanup: Deletes targets from campaigns
completed >24h ago. Uses 10% or 50k min per run, processes in 10k
batches. Added DB index, integrated into hourly cron, includes tests.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Automatic cleanup of live query campaign targets 24 hours after
campaign completion to reduce clutter and storage usage.
- Chores
- Added a database index to speed up live query target operations for
improved performance at scale.
- Enhanced scheduled maintenance to log cleanup counts and execution
time for better observability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #31536
# Details
This PR adds a new API as specced in [the API
PR](9bf150580b/docs/REST%20API/rest-api.md (list-hosts-targeted-in-batch-script))
for scheduled scripts.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
ran a batch script on 100 hosts and ran the API in Postman for each
status, then canceled the batch and ran the API to check the canceled
status.
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Closes#31581
Note:
- When no team id is provided it lists all installers, but they don't
have software_package fielded. I don't know if this is the intended
behavior or not.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
for #31876
# Details
This PR updates the validation for password requirements to be
consistent in all places, and to show more specific error messages when
the entered password does not meet the requirements.
Previously the `valid_password` helper just returned a boolean
indicating whether a password was valid. It now returns an object with
`isValid`, `error` and `error_code` so that different types of password
issues can be surfaced. This allows us to continue having a single
source of truth for password validation, while providing more helpful
error messages when a password doesn't meet our criteria.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
- [X] First time setup (adding the initial user)
- [X] Add user in settings -> manage users
- [X] Changer user password in settings -> manage users
- [X] Password reset form
