Fixes#31580
Fixes issues
- When updating a script to exactly match the content of another script,
we fail
- When updating one script which happens to match content of another
script, both get updated and not just the one being edited
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Resolved error when updating a script to exactly match another
script’s contents.
* Improved handling of script content updates: identical contents are
deduplicated and unused versions are cleaned up.
* Scheduled/pending runs are canceled on content updates with clearer
cancellation messaging.
* **Documentation**
* Added changelog entry describing the fix.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## For #31226
New features:
- Dynamic header for each possible state of a batch script run: Started,
Scheduled, and Finished (corresponds to tabs at
`/controls/scripts/progress`
- Unique tabs for each possible status of hosts targeted by a batch
script run: Ran, Errored, Pending, Incompatible, Canceled.
- Within each tab, sortable, paginated host results with output preview
and execution time.
- View script/run details, cancel a batch, view manage hosts page
filtered for the script batch run and a status.
- Global script batch runs activities and and Scripts progress rows now
navigate to this details page.
Cleanups and improvements:
- Expand tab count badge options using “alert”/“pending” variants across
hosts, policies, and query results.
- Misc cleanups and improvements
- [x] Changes file added for user-visible changes in `changes/`,
- [x] Updated automated tests - new tests tracked for follow-up work
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Fixes#32313
OpenTelemetry Tracing
- Added tracing to async task collectors: FlushHostsLastSeen,
collectHostsLastSeen, collectLabelQueryExecutions,
collectPolicyQueryExecutions, collectScheduledQueryStats
- Updated HTTP middleware to use OTEL semantic convention for span names
({method} {route})
- Added OTELEnabled() helper to FleetConfig
Optimizations
- Reduced OTEL batch size from 512 to 256 spans to prevent gRPC message
size errors
- Enabled gzip compression for trace exports
NOTE: I tried to improve OTEL instrumentation for cron jobs, but it got
too complicated due to goroutines in `schedule.go` so that effort should
be separate. We do have SQL instrumentation for cron jobs, but we are
missing root spans for cron jobs as a whole.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded OpenTelemetry tracing for async tasks (host last seen, label
membership, policy membership, scheduled query stats) to provide richer
observability.
* More descriptive HTTP span names using “METHOD /route” for clearer
trace analysis.
* **Bug Fixes**
* Improved OTLP gRPC exporter reliability by enabling gzip compression
and reducing export batch size, mitigating intermittent gRPC errors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32060
This PR adds:
- new default_team_config_json table
- caching of config from that table, including deep copy methods -- all
of this is not absolutely needed for this change since we are only using
`webhook_settings.failing_policies_webhook` here but added for
completeness/future
- teams/0 API updates
- GitOps updates
- generate gitops updates
Future PRs will add:
- ticket automation
- primo mode migration
- frontend changes
- documentation
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Configure failing-policy webhooks for “No team” via GitOps
(no-team.yml) and API, including enable/disable, destination URL, policy
IDs, and batch size; settings clear when omitted.
- GitOps and CLI now read/apply the real “No team” settings with dry-run
support.
- Policy automation evaluates hosts without a team and triggers “No
team” webhooks when applicable.
- GET/PATCH team 0 returns/accepts a minimal, webhook-focused config.
- Chores
- Added persistence and caching for the default “No team” configuration.
- Introduced a database table to store the default configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes#32393
httpsig-go library has encorporated the changes needed to support TPM,
so we are removing our local version of this library.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
Changes:
- Updated the `receive-redirect-from-microsoft` endpoint to redirect
users to their Fleet instance if an Entra admin did not consent to the
permissions requested by Fleet's compliance partner integration.
- Fixed a bug that prevented users from being redirected to their Fleet
instance if their Entra configuration is missing a required group
- Updated the `receive-redirect-from-microsoft` endpoint to require a
`state` input
Fixes#31432
- Added campaign target cleanup: Deletes targets from campaigns
completed >24h ago. Uses 10% or 50k min per run, processes in 10k
batches. Added DB index, integrated into hourly cron, includes tests.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Automatic cleanup of live query campaign targets 24 hours after
campaign completion to reduce clutter and storage usage.
- Chores
- Added a database index to speed up live query target operations for
improved performance at scale.
- Enhanced scheduled maintenance to log cleanup counts and execution
time for better observability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #32112
# Details
This PR reverts some earlier work using `runuser` and `runcon` intended
to allow Orbit to launch Fleet Desktop in the logged-in user's SELinux
context. This didn't work for out-of-the-box SELinux enforcement setups,
with a side-effect of each failed attempt to launch the desktop app
creating a new kernel keyring that doesn't get cleaned up until Orbit
quits (or GC runs, although I didn't see that happen in testing). While
using `runuser` has some possible benefits over using `sudo` to launch
processes, it also (when using `-l` to start a login shell) creates a
new keyring. This is not an issue if the command to launch the process
succeeds, but if it fails and retries over and over, we start amassing
keyrings.
It is the opinion of several other engineers (and now myself as well)
that the real solution to various Fleet Desktop launching issues is to
launch the desktop app as a user service using `systemctl --user` so
that it automatically inherits the user's environment, context, etc.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
# Testing
- [X] QA'd all new/changed functionality manually
- [ ] ~Ubuntu with SELinux on~ this is a very uncommon setup
- [x] Ubuntu with SELinux off
- [X] Fedora with SELinux on - double checked that `getenforce` returned
`Enforcing` this time
- [X] Fedora with SELinux off
- [ ] ~Debian with SELinux on~ this is a very uncommon setup
- [X] Debian with SELinux off
- [x] `runWithOutput` still works (tested with `go run
./tools/dialog/main.go --dialog=zenity` on all platforms)
- [ ] ~`runWithStdin` still works~ (this isn't currently used by Linux)
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [X] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [ ] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
## Database migrations
- [ ] Checked table schema to confirm autoupdate
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
## fleetd/orbit/Fleet Desktop
- [ ] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
for #31536
# Details
This PR adds a new API as specced in [the API
PR](9bf150580b/docs/REST%20API/rest-api.md (list-hosts-targeted-in-batch-script))
for scheduled scripts.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
ran a batch script on 100 hosts and ran the API in Postman for each
status, then canceled the batch and ran the API to check the canceled
status.
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Fixes: #31581
Undo changes to defFilter, they weren't necessary
Add unit test for team_id=0
# Checklist for submitter
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes#32347
# Checklist for submitter
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added health checks and elapsed-time logging during server startup and
host enrollment in the integration workflow.
- Bug Fixes
- Reduced flakiness by adding bounded login retries and server readiness
verification before proceeding.
- Tests
- Periodic diagnostics for host enrollment status to aid visibility
during runs.
- Chores
- Increased server startup timeout from 10 to 15 minutes in the
integration workflow.
- Minor workflow formatting cleanups for consistency.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #32231
# Details
This PR adjusts the queries for listing batch scripts slightly to count
_every_ row in `batch_activities` matching the filters, regardless of
whether any `batch_activity_host_results` rows exist for it. This
handles the edge case of a batch script where all the hosts have been
deleted.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [ ] Added/updated automated tests
I didn't add tests for this because these tests have already changed
quite a bit in https://github.com/fleetdm/fleet/pull/32174. I can add
tests in there when this merges.
- [X] QA'd all new/changed functionality manually
* Select a host in Manage Hosts, click Run Script, select a script and
do Run Now
* Delete that host
* Go to the batch scripts list (Controls -> Scripts -> Batch Progress)
* Verify that the batch script is still listed.
We don't have clear expectations for what numbers should be displayed
for the progress of a batch like this, but this PR at least ensures the
batch doesn't disappear.
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
- @noahtalerman: During #g-software design review on 2025-08-15, we
learned that Fleet builds and tests against Amazon Linux
---------
Co-authored-by: Rachael Shaw <r@rachael.wtf>
Related to: #32228
Changes:
- Updated the `receive-redirect-from-microsoft` endpoint to assign the
created compliance policy to an Entra ID group named "Fleet conditional
access"
handles the edge case mentioned in QA Notes #27689 and
- Use consistent *int64 for TeamID across all policy activity types
- Fix JSON omitempty issue causing team_id: 0 to be omitted(to fix a
small copy test bug in Frontend)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Current Output
<img width="639" height="166" alt="image"
src="https://github.com/user-attachments/assets/ee17b3ee-65c9-430f-ae3b-3c350f7e20f9"
/>
<img width="647" height="524" alt="image"
src="https://github.com/user-attachments/assets/9bc01988-ccca-4ea4-84bf-3a15f35ba428"
/>
---------
Co-authored-by: Scott Gress <scottmgress@gmail.com>
Closes#31581
Note:
- When no team id is provided it lists all installers, but they don't
have software_package fielded. I don't know if this is the intended
behavior or not.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
