Resolves#32419.
I took a stab at it while fixing #32420.
Sorry, missed to record with audio:
- I test with the proxy being down (to simulate failure when deleting)
and that the delete modal is not closed.
- Spinner during the delete API request.
- Cancel button disabled during the delete API request/.
- Tenant ID is cleared after successful deletion.
https://github.com/user-attachments/assets/dbad0613-a8bd-455d-8741-83c626328437
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [X] QA'd all new/changed functionality manually
**Related issue:** Resolves#31968
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [ ] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#30691
# Details
This PR fixes an issue where the cursor in the SQL editor is positioned
incorrectly on ChromeOS and Windows.
Also fixed an incorrect style declaration that was preventing keywords
from being bolded.
Before:
<img width="364" height="103" alt="image"
src="https://github.com/user-attachments/assets/19d9632b-b57b-4a0f-b019-99a7c145da8d"
/>
After:
<img width="363" height="108" alt="image"
src="https://github.com/user-attachments/assets/cbe9aefe-1920-41e5-9212-eb0f8cb51e4b"
/>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] QA'd all new/changed functionality manually
Tested on Chrome, Firefox, Safari and Microsoft Edge (where appropriate)
on MacOS, Linux and Windows.
Note that the specified padding does not render on ChromeOS or Windows
(it doesn't in production right now either).
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:**
Resolves#29053Resolves#33298
For reference, the diffs for merging Konstantin's changes into my
original PR are here: https://github.com/fleetdm/fleet/pull/33390
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- Bug Fixes
- Fixed duplicate macOS software entries caused by users renaming apps,
ensuring accurate, consolidated inventory.
- Documentation
- Documented improved software ingestion performance by pre-inserting
data in smaller batches to reduce database lock times during host
check-ins.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fixes: #31390
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
for #32014
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- GitOps manual labels can now reference hosts by Fleet host ID in
addition to hostname, hardware serial, or UUID.
- GitOps YAML/JSON accepts integers for host IDs; numeric IDs are
handled seamlessly alongside strings.
- Validation
- Stronger input validation for label hosts: only strings or integers
are allowed.
- Clear error returned for invalid types (e.g., floats) in hosts lists.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Resolves#33147
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- Bug Fixes
- Resolved MySQL performance regressions from 4.73.0/4.73.1 affecting OS
versions and software titles views, improving load times and reducing
timeouts.
- Refactor
- Optimized OS vulnerabilities fetching by batching multiple OS versions
in a single request.
- Added a supporting database index to speed kernel-related
vulnerability queries.
- Tests
- Added comprehensive tests for multi-OS vulnerability retrieval, CVSS
enrichment, team-scoped data, and service endpoint behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes: #31845
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [x] Setting(s) is/are explicitly excluded from GitOps
Fixes: #31969
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
fixes: #29650
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually (Not possible for me
to get it into the same state, I just never get a response from my
device, but never the device token is inactive, I think that might be a
very long time process?, but verified that if the error comes back it
successfully turns off MDM)
Fixes#32331
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Expanded OpenTelemetry coverage to include scheduled jobs for improved
tracing of cron executions.
- Documentation
- Updated changelog to reflect expanded OpenTelemetry coverage.
- Refactor
- Updated scheduled job processing to use contextual tracing across
operations with improved span lifecycle handling.
- Tests
- Adjusted tests to pass context into scheduled job runs.
- Chores
- Removed obsolete commented debug logs in SCIM middleware.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
resolves#32356
This updates the host details and my device pages to show the users card
that will show the idp info on android devices
this also adds some tests for the various rendering states of the users
card component
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes: #31989
# Adding sw_edition to CPE generation and translation
This PR adds the ability to override sw_edition with cpe translations.
This adds a new column to cpe.sqlite that is generated daily.
Old versions of fleet will still work with the new cpe db and
translations.
Versions from this change forward will require the new cpe db for cpe
translations to work.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Backwards Compatibility
Testing with physical machines and for Firefox ESR fix
| Fleet version | cpe db | translations | vuln. soft. # | Firefox ESR
cpe | Firefox ESR vuln. # |
| ------- | ------ | ------------ | ------------- | ---------------- |
------------------- |
| Updated | old | old | 58 | `:*:macos:*:*` | 168 |
| Updated | new | new | 58 | `:esr:macos:*:*` | 92 |
| 4.71.1 | old | old | 58 | `:*:macos:*:*` | 168 |
| 4.71.1 | new | new | 58 | `:*:macos:*:*` | 168 |
Testing with osquery-perf hosts
| Fleet version | cpe db | translations | vuln. soft. # |
Vulnerabilities |
| ------- | ------ | ------------ | ------------- | --------------- |
| Updated | old | old | 156/161 | 3136 |
| Updated | new | new | 156/161 | 3136 |
| 4.71.1 | old | old | 156/161 | 3951 |
| 4.71.1 | new | new | 156/161 | 3951 |
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
Fixes#31974
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
Fixes#32580
- Added retry logic for software installs
- Added sending intermediate results to Fleet server
I QA'd this on Linux (see video below). For macOS and Windows, I QA'd it
by having the server force retries on normal (non-setup experience
software installs).
Demo video: https://youtu.be/dbu78G6bXf8
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Automatic retries for software installations, with exponential backoff
on transient/network errors.
- Intermediate failures are recorded without closing the original
request; subsequent attempts continue automatically.
- Activity feed entries are created for intermediate failures, including
install identifiers.
- Setup experience installs now retry automatically (up to 3 attempts).
- Tests
- Expanded test coverage for retry behavior, error classification, and
intermediate failure reporting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32331
Manually tested all paths. `/test` path removed in
https://github.com/fleetdm/fleet/pull/32962
Also added support for sending errors to OpenTelemetry, like we do for
APM/Sentry.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added OpenTelemetry tracing across core HTTP endpoints (health,
version, assets, metrics, enroll/root, debug, Apple MDM, SCEP, SCIM)
with dynamic per-request route instrumentation.
* Enhanced error reporting to include OpenTelemetry spans/events with
contextual user/host attributes.
* **Tests**
* Added unit tests validating SCIM and error-handling telemetry, span
naming, and sensitive-data redaction.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32996
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* Bug Fixes
* Corrected team filtering when listing MDM commands to ensure accurate
results. Team-scoped and global commands now display correctly for users
with appropriate access, resolving cases of missing or incorrect entries
when filtering by team.
* Tests
* Added comprehensive coverage for team-scoped MDM command listings,
role-based visibility (team users vs. admins), and hostname ordering to
prevent regressions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
fixes: #32121
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
for #32711
# Details
This PR fixes an issue where batch scripts that are canceled after
starting may sometimes continue to be displayed in the "started" list
rather than moving to "finished".
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [X] Added/updated automated tests
Added new test which fails without the fix, and passes with it.
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
fixes: #32164
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Speculative fix for #30915
For why this is needed, see
https://github.com/fleetdm/fleet/issues/30915#issuecomment-3259641371
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved reliability of Apple device profile installation and removal
by performing coordinated, read-only transactional reads. Reduces race
conditions and intermittent discrepancies during profile syncs, leading
to more consistent outcomes across fleets.
* **Tests**
* Added tests to verify the combined install/remove results remain
consistent with the individual lists, ensuring accurate and stable
behavior under various state changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32280
- Removed osqueryd.tar.gz from macOS package and desktop.tar.gz from
macOS and Linux packages and replaced them with .sha512 hash caches.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Eliminated duplicate osqueryd and Fleet Desktop binaries in Linux and
macOS packages, preventing duplicate entries in .deb/.pkg and ensuring
cleaner installs.
* **Chores**
* Added packaging cleanup to remove leftover tar.gz artifacts, reducing
package size and avoiding accidental inclusion in builds.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Adds support for reading server `private_key` from AWS Secrets Manager.
Combined with #31075, this should allow removing all common sensitive
secrets from the environment/config (if I missed any let me know). This
works with localstack for local development (set
`AWS_ENDPOINT_URL=$LOCALSTACK_URL`, `AWS_ACCESS_KEY_ID=test`, and
`AWS_SECRET_ACCESS_KEY=test`).
I did not include config options for `AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY` because they are a bad practice vs role
credentials and defeat the purpose of this feature which is to remove
secrets from the environment/config.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Scott Gress <scott@fleetdm.com>
Fixes: #32067
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes#29909
- Do not update DB if rules haven't changed
- Cache Yara rules when retrieved by hosts. This should reduce DB
accesses with large number of hosts retrieving large numbers of rules
I manually QA'd using OpenTelemetry (APM would also work) and monitoring
the DB accesses when updating or retrieving yara rules.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance Improvements**
* Faster config saves when many YARA rules are present (incremental
updates, reduced work).
* Lower latency and load when many hosts fetch YARA rules (caching and
smarter retrieval).
* More efficient handling of unchanged, added, modified, and removed
YARA rules.
* **Documentation**
* Changelog entry noting YARA rules performance and fetch improvements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## PR 2/2 for #32037
- Implements update for the Linux setup experience from the end-user's
point of view (the "My device" page).
- Works in concert with the new endpoints implemented in
https://github.com/fleetdm/fleet/pull/32493
- My device page calls a new endpoint to get in-progress setup
experience software installations. If there are any, the page is
replaced with a "Setting up your device" page
- The UI polls this endpoint until all such installations are either
successful or failed (including canceled)
- Setting up your device page includes a table displaying the name and
status of each software installation
- Once all installations are finished (succeed/fail), renders the
regular My device page
- Add a handler for the new API call for relevant tests
## Testing
Can use [this branch with fake
data](https://github.com/fleetdm/fleet/tree/32037-end-user-fake-data) to
help test this PR
- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated automated tests - additional tests coming in
follow-up
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
resolves#31166
This adds a user icon and tooltip if the custom profile is user scoped
<img width="236" height="114" alt="image"
src="https://github.com/user-attachments/assets/be8e241d-6f70-40dc-808d-625245eb771f"
/>
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] QA'd all new/changed functionality manually
For #30849.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [n/a] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
for #31202
# Details
This PR updates the filename generation code in `generate-gitops` to be
more permissive. It will still replace spaces with dashes, but otherwise
all characters (including emojis).
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- generate-gitops now preserves emojis and other special characters in
generated filenames and outputs.
- Team names with emojis are fully supported, including corresponding
resource paths.
- Bug Fixes
- Replaced escaped Unicode sequences in YAML output with actual
characters, improving readability and parity with source data.
- Tests
- Updated test data and scenarios to include emoji-containing team names
and paths to ensure coverage for special character handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32295
The issue was identified/fixed using this performance test:
https://gist.github.com/getvictor/b289b7b14981fb7bf77e57c80af117d1
With the fix:
- 100 records: 2.6ms (similar)
- 1,000 records: ~7ms (32x faster)
- 5,000 records: ~10ms (530x faster)
- 10,000 records: 13ms (1,430x faster)
- 20,000 records: ~25ms (2,960x faster)
- 40,000 records: 50ms (6,000x faster)
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Script details now surface the most relevant “latest” status per
script, prioritizing upcoming executions when present.
- Performance Improvements
- Significantly faster loading of host script results and script
details, especially at large scale.
- Improved responsiveness when filtering/sorting script results.
- Documentation
- Added changelog entry describing the optimization to script results
handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
There are still some TODOs particularly within Gitops test code which
will be worked on in a followup PR
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [x] Alerted the release DRI if additional load testing is needed
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
---------
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
For #32040.
---
Backend changes to unblock the development of the orbit and frontend
changes.
New GET and PUT APIs for setting/getting software for Linux Setup
Experience:
```
curl -k -X GET -H "Authorization: Bearer $TEST_TOKEN" https://localhost:8080/api/latest/fleet/setup_experience/linux/software?team_id=8&per_page=3000
curl -k -X PUT -H "Authorization: Bearer $TEST_TOKEN" https://localhost:8080/api/latest/fleet/setup_experience/linux/software -d '{"team_id":8,"software_title_ids":[3000, 3001, 3007]}'
```
New setup_experience/init API called by orbit to trigger the Linux setup
experience on the device:
```
curl -v -k -X POST -H "Content-Type: application/json" "https://localhost:8080/api/fleet/orbit/setup_experience/init" -d '{"orbit_node_key": "ynYEtFsvv9xZ7rX619UE8of1I28H+GCj"}'
```
Get status API to call on "My device":
```
curl -v -k -X POST "https://localhost:8080/api/latest/fleet/device/7d940b6e-130a-493b-b58a-2b6e9f9f8bfc/setup_experience/status"
```
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added Linux support for Setup Experience alongside macOS.
- Introduced platform-specific admin APIs to configure and retrieve
Setup Experience software (macOS/Linux).
- Added device API to report Setup Experience status and an Orbit API to
initialize Setup Experience on non-macOS devices.
- Setup Experience now gates policy queries on Linux until setup is
complete.
- New activity log entry when Setup Experience software is edited
(includes platform and team).
- Documentation
- Updated audit logs reference to include the new “edited setup
experience software” event.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32160
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Dedicated automations for “No team,” including failing policy webhooks
and Jira/Zendesk ticketing, separate from global settings.
- Policies without a team now use the default team automation
configuration instead of global.
- Chores
- Database migration splits global vs “No team” policy IDs and copies
applicable automation settings to the default team.
- To enable this behavior, set FLEET_PARTNERSHIPS_ENABLE_PRIMO=1 before
running migrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #1817
# Details
This PR gives Fleet servers the ability to connect to RDS MySQL and
Elasticache Redis via AWS [Identity and Access Management
(IAM)](https://aws.amazon.com/iam/). It is based almost entirely on the
work of @titanous, branched from his [original pull
request](https://github.com/fleetdm/fleet/pull/31075). The main
differences between his branch and this are:
1. Removal of auto-detection of AWS region (and cache name for
Elasticache) in favor of specifying these values in configuration. The
auto-detection is admittedly handy but parsing AWS host URLs is not
considered a best practice.
2. Relying on the existence of these new configs to determine whether or
not to connect via IAM. This sidesteps a thorny issue of whether to try
an IAM-based Elasticache connection when a password is not supplied,
since this is technically a valid setup.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually - besides using
@titanous's excellent test tool, I verified the following end-to-end:
- [X] regular (non RDS) MySQL connection
- [X] RDS MySQL connection using username/password
- [X] RDS MySQL connection using IAM (no role)
- [X] RDS MySQL connection using IAM (assuming role)
- [X] regular (non Elasticache) Redis connection
- [X] Elasticache Redis connection using username/password
- [X] Elasticache Redis connection using NO password (without IAM)
- [X] Elasticache Redis connection using IAM (no role)
- [X] Elasticache Redis connection using IAM (assuming role)
---------
Co-authored-by: Jonathan Rudenberg <jonathan@titanous.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes#32550
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Restores ticket integration options in Policies > Other workflows
modal for team policies, ensuring available integrations are correctly
shown.
* Improves permission handling for managing policies, providing
consistent access to add or delete policies across related workflows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #30095.
#32482 is additional cleanup. Merging this to unblock orchestration
Linux setup experience work. Code has already been reviewed prior to
merging into the feature branch.
---------
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
Co-authored-by: Anthony Maxwell <133805840+Illbjorn@users.noreply.github.com>
- Added Jira and Zendesk integrations for "No team". (These are not
supported by GitOps for teams)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [x] Setting(s) is/are explicitly excluded from GitOps
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Default (No Team) responses now include limited integrations (Jira,
Zendesk).
- You can configure or clear Jira/Zendesk integrations for the Default
(No Team) settings.
- Bug Fixes
- More consistent handling of the Default (No Team) when fetching team
details.
- Improved validation to prevent conflicting automation settings between
webhooks and integrations.
- Documentation
- Clarified that Jira/Zendesk integrations aren’t supported via GitOps
or at the team level (including No Team).
- Noted that certain options (e.g., Google Calendar, Conditional Access)
aren’t supported for the Default (No Team).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes: #30403
Keys for deletedTitles map were generated differently, causing the same
software title to be marked removed even when a new version of the same
title was inserted.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
For #28713
Refactored the PATH fleet/config end-point to use the primary DB node
for both persisting changes and fetching modified App Config to avoid
stale UI due to read replica delay.
Fixes#31580
Fixes issues
- When updating a script to exactly match the content of another script,
we fail
- When updating one script which happens to match content of another
script, both get updated and not just the one being edited
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Resolved error when updating a script to exactly match another
script’s contents.
* Improved handling of script content updates: identical contents are
deduplicated and unused versions are cleaned up.
* Scheduled/pending runs are canceled on content updates with clearer
cancellation messaging.
* **Documentation**
* Added changelog entry describing the fix.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
## For #31226
New features:
- Dynamic header for each possible state of a batch script run: Started,
Scheduled, and Finished (corresponds to tabs at
`/controls/scripts/progress`
- Unique tabs for each possible status of hosts targeted by a batch
script run: Ran, Errored, Pending, Incompatible, Canceled.
- Within each tab, sortable, paginated host results with output preview
and execution time.
- View script/run details, cancel a batch, view manage hosts page
filtered for the script batch run and a status.
- Global script batch runs activities and and Scripts progress rows now
navigate to this details page.
Cleanups and improvements:
- Expand tab count badge options using “alert”/“pending” variants across
hosts, policies, and query results.
- Misc cleanups and improvements
- [x] Changes file added for user-visible changes in `changes/`,
- [x] Updated automated tests - new tests tracked for follow-up work
- [x] QA'd all new/changed functionality manually
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
Fixes#32313
OpenTelemetry Tracing
- Added tracing to async task collectors: FlushHostsLastSeen,
collectHostsLastSeen, collectLabelQueryExecutions,
collectPolicyQueryExecutions, collectScheduledQueryStats
- Updated HTTP middleware to use OTEL semantic convention for span names
({method} {route})
- Added OTELEnabled() helper to FleetConfig
Optimizations
- Reduced OTEL batch size from 512 to 256 spans to prevent gRPC message
size errors
- Enabled gzip compression for trace exports
NOTE: I tried to improve OTEL instrumentation for cron jobs, but it got
too complicated due to goroutines in `schedule.go` so that effort should
be separate. We do have SQL instrumentation for cron jobs, but we are
missing root spans for cron jobs as a whole.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded OpenTelemetry tracing for async tasks (host last seen, label
membership, policy membership, scheduled query stats) to provide richer
observability.
* More descriptive HTTP span names using “METHOD /route” for clearer
trace analysis.
* **Bug Fixes**
* Improved OTLP gRPC exporter reliability by enabling gzip compression
and reducing export batch size, mitigating intermittent gRPC errors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32060
This PR adds:
- new default_team_config_json table
- caching of config from that table, including deep copy methods -- all
of this is not absolutely needed for this change since we are only using
`webhook_settings.failing_policies_webhook` here but added for
completeness/future
- teams/0 API updates
- GitOps updates
- generate gitops updates
Future PRs will add:
- ticket automation
- primo mode migration
- frontend changes
- documentation
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Configure failing-policy webhooks for “No team” via GitOps
(no-team.yml) and API, including enable/disable, destination URL, policy
IDs, and batch size; settings clear when omitted.
- GitOps and CLI now read/apply the real “No team” settings with dry-run
support.
- Policy automation evaluates hosts without a team and triggers “No
team” webhooks when applicable.
- GET/PATCH team 0 returns/accepts a minimal, webhook-focused config.
- Chores
- Added persistence and caching for the default “No team” configuration.
- Introduced a database table to store the default configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes#31432
- Added campaign target cleanup: Deletes targets from campaigns
completed >24h ago. Uses 10% or 50k min per run, processes in 10k
batches. Added DB index, integrated into hourly cron, includes tests.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Automatic cleanup of live query campaign targets 24 hours after
campaign completion to reduce clutter and storage usage.
- Chores
- Added a database index to speed up live query target operations for
improved performance at scale.
- Enhanced scheduled maintenance to log cleanup counts and execution
time for better observability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
