<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#44723
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Strengthened validation of sorting/order parameters across many list
and cursor-based endpoints — unsupported sort keys now return explicit
errors and prevent unsafe queries.
* Labels listing: label-list pagination query name changed; ordering by
host_count is rejected when host counts are disabled (validated at
request parsing).
* **Tests**
* Added/expanded tests covering allowed order keys, rejection of unknown
keys, and pagination behavior for multiple listing APIs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40459
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
Recording:
https://drive.google.com/file/d/1_XqLyy-oY-WnIa97R4t9HihiBq3Fui6n/view?usp=drive_link
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Wiping a host now cancels all upcoming and queued activities for that
host in a single, atomic operation to avoid intermediate activations.
* **Bug Fixes**
* Wipe response handling now distinguishes success vs failure and
reliably cancels queued activities; datastore errors during host lookup
or cancellation are surfaced.
* Device lock/erase flows consistently update and propagate datastore
errors.
* **Tests**
* Added integration and datastore tests validating wipe clears upcoming
activities across macOS, Windows, Linux, and mixed-host scenarios.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
**Related issue:** Resolves#42290
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
- [x] Timeouts are implemented and retries are limited to avoid infinite
loops
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Failed Windows MDM wipe attempts now create a tracked "Failed wipe"
activity showing the affected host and display name for visibility; UI
filter and activity feed now surface this type.
* **Bug Fixes**
* Improved detection and reporting of wipe result statuses so real
failures are reliably surfaced.
* Duplicate failure responses are suppressed to avoid repeated alerts.
* **Tests**
* Added tests validating wipe-failure activity creation and related
control flows.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Marko Lisica <83164494+marko-lisica@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#42799
When a macOS device acknowledges a lock command it can immediately send
a trailing Idle check-in. CleanAppleMDMLock now requires that unlock_ref
to be set at least 5 minutes ago before clearing the lock state,
preventing that trailing Idle to prematurely clearing the MDM lock
state.
This commit introduces support for Python (.py) scripts on macOS and
Linux, including validation for Python shebangs and updates to
documentation, UI, error messages, and backend validation logic. It also
updates tests and file upload handling to recognize and properly process
Python scripts alongside existing shell (.sh) and PowerShell (.ps1)
scripts.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves #
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [ ] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
## Database migrations
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
## fleetd/orbit/Fleet Desktop
- [ ] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
---------
Co-authored-by: Jordan Montgomery <elijah.jordan.montgomery@gmail.com>
Co-authored-by: melpike <79950145+melpike@users.noreply.github.com>
Co-authored-by: jkatz01 <yehonatankatz@gmail.com>
Co-authored-by: Jonathan Katz <44128041+jkatz01@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#36682
- Added similar "not found" handling than the one we have when calling
`getHostScriptExecutionResultDB` within the `GetHostLockWipeStatus`
function for consistency => now return a 404 instead of a 500 when no
rows are returned.
- Applied similar changes to
https://github.com/fleetdm/fleet/issues/33090 (PR:
https://github.com/fleetdm/fleet/pull/40009) => logging an error instead
of returning a 404, so that the Host details page does not crash.
The actual fix would be to remove the `host_mdm_actions` record that
contains the orphan `lock_ref` mentioned in the issue.
I'm not sure what the root cause is, but I think setting up some
monitoring on the logs I added should help determining that.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #40348
# Details
This PR updates a number of error message on the server to use `fleet`
and `report` instead of `team` or `query` where applicable.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
this is all internal, i don't think it warrants a changelog
## Testing
- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
I did not go trying to trigger all these errors. It's text changes.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40054
# Checklist for submitter
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- Already present in previous PR
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Chores**
* Migrated logging to a structured, context-aware backend for clearer,
richer diagnostics and consistent log formatting.
* Introduced broader context propagation and adjusted internal
interfaces to support the new logging approach (no end-user behavior
changes).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33090
All the calls to getHostScriptExecutionResultDB within
GetHostLockWipeStatus now log an error with the hostID and execID if no
rows are found. (As agreed with @lucasmrod instead of modifying the DB
as I proposed in the issue, at least for now).
## Testing
- [x] QA'd all new/changed functionality manually
- Set the unlock_reference to a dummy value for a host_mdm_action =>
reproduced 404.
- Restarted the server with the fix => host details page loaded.
https://github.com/user-attachments/assets/4742d2ab-2af1-4837-9b34-895ea7e4463chttps://github.com/user-attachments/assets/50a20523-6ad7-41f3-8546-c6f9bc72f0be
Related issue: #31916
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [x] Alerted the release DRI if additional load testing is needed
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed automation attempt tracking for policy automations to properly
reset attempt counters when policies, scripts, or software installers
are updated.
* Resolved handling of legacy policy automation attempts in the
database.
* **Tests**
* Added comprehensive integration tests for policy automation retry
behavior and state management.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#31916
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Script and software installer policy automations now automatically
retry up to three times on failure.
* Retry attempt counters automatically reset when policies transition
from failing to passing state.
* Enhanced attempt tracking for improved monitoring and troubleshooting
of policy automation executions.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33509
All changes were approved in PRs to this feature branch.
---------
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com>
Co-authored-by: George Karr <georgekarrv@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#36315
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
- @kc9wwh: No issues with building or enrolling devices. Got it behind
SSL cert and MDM working, but I would need a supervised device to test
further.
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Jordan Montgomery <elijah.jordan.montgomery@gmail.com>
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33416
It's been decided to ship the feature and in the guide mention the apple
bug, that we are currently tracking.
[Slack
🧵](https://fleetdm.slack.com/archives/C03C41L5YEL/p1760448150025089?thread_ts=1760433366.092499&cid=C03C41L5YEL)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* New Features
* Added Lost Mode support to lock iOS and iPadOS devices.
* Added ability to disable Lost Mode to unlock iOS/iPadOS devices.
* Improvements
* More consistent lock/unlock experience across macOS, iOS/iPadOS,
Windows, and Linux, with clearer status and activity updates.
* iOS/iPadOS now shows pending unlock status while Lost Mode disable is
in progress.
* Tests
* Added comprehensive end-to-end tests covering lock/unlock/wipe across
Apple, Windows, and Linux devices.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#32220
# Details
This PR fixes an issue where hosts whose running scripts were canceled
by Orbit (e.g. due to timing out) were reported as being still "pending"
on the batch script details view. This was due to our only counting runs
as errored if the error code was > 0, and ignoring negative error codes
(which is what Orbit uses for this case).
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
Changed a couple of places where we were using `1` for an error code to
`-1`
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [X] Alerted the release DRI if additional load testing is needed
for #32560
# Details
This PR updates the way that batch scripts are sorted in the "Batch
Progress" UI to
# Checklist for submitter
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
for #32711
# Details
This PR fixes an issue where batch scripts that are canceled after
starting may sometimes continue to be displayed in the "started" list
rather than moving to "finished".
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [X] Added/updated automated tests
Added new test which fails without the fix, and passes with it.
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
Fixes#32295
The issue was identified/fixed using this performance test:
https://gist.github.com/getvictor/b289b7b14981fb7bf77e57c80af117d1
With the fix:
- 100 records: 2.6ms (similar)
- 1,000 records: ~7ms (32x faster)
- 5,000 records: ~10ms (530x faster)
- 10,000 records: 13ms (1,430x faster)
- 20,000 records: ~25ms (2,960x faster)
- 40,000 records: 50ms (6,000x faster)
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Script details now surface the most relevant “latest” status per
script, prioritizing upcoming executions when present.
- Performance Improvements
- Significantly faster loading of host script results and script
details, especially at large scale.
- Improved responsiveness when filtering/sorting script results.
- Documentation
- Added changelog entry describing the optimization to script results
handling.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
# Details
Applying the patches from
https://github.com/fleetdm/fleet/pull/32516#discussion_r2316985807 and
https://github.com/fleetdm/fleet/pull/32563
onto `main`.
This fixes:
* Batches where all remaining hosts are deleted will be correctly marked
as "finished"
* Batches scheduled for the future, and then canceled, will have al
hosts marked as "canceled" rather than pending
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
- [X] Started a new batch script run with one host, deleted that host,
and triggered the batch_activity_completion_checker schedule, and
verified that the batch was moved to "finished".
- [X] Scheduled script
- Created a new batch script run scheduled for a future date
- Canceled that batch run
- Clicked on the batch run in the "finished" tab of the batch scripts
list
- Verified that the number of canceled hosts = the number of targeted
hosts for that batch, and all other numbers were 0.
- Verified that clicking the canceled hosts number navigated to the
correct list of canceled hosts
- [X] "Run now" script
- Created a new batch script run with "run now"
- Waited for at least one host to run.
- Canceled that batch
- Clicked on the batch run in the "finished" tab of the batch scripts
list
- Verified that the number of canceled hosts = the number that were
still pending when I canceled the script, and that the # of pending
hosts was 0
- Verified that clicking the canceled hosts number navigated to the
correct list of canceled hosts
- [X] Multiple batches with the same hosts don't bleed into each other
- Created another batch script with the same hosts, scheduled for the
future
- Verified that the "pending" host list is correct
Fixes#31580
Fixes issues
- When updating a script to exactly match the content of another script,
we fail
- When updating one script which happens to match content of another
script, both get updated and not just the one being edited
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Resolved error when updating a script to exactly match another
script’s contents.
* Improved handling of script content updates: identical contents are
deduplicated and unused versions are cleaned up.
* Scheduled/pending runs are canceled on content updates with clearer
cancellation messaging.
* **Documentation**
* Added changelog entry describing the fix.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #32231
# Details
This PR adjusts the queries for listing batch scripts slightly to count
_every_ row in `batch_activities` matching the filters, regardless of
whether any `batch_activity_host_results` rows exist for it. This
handles the edge case of a batch script where all the hosts have been
deleted.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [ ] Added/updated automated tests
I didn't add tests for this because these tests have already changed
quite a bit in https://github.com/fleetdm/fleet/pull/32174. I can add
tests in there when this merges.
- [X] QA'd all new/changed functionality manually
* Select a host in Manage Hosts, click Run Script, select a script and
do Run Now
* Delete that host
* Go to the batch scripts list (Controls -> Scripts -> Batch Progress)
* Verify that the batch script is still listed.
We don't have clear expectations for what numbers should be displayed
for the progress of a batch like this, but this PR at least ensures the
batch doesn't disappear.
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
#31532
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
for #31555
# Details
This PR adds a new cron schedule "batch_activity_completion_checker"
that runs every 5 minutes and checks whether any batch activities marked
as "started" have completed their runs. In general this is done by
determining whether the sum of the "ran", "incompatible", "errored" and
"canceled" hosts equals the number of "targeted" hosts for the activity.
How that is computed will vary by batch activity type (currently we just
have batch scripts).
When an activity is marked as finished, we cache the final tally of host
statuses (ran, incompatible, errored, canceled) on the record. This is
important so that future queries on activity records don't have to do
the expensive query to compute the host counts on activities where those
counts will never change.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
Started a new batch script run using the update run modal (see
https://github.com/fleetdm/fleet/pull/31604) and then triggered the new
job using `fleetctl trigger --name batch_activity_completion_checker`,
and verified that the `batch_activities` record status was `finished`
and the expected fields were populated.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced an automated process that regularly marks completed batch
activities, ensuring more accurate and up-to-date activity statuses.
* **Bug Fixes**
* Improved reliability in updating the status of batch activities when
all targeted hosts have finished their tasks.
* **Tests**
* Added comprehensive tests to verify correct marking of completed batch
activities.
* **Chores**
* Enhanced internal scheduling and datastore interfaces to support the
new completion-checking process.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
for #31623
for #31526
# Details
This PR adds two new endpoints:
* `GET /scripts/batch/:batch_execution_id` returns the status of a
single batch script execution
* `GET /scripts/batch` returns a paginated list of batch script
executions, filtered by team and status
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
Added new batch script runs in UI, used Postman to list them and get
details.
for #31505
# Checklist for submitter
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
no changed functionality
## Database migrations
- [X] Checked table schema to confirm autoupdate
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
collate for both these tables is previously set to `utf8mb4_0900_ai_ci`,
but they don't contain user-settable strings
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Expanded batch activity tracking with new fields for job ID, status,
and activity type.
* Enhanced database structure to support more flexible batch activity
and host result management.
* **Bug Fixes**
* Updated references throughout the application to use the new table
names for batch activities and host results, ensuring consistency.
* **Tests**
* Added and updated tests to validate the new batch activity schema and
migration behavior.
* **Chores**
* Applied database migration to rename and restructure relevant tables
for improved clarity and extensibility.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #28846.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
Tests (automated and manual) and contributor API docs will arrive in
another PR
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
for #29443
# Checklist for submitter
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
## Details
This PR adds two new `HostListOptions` for the `ListHosts` method:
* `BatchScriptExecutionIDFilter`: if set, only returns hosts that were
targeted by the specified batch script run
* `BatchScriptExecutionStatus`: if set, further filters hosts by the
status of their script execution in the specified batch
## Testing
I added new automated tests that check:
* Starting a batch script run on 5 hosts and calling `ListHosts` with
only `BatchScriptExecutionIDFilter` set returns all 5 targeted hosts
* Calling `ListHosts` with `BatchScriptExecutionStatus` set to
`pending`, `run`, `errored` or `cancelled` returns the expected set of
hosts
* Simulating various script run outcomes (success, failure, cancel) and
calling `ListHosts` with each status again returns the expected set of
hosts
I also tested manually by executing a batch script in the UI and calling
the API via Postman with each status.
# Checklist for submitter
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
## Details
This PR adds a new `GET /scripts/batch/summary/:batch_execution_id`
endpoint that returns a summary of the current state of a batch script
execution, including some basic info about the script being executed and
a breakdown of how hosts have responded. See
https://github.com/fleetdm/fleet/pull/29200 for API response.
For #26082.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #25144.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #22875.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
When a policy is deleted clean up any pending software installs or
scripts generated from the policy
https://github.com/fleetdm/fleet/issues/23886
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
