Commit graph

3799 commits

Author SHA1 Message Date
Victor Lyuboslavsky
31f36a6314
Add "No Team" integration configurations for Jira and Zendesk (#32387)
- Added Jira and Zendesk integrations for "No team". (These are not
supported by GitOps for teams)

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

## New Fleet configuration settings

- [x] Setting(s) is/are explicitly excluded from GitOps


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
- Default (No Team) responses now include limited integrations (Jira,
Zendesk).
- You can configure or clear Jira/Zendesk integrations for the Default
(No Team) settings.

- Bug Fixes
- More consistent handling of the Default (No Team) when fetching team
details.
- Improved validation to prevent conflicting automation settings between
webhooks and integrations.

- Documentation
- Clarified that Jira/Zendesk integrations aren’t supported via GitOps
or at the team level (including No Team).
- Noted that certain options (e.g., Google Calendar, Conditional Access)
aren’t supported for the Default (No Team).

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-09-02 18:02:34 -05:00
Jonathan Katz
7c375c6e54
#30403 Fix fleet installed host count discrepancy (#32455)
Fixes: #30403 
Keys for deletedTitles map were generated differently, causing the same
software title to be marked removed even when a new version of the same
title was inserted.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.


## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:
2025-09-02 15:05:42 -04:00
Carlo
6879f879f4
Exclude 'pending' hosts from missing host counts (#32297)
Implements ABM pending host exclusion for #31359.
2025-09-02 08:21:25 -04:00
Lucas Manuel Rodriguez
4dd22e639e
Fix panic on some tools that use service.NewClient (#32462)
Fixes the following panic found by @AndreyKizimenko while doing
loadtesting using the `./tools/loadtest/scripts_and_profiles/` tool.

```
2025-08-29T15:57:56Z: Creating 7 teams... (press enter to proceed) 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x18 pc=0x1047fd598]

goroutine 1 [running]:
fmt.Fprint({0x0, 0x0}, {0x140001fb378, 0x1, 0x1})
	/Users/andrey/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.darwin-arm64/src/fmt/print.go:263 +0x48
github.com/fatih/color.(*Color).SetWriter(0x140005141a0, {0x0, 0x0})
	/Users/andrey/go/pkg/mod/github.com/fatih/color@v1.16.0/color.go:197 +0x88
github.com/fatih/color.(*Color).Fprintf(0x140005141a0, {0x0, 0x0}, {0x10515a87a, 0xf9}, {0x0, 0x0, 0x0})
	/Users/andrey/go/pkg/mod/github.com/fatih/color@v1.16.0/color.go:251 +0x50
github.com/fleetdm/fleet/v4/server/fleet.WriteExpiredLicenseBanner({0x0, 0x0})
	/Users/andrey/repositories/fleet/server/fleet/utils.go:17 +0x70
github.com/fleetdm/fleet/v4/server/service.(*Client).doContextWithBodyAndHeaders(0x140001fd810, {0x10599fde8, 0x1062ae040}, {0x1050ea452, 0x4}, {0x1051043a7?, 0x140000c61a0?}, {0x0, 0x0}, {0x140001aaf30, ...}, ...)
	/Users/andrey/repositories/fleet/server/service/client.go:134 +0x408
github.com/fleetdm/fleet/v4/server/service.(*Client).doContextWithHeaders(0x140001fd810?, {0x10599fde8?, 0x1062ae040?}, {0x1050ea452?, 0x4?}, {0x1051043a7?, 0x17?}, {0x0?, 0x0?}, {0x10580b160?, ...}, ...)
	/Users/andrey/repositories/fleet/server/service/client.go:156 +0x1a0
github.com/fleetdm/fleet/v4/server/service.(*Client).AuthenticatedDo(0x140001fd810, {0x1050ea452, 0x4}, {0x1051043a7, 0x17}, {0x0, 0x0}, {0x10580b160, 0x140001fd860})
	/Users/andrey/repositories/fleet/server/service/client.go:183 +0x204
github.com/fleetdm/fleet/v4/server/service.(*Client).authenticatedRequestWithQuery(0x140001fd810, {0x10580b160?, 0x140001fd860?}, {0x1050ea452, 0x4}, {0x1051043a7, 0x17}, {0x1057a8e80, 0x1400000ea20}, {0x0?, ...})
	/Users/andrey/repositories/fleet/server/service/client.go:263 +0x74
github.com/fleetdm/fleet/v4/server/service.(*Client).authenticatedRequest(...)
	/Users/andrey/repositories/fleet/server/service/client.go:273
github.com/fleetdm/fleet/v4/server/service.(*Client).CreateTeam(0x140001fd810, {0x140003584e0, 0x0, {0x0, 0x0, 0x0}, 0x0, 0x0, 0x0, 0x0})
	/Users/andrey/repositories/fleet/server/service/client_teams.go:30 +0xe0
main.main()
	/Users/andrey/repositories/fleet/tools/loadtest/scripts_and_profiles/main.go:74 +0x530
exit status 2
```
2025-09-01 06:50:11 -03:00
Rachael Shaw
7fa36bbaa1
Update calendar event text to match recent changes to "My device" page (#32334)
We changed "No" to "Fail" as part of the conditional access story.
2025-08-29 14:52:44 -05:00
Juan Fernandez
30fbb40377
Updating App config should use primary (#32428)
For #28713

Refactored the PATH fleet/config end-point to use the primary DB node
for both persisting changes and fetching modified App Config to avoid
stale UI due to read replica delay.
2025-08-29 14:02:59 -04:00
Victor Lyuboslavsky
cbea2bf12e
Fixed error when updating a script to exactly match the contents of another script. (#32438)
Fixes #31580 

Fixes issues
- When updating a script to exactly match the content of another script,
we fail
- When updating one script which happens to match content of another
script, both get updated and not just the one being edited

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Resolved error when updating a script to exactly match another
script’s contents.
* Improved handling of script content updates: identical contents are
deduplicated and unused versions are cleaned up.
* Scheduled/pending runs are canceled on content updates with clearer
cancellation messaging.

* **Documentation**
  * Added changelog entry describing the fix.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-29 12:38:37 -05:00
Victor Lyuboslavsky
8783bba7e0
Fixing flaky TestTurnOnLifecycleEventsApple (#32450)
Fixes #32448
2025-08-29 11:23:55 -05:00
Victor Lyuboslavsky
84e45f6fa1
OpenTelemetry minor improvements (#32324)
Fixes #32313

  OpenTelemetry Tracing

- Added tracing to async task collectors: FlushHostsLastSeen,
collectHostsLastSeen, collectLabelQueryExecutions,
collectPolicyQueryExecutions, collectScheduledQueryStats
- Updated HTTP middleware to use OTEL semantic convention for span names
({method} {route})
  - Added OTELEnabled() helper to FleetConfig

  Optimizations

- Reduced OTEL batch size from 512 to 256 spans to prevent gRPC message
size errors
  - Enabled gzip compression for trace exports

NOTE: I tried to improve OTEL instrumentation for cron jobs, but it got
too complicated due to goroutines in `schedule.go` so that effort should
be separate. We do have SQL instrumentation for cron jobs, but we are
missing root spans for cron jobs as a whole.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Expanded OpenTelemetry tracing for async tasks (host last seen, label
membership, policy membership, scheduled query stats) to provide richer
observability.
* More descriptive HTTP span names using “METHOD /route” for clearer
trace analysis.

* **Bug Fixes**
* Improved OTLP gRPC exporter reliability by enabling gzip compression
and reducing export batch size, mitigating intermittent gRPC errors.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-28 19:32:46 -05:00
Victor Lyuboslavsky
a23d24bf96
Allow configuring webhook policy automations for "No team" (#32129)
Fixes #32060 

This PR adds:
- new default_team_config_json table
- caching of config from that table, including deep copy methods -- all
of this is not absolutely needed for this change since we are only using
`webhook_settings.failing_policies_webhook` here but added for
completeness/future
- teams/0 API updates
- GitOps updates
- generate gitops updates

Future PRs will add:
- ticket automation
- primo mode migration
- frontend changes
- documentation

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked table schema to confirm autoupdate

## New Fleet configuration settings

- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
- Configure failing-policy webhooks for “No team” via GitOps
(no-team.yml) and API, including enable/disable, destination URL, policy
IDs, and batch size; settings clear when omitted.
- GitOps and CLI now read/apply the real “No team” settings with dry-run
support.
- Policy automation evaluates hosts without a team and triggers “No
team” webhooks when applicable.
  - GET/PATCH team 0 returns/accepts a minimal, webhook-focused config.

- Chores
- Added persistence and caching for the default “No team” configuration.
  - Introduced a database table to store the default configuration.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-08-28 16:38:27 -05:00
Victor Lyuboslavsky
b3216a1727
Add CleanupCompletedCampaignTargets to cleanup old campaign targets. (#32385)
Fixes #31432 

- Added campaign target cleanup: Deletes targets from campaigns
completed >24h ago. Uses 10% or 50k min per run, processes in 10k
batches. Added DB index, integrated into hourly cron, includes tests.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
- Automatic cleanup of live query campaign targets 24 hours after
campaign completion to reduce clutter and storage usage.

- Chores
- Added a database index to speed up live query target operations for
improved performance at scale.
- Enhanced scheduled maintenance to log cleanup counts and execution
time for better observability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-28 11:04:05 -05:00
Scott Gress
a87498421b
Add "batch script host results" API (#32174)
for #31536

# Details

This PR adds a new API as specced in [the API
PR](9bf150580b/docs/REST%20API/rest-api.md (list-hosts-targeted-in-batch-script))
for scheduled scripts.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually
ran a batch script on 100 hosts and ran the API in Postman for each
status, then canceled the batch and ran the API to check the canceled
status.

---------

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
2025-08-27 16:39:43 -05:00
Jonathan Katz
3cd68be3a5
Followup for #32284, packages_only works for team_id=0 (#32352)
Fixes: #31581 

Undo changes to defFilter, they weren't necessary
Add unit test for team_id=0

# Checklist for submitter


## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-08-27 16:40:48 -04:00
Scott Gress
c920007851
Update how deleted hosts show in batch scripts (#32290)
for #32231

# Details

This PR adjusts the queries for listing batch scripts slightly to count
_every_ row in `batch_activities` matching the filters, regardless of
whether any `batch_activity_host_results` rows exist for it. This
handles the edge case of a batch script where all the hosts have been
deleted.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [ ] Added/updated automated tests
I didn't add tests for this because these tests have already changed
quite a bit in https://github.com/fleetdm/fleet/pull/32174. I can add
tests in there when this merges.

- [X] QA'd all new/changed functionality manually

* Select a host in Manage Hosts, click Run Script, select a script and
do Run Now
* Delete that host
* Go to the batch scripts list (Controls -> Scripts -> Batch Progress)
* Verify that the batch script is still listed.

We don't have clear expectations for what numbers should be displayed
for the progress of a batch like this, but this PR at least ensures the
batch doesn't disappear.

For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results
2025-08-27 14:10:01 -05:00
Ian Littman
16311fbe26
Revise generated CPE for Docker Desktop for macOS to match more recent CVEs, make Docker CVE CPEs consistent (#32335) 2025-08-27 10:11:21 -06:00
Dhruv Trivedi
431decc3ce
fix: standardize team_id handling in policy activities (#32151)
handles the edge case mentioned in QA Notes #27689 and
- Use consistent *int64 for TeamID across all policy activity types 
- Fix JSON omitempty issue causing team_id: 0 to be omitted(to fix a
small copy test bug in Frontend)

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

Current Output
<img width="639" height="166" alt="image"
src="https://github.com/user-attachments/assets/ee17b3ee-65c9-430f-ae3b-3c350f7e20f9"
/>
<img width="647" height="524" alt="image"
src="https://github.com/user-attachments/assets/9bc01988-ccca-4ea4-84bf-3a15f35ba428"
/>

---------

Co-authored-by: Scott Gress <scottmgress@gmail.com>
2025-08-27 08:43:36 -05:00
Jonathan Katz
d3742e5227
31581 Fix packages_only flag to only show items with software_package (#32284)
Closes #31581

Note:
- When no team id is provided it lists all installers, but they don't
have software_package fielded. I don't know if this is the intended
behavior or not.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.


## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-08-26 21:53:46 -04:00
Juan Fernandez
56c0773ff3
Validate Require BitLocker PIN (#32240)
For #28133

Validate that if 'Require BitLocker PIN' is enabled, disk encryption
must be enabled as well.
2025-08-26 13:23:24 -04:00
Magnus Jensen
d56d656d05
Add full name IdP Fleet variable to Apple configuration profiles (#32246)
fixes: #30888 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked table schema to confirm autoupdate

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* New Features
* Added support for an IdP full name variable in Apple configuration
profiles, enabling substitution of a user’s full name.
* Profiles automatically re-deploy when a user’s IdP full name changes,
is removed, or the user is assigned to a host.
* Bug Fixes
* Added clearer failure handling: profiles that require an IdP full name
now report a delivery failure with a specific message if the value is
missing.
* Tests
  * Expanded test coverage for full name handling and failure scenarios.
* Chores
  * Seeded the new variable in the database.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-26 17:55:58 +02:00
Jordan Montgomery
e9f595a2e2
31167: SUSP api (#32163)
For #31167 

Adds API Changes for showing user-scoped profile scoeps and managed
local user accounts

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually
2025-08-26 11:31:06 -04:00
Jahziel Villasana-Espinoza
262ba193bc
filter out kernels with 0 hosts (#32292)
Closes https://github.com/fleetdm/fleet/issues/32257 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.



- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-08-25 20:55:57 -04:00
Victor Lyuboslavsky
a5a0387b9e
Downgrade "denylisted" error to warning (#32276)
Fixes #32274 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [x] QA'd all new/changed functionality manually

Used this query for QA. Got denylisted eventually.
```sql
SELECT * FROM time WHERE unix_time = unix_time AND sleep(300) = 0;
```
2025-08-25 13:45:36 -05:00
Carlo
8bc8d01f0a
Merge Android datastore into main Fleet datastore (#32233)
Resolves #31218
2025-08-25 11:41:28 -04:00
Jonathan Katz
87880ee28d
#31283 Fix false negative with manual mutation to powershell versions 7.5* (#32230)
Fixes: #31283 
Fixes: #32197 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually

## Test Plan

I manually tested with these powershell versions
Note more CVEs show up than before
CVEs [40733](https://nvd.nist.gov/vuln/detail/CVE-2022-40733) and
[40732](https://nvd.nist.gov/vuln/detail/CVE-2022-40732) are weird but
are expected to show up with this fix.

<google-sheets-html-origin><style type="text/css"><!--td {border: 1px
solid #cccccc;}br {mso-data-placement:same-cell;}--></style>
OS | Name | Version | expected CVE
-- | -- | -- | --
macos | PowerShell | 7.5.0-rc.1 | 30399, 21171
macos | PowerShell | 7.5.0 | 30399
windows | PowerShell 7-preview-x86 | 7.5.0.101 | 30399, 21171, 40733,
40732
windows | PowerShell 7-x64 | 7.5.0.0 | 30399, 40733, 40732
windows | PowerShell 7.5.2.0-x64 | 7.5.2.0 | 40733, 40732
ubuntu | powershell | 7.5.0-1.deb | -
ubuntu | powershell-preview | 7.5.0-rc.1-1.deb | -
2025-08-22 17:41:54 -04:00
Juan Fernandez
b2d84ebd61
Fixed issue due to UUID mistmatch on BitLocker CSP cmd. (#32210)
For #28133.

When generating the payload for the BitLocker CSP used for setting the
TPM PIN policy, make sure the UUID used in the Atomic enclousure matches
the UUID used for the whole command.
2025-08-22 14:32:23 -04:00
Victor Lyuboslavsky
4129b52fc6
Prevent deadlocks by adding FOR UPDATE locks (#32173)
Fixes #31173 

Reproduced and fixed in loadtest environment. Uncovered another source
of deadlocks, filed as a separate:
https://github.com/fleetdm/fleet/issues/32201
- Also, still seeing some deadlocks (a lot fewer) in DB, and they are
hidden from the API results by retries. They may still be happening
because locks happen row by row and not all at once. A potential fix
would be to lock the whole policy_membership table.

Additional frontend fix, which is needed to prevent potential timeouts:
https://github.com/fleetdm/fleet/pull/32212

Backend + frontend fix should be a sufficient fix for this issue
(ignoring the issue with the long software transaction).

Also, this PR contains some refactoring to split out the 1-host use
case.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* Bug Fixes
* Resolved rare deadlocks during concurrent policy updates and bulk
automations.
* Correctly clears stale MDM data and actions on host re-enrollment and
platform changes.
* Performance Improvements
* Optimized policy issue recalculation with per-host updates to reduce
contention.
* Improved concurrency handling for bulk policy updates to avoid lock
contention.
* Reliability
* More robust host enrollment: updates seen time, display name, and
label membership consistently.
* Ensures accurate policy-issue counts after membership changes and
re-enrollment.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-22 12:36:03 -05:00
Carlo
8212180819
Extract Android disk storage data (#32133)
Implements Android storage data extraction for issue #27080.
2025-08-22 12:27:15 -04:00
Juan Fernandez
9ec63e37a2
Consider team settings when getting detail queries (#32141)
For #28133.

This fixes a bug that prevented TPM PIN detail queries to be included if
the required settings are only set at the Team level.
2025-08-22 11:14:32 -04:00
Victor Lyuboslavsky
1d7aab04ab
Fix GitOps dry run issue with validating profiles with secrets (#32104)
Fixes #31477 

Docs PR: https://github.com/fleetdm/fleet/pull/32116

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
- GitOps now supports FLEET_SECRET_ placeholders in macOS
(.mobileconfig/.xml) profiles. Secrets are expanded only for validation,
while remaining unexpanded in uploaded content.
- Improved environment variable handling: non-secret vars expand as
before; server-side secrets are preserved.
- Validation enforces that profile display names cannot contain
FLEET_SECRET_ values.

- Bug Fixes
- Resolves validation issues when FLEET_SECRET_ appears in <data> tags
by performing safe client-side expansion for validation.
  - More accurate error reporting during profile parsing and validation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
2025-08-22 09:37:12 -05:00
Lucas Manuel Rodriguez
53b7a0628a
Prevent IT admins from deleting a secret variable in use (#32161)
#31977

Tested with:
224 scripts, 102 Apple configuration profiles, 105 Apple declarations,
37 Windows configuration profiles. Scattered in 7 teams.

No variables on profiles/scripts:
- scan scripts: 22.079416ms
- scan apple profiles: 1.865083ms
- scan apple declarations: 683.75µs
- scan windows profiles: 531.167µs

A variable "X" on with profiles/scripts that have variable "Y":
- scan scripts: 26.305459ms
- scan apple profiles: 1.841708ms
- scan apple declarations: 756.917µs
- scan windows profiles: 571.875µs

---

## Testing

- [X] Added/updated automated tests

- [X] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [X] Alerted the release DRI if additional load testing is needed

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
- Prevents deleting a secret that’s still in use. The system now checks
scripts, Apple/Windows MDM profiles, and Apple declarations across teams
before allowing deletion.
- The delete dialog now shows a clear message indicating which item (and
team) is using the secret, replacing the previous generic error.
- Deletion succeeds once all references are removed, ensuring safer
secret management.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-22 11:22:37 -03:00
Magnus Jensen
90b46dfbe7
Disable failing test case for TestTranslateCPEToCVE (#32193)
Recently seeing this fail constantly, and got the following comment:
https://github.com/fleetdm/fleet/issues/30118#issuecomment-3213352974
2025-08-22 14:32:58 +02:00
Anthony Maxwell
41fd8409e9
Feat: Perform S3 Uploads with Upload Manager (#32010)
# Overview

This PR implements the S3 upload manager under-the-hood of our
`datastore/s3` client's `Put()` method.

# Description

As surfaced by #31667, the current S3 implementation utilizes the `PUT`
operation which means services, such as MinIO, take issue with attempted
uploads that are [too
large](https://github.com/minio/minio/blob/master/cmd/streaming-signature-v4.go#L260).
The `PUT` operation can also present challenges in memory-constrained
environments as the entire upload target is read into memory before it's
shipped.

# Notes

- See the `TODO` comment section, there's more cool stuff we can and
should do with this in the future!

# Standard Pull Request Details

## Testing

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results

---------

Signed-off-by: Illbjorn <am@hades.so>
2025-08-20 15:20:53 -04:00
Dante Catalfamo
32fb86c285
Don't flood the terminal with binary output when downloading pkg (#32081)
#31736
2025-08-20 12:16:53 -04:00
Juan Fernandez
51a5b6166a
Refactor failing policies total on Host endpoint (#31906)
For #29795

Refactored the way we compute the number of failing policies to avoid
discrepancies due to either read replica delays or due to async nature
of the failing policy computation stored in host issues.
2025-08-19 13:39:32 -04:00
Jonathan Katz
3388740f0e
4498 empty software (#31940)
Fixes: #4498 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- Added unit test 
- Changed existing unit tests to accept empty array instead of null

- [x] QA'd all new/changed functionality manually
- Tested that Fleet UI > host details, returns `software: []` instead of
nothing.
- Tested that with exclude_software=false, software returns the full
array for host.

---------

Co-authored-by: Anthony Maxwell <133805840+Illbjorn@users.noreply.github.com>
2025-08-19 10:38:53 -04:00
Ian Littman
c461e097a8
Don't pass the default deb auto-install policy if install status is e.g. uninstalled (#32005)
Fixes #29894 and probably #31980.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-08-18 17:37:06 -05:00
Magnus Jensen
9a859736c2
IdP Authentication before BYOD (#32017)
fixes: #29222 

This is a feature branch that was completed last week, but did not get
merged in time.

All pr's going in was approved, and reviewed.

I will after this is merged, do a cherry pick onto the RC 4.73 branch,
and initiate the FR merge process.

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
2025-08-18 18:31:53 +02:00
Luke Heath
69388c69a7
Remove five character requirement for entity id in SSO and IdP settings (#30712) 2025-08-15 16:34:30 -05:00
Konstantin Sykulev
9a821efe8d
When iterating through softwares LastOpenedAt timestamp is copied as to not modify original records (#31946)
fixes #31932

The problem here was that `hostInstalledSoftware` returns a slice of
pointers (`[]*hostSoftware`), so when iterating through and assigning
`LastOpenedAt` the original records were getting modified. This code
duplicates the records being put into `bySoftwareTitleID` so that the
records being stored in `bySoftwareID` are the original records.

As a side benefit to this I modified the logic to store the most recent
`LastOpenedAt` for the software title. I think we may be doing something
similar to this on the front end to show the "last used" column when we
have multiple version of a software installed. But this can potentially
be fetched from the API now.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
2025-08-15 09:44:01 -05:00
Dante Catalfamo
ede1546108
Add activity for cancel batch script (#31937)
#31532 

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
2025-08-14 21:35:19 -05:00
Lucas Manuel Rodriguez
58233817f0
Add backend APIs for adding, deleting and listing secret variables (#31936)
For #31055.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [X] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-08-14 19:33:47 -03:00
Jahziel Villasana-Espinoza
56284ef87b
make sure vulns are shown in list OS page, and fix the names (#31930)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.


## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-08-14 16:47:58 -04:00
Carlo
d09e631b5c
Fix Android MDM missing private key validation #31353 (#31863)
Fixes #31353. Adds private key validation to Android MDM enterprise signup to prevent failed Android enablement when server private key is not configured.
2025-08-14 15:47:23 -04:00
Dante Catalfamo
259bcf6afd
Batch script cron schedule (#31808)
#31521
2025-08-14 14:44:47 -04:00
Scott Gress
443a55111f
Add "incompatible with script" filter for hosts (#31868)
for #31282

# Details 

This PR adds the ability to filter hosts by the "incompatible with batch
script" status. These hosts were previously included in the "Error"
state for a batch script when viewing the script summary.

The current script summary modal doesn't include a row for incompatible
(this modal will be replaced in the next iteration of the batch script
scheduling feature). To see the filter at work, you can either use the
API directly, or:

1. View the summary modal for a batch script by clicking on its activity
item in the global feed
2. Click on the number in any row (e.g. "Error" or "Pending")
3. Change the dropdown beneath the team selector to "Incompatible"
<img width="472" height="339" alt="image"
src="https://github.com/user-attachments/assets/04c6bc05-fe88-4be3-91ca-8b7162e1c6f3"
/>

Also renamed `cancelled` to `canceled` in a couple places to make the
spelling consistent.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually
2025-08-14 11:55:19 -05:00
Jahziel Villasana-Espinoza
153f73c8ca
30738 linux vulns (#31893)
- **linux vulns API changes (#31490)**
- **31214 linux vulns optimization (#31722)**

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
2025-08-14 10:13:37 -04:00
Pascal Matthiesen
6081da7673
feat: allow different cisa url to be provided (#31728)
Summary

• Allow custom CISA vulnerability data source URL to work around blocked
requests
  • Updates vulnerability sync logic to use configurable CISA endpoint
• Enables organizations to use CISA mirrors when direct access is
blocked

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-08-13 13:35:45 -05:00
Victor Lyuboslavsky
a78e1b24ca
Labels optimization recommended in code review. (#31856)
Fixes #29596
2025-08-13 17:42:33 +02:00
Juan Fernandez
684b0555d3
Refactoring suggested in #31634 (#31839)
For #30384

Minor refactorings suggested in
https://github.com/fleetdm/fleet/pull/31634.

No new changes introduced, so change file not required.
2025-08-13 10:24:32 -04:00
Ian Littman
da9bac09eb
Add support for install/uninstall script overrides, pre-install query, post-install script in FMA GitOps (#31803)
Also removed the automatic install flag on YAML FMAs as it's
undocumented/unspec'd

Fixes #25636.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-08-13 07:48:36 -05:00