Commit graph

3986 commits

Author SHA1 Message Date
Magnus Jensen
f2341497c8
DCSW: Allow uploading and validating Windows SCEP profiles (#34691)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34247 

Allows for uploading and validating SCEP profiles for Windows.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.
## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-10-27 10:48:37 -03:00
Victor Lyuboslavsky
27b8d1aa4b
Fixed issue searching software versions (#34770)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34713

Fix for unreleased but. Needs to be cherry picked into 4.76.0

# Checklist for submitter

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-10-25 16:02:02 -05:00
jacobshandling
ab0065ab39
(releases on merge to main) Detect JetBrains IDE plugin vulnerabilities (#34331)
**Related issue:** Resolves #32266


[Demo](https://drive.google.com/file/d/1ZDYJkWkxZ519le8v9qGmcrL8YkP-uivW/view?usp=sharing)

- [x] Changes file added for user-visible changes in `changes/`,
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-10-24 12:06:35 -07:00
Tim Lee
c5d7c9f626
31970 NPM vuln support (#33100) 2025-10-24 12:54:57 -06:00
Jordan Montgomery
79b886455a
Update host expiry logic to account for Apple MDM checkin times (#34698)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #32499

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved host expiry logic to correctly identify and preserve Apple
MDM-enrolled hosts that don't check in through Orbit, preventing
unintended host deletions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-10-24 13:12:03 -04:00
Victor Lyuboslavsky
0db1b472a1
Okta conditional access configs (#34566)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34533

This is the first sub-task out of several. Changes file will be added in
a subsequent PR.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

## New Fleet configuration settings

- [x] Setting(s) is/are explicitly **excluded** from GitOps

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added Okta Conditional Access support (IDP, ACS URL, audience,
certificate) and exposed conditional access in AppConfig/API
  * App activity logging for adding/removing Okta conditional access

* **Bug Fixes**
  * Fixed typo in conditional access validation messaging

* **Tests**
* Added tests for Okta Conditional Access lifecycle, license gating, and
GitOps export exclusion

* **Documentation**
  * Added audit-log entries for Okta conditional access add/delete
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-10-24 10:11:14 -05:00
Magnus Jensen
ba15654498
DCSW: Support all IDP variables in Windows config profiles (#34707)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves
https://fleetdm.slack.com/archives/C03C41L5YEL/p1761232938314509, but
also https://github.com/fleetdm/fleet/issues/34363, as it would have
been stupid to only add IDP_USERNAME (to allow for an email) when they
are so close together and no extra effort to support the rest.

Also does another fix to ensure fleet variables are correctly ordered by
longest name to shortest, to ensure variables that are used in longer
format (USERNAME, USERNAME_LOCAL_PART) that the LOCAL_PART one gets
processed first.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.


## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-10-24 10:10:58 -03:00
Carlo
855ead6338
Fix script package unsupported fields in GitOps (#34703)
Fixes #34643. Fixes script packages (`.sh` and `.ps1`) incorrectly outputting unsupported fields in GitOps YAML and accepting invalid configuration during upload.
2025-10-23 17:45:08 -04:00
Carlo
821ed9d8b9
Add .sh packages to Linux setup experience (#34710)
Fixes #34654. Updates the SQL query to recognize `.sh` as a valid distribution-agnostic installer extension.
2025-10-23 17:20:47 -04:00
Jahziel Villasana-Espinoza
5c7416711f
enable app reports on default profile (#34693)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** No ticket, just a bug noticed while testing

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

## Testing

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-10-23 13:33:35 -04:00
Magnus Jensen
5105e1914b
Windows Var Replacement: use updated profile contents instead of incoming (#34681)
This fixes a bug, but also fixes the flaky test runs seen in
[Slack](https://fleetdm.slack.com/archives/C019WG4GH0A/p1761192670589249).
2025-10-23 10:04:24 -03:00
Magnus Jensen
f682f063bd
Check if IsMDMLostModeEnabled is present before taking value (#34653)
Resolves the dogfood down, issue.
https://fleetdm.slack.com/archives/C071NNMSP2R/p1761162242092529
2025-10-22 17:19:32 -03:00
Magnus Jensen
d6a23a79ee
Modify Windows replacement code to allow Custom SCEP variables (#34633)
and refactor to share with apple mdm

<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34246 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
2025-10-22 15:46:48 -03:00
Jordan Montgomery
7593d102fb
Experimental fleet server config for custom updates & disk encryption settings (#34598)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #33316

Merges in changes made in this community PR:
https://github.com/fleetdm/fleet/pull/33665

Adds support for Windows and tests, also blocks the feature on fleet
free

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

---------

Co-authored-by: Wesley Whetstone <wesw@stripe.com>
Co-authored-by: Wesley Whetstone <jckwhet@gmail.com>
2025-10-22 13:51:10 -04:00
Tim Lee
d4004a4f8e
IDP user update API (#34332) 2025-10-21 12:02:25 -06:00
Scott Gress
7fa8793b1e
Add macos web setup experience capability check (#34582)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34596 

# Details

This PR adds back the "legacy" macOS setup experience, and a new
`CapabilityMacOSWebSetupExperience` capability check. The legacy
experience will be shown whenever the
`CapabilityMacOSWebSetupExperience` capability is _not_ detected in the
Fleet server response.

The majority of the code is just copying the `Run` and
`startSwiftDialog` from the [4.75.1 patch
release](https://github.com/fleetdm/fleet/blob/rc-patch-fleet-v4.75.1/orbit/pkg/setup_experience/setup_experience.go)
and adding them back as `RunLegacy` and `startSwiftDialogLegacy`, then
adding the little bit of plumbing to have `Run()` call `RunLegacy()`
when the new capability is not detected.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

## Testing

- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually
Tested on a VM that the setup experience starts in "legacy" mode if
`CapabilityMacOSWebSetupExperience` is not set in the server, and that
it starts in the new web view if the capability _is_ set.


For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed

## fleetd/orbit/Fleet Desktop

- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
I have not re-verified this, but the changes will only run on macOS
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
This should not be affected.
2025-10-21 12:32:16 -05:00
Konstantin Sykulev
c42a732ef7
Unbatching checksum migration (#34586)
**Related issue:** https://github.com/fleetdm/fleet/issues/34559
PR for 4.75.1: https://github.com/fleetdm/fleet/pull/34485

needs to be added in main to be consistent.
2025-10-21 11:21:33 -05:00
Ian Littman
e995891359
Revise 4.76 migrations to land after all 4.75 migrations (#34479)
- [x] QA'd locally blank -> 4.75 -> this branch
2025-10-17 17:55:57 -05:00
Ian Littman
52410a3328
Add CPE translations to map various software where name/bundle ID based matching doesn't work (#34233)
Fixes #34282. Must be QA'd on-branch so split from the software name
cleanup PR for #34159.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests

---------

Co-authored-by: Scott Gress <scott@fleetdm.com>
2025-10-17 16:09:14 -05:00
Jordan Montgomery
23a4b781fd
ISII: Make setup_experience gitops flag on app_store_apps enable setup for all platforms an app supports (#34461)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #30890 

Unreleased bug(or requirements miss) in iOS/iPadOS setup experience. Per
@noahtalerman we want setup_experience on VPP apps to apply to all
platforms an app is available for:
https://github.com/fleetdm/fleet/issues/30890#issuecomment-3408445511

This will change with the upcoming inclusion of the `platform` field and
it will be possible to specify settings for each individual OS

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-10-17 15:16:34 -04:00
jacobshandling
1c0b004676
Server changes for matching JetBrains IDE vulnerabilities (#34459)
## Addresses #32266 

These are the same changes present in
https://github.com/fleetdm/fleet/pull/34331, excluding changes that
would instantly be released. This separation will allow these
already-approved server changes to be merged before the cut while
allowing more time for QA to get to the QA-on-branch parts of the issue
2025-10-17 11:50:29 -07:00
Sarah Gillespie
e11ddc9866
Support $FLEET_VAR_SCEP_RENEWAL_ID in OU field (#34403) 2025-10-17 12:47:04 -05:00
jacobshandling
5f626e2a8c
Add gigs_all_disk_space vital collection, storage, service, and UI rendering for Linux hosts (#34077)
## Addresses #31671 

- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added total disk space metrics for all partitions on Linux hosts. The
disk space indicator now displays comprehensive storage information
including root partition and all other partitions, improving visibility
into host storage capacity.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-10-17 08:24:23 -07:00
Victor Lyuboslavsky
9295a82e83
Improved MySQL query performance software versions and vulnerabilities endpoints (#34262)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #32178

Software optimization: skipping an unnecessary software_cve join when
vulnerability details are not needed. Vulnerabilities are still
returned, so functionality remains unchanged.

Vulnerabilities optimization: Query vulnerability_host_counts directly
and LEFT JOIN for metadata. This eliminates the expensive UNION of all
CVE rows that was causing performance issues.

Previous approach: UNION all CVEs (many rows) → JOIN
vulnerability_host_counts → filter
New approach: Start with filtered vulnerability_host_counts → LEFT JOIN
for metadata
This reduces the working set before any expensive operations

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [ ] Added/updated automated tests (see below for the test)
- [x] QA'd all new/changed functionality manually
- Planning to test in loadtest after also improving the software
endpoint

Performance test for replicating the problem and testing the fix:
```go
package mysql

import (
	"context"
	"fmt"
	"testing"
	"time"

	"github.com/fleetdm/fleet/v4/server/fleet"
	"github.com/fleetdm/fleet/v4/server/ptr"
	"github.com/fleetdm/fleet/v4/server/test"
	"github.com/stretchr/testify/require"
)

// TestListVulnerabilitiesPerformance is a performance test that replicates
// the production performance problem with ListVulnerabilities.
//
// This test creates a realistic dataset with thousands of CVEs and measures
// query performance under various conditions. Run with:
//
//	go test -v -run TestListVulnerabilitiesPerformance ./server/datastore/mysql
//
// To see detailed timing output, set the environment variable:
//
//	VERBOSE=1 go test -v -run TestListVulnerabilitiesPerformance ./server/datastore/mysql
func TestListVulnerabilitiesPerformance(t *testing.T) {
	if testing.Short() {
		t.Skip("skipping performance test in short mode")
	}

	ds := CreateMySQLDS(t)
	defer TruncateTables(t, ds)

	ctx := context.Background()

	// Create a realistic dataset
	t.Log("Setting up test data...")
	setupPerformanceTestData(t, ds)
	t.Log("Test data setup complete")

	// Test cases covering common query patterns
	testCases := []struct {
		name string
		opts fleet.VulnListOptions
	}{
		{
			name: "Global list - first page, sorted by host count",
			opts: fleet.VulnListOptions{
				IsEE: true,
				ListOptions: fleet.ListOptions{
					Page:           0,
					PerPage:        20,
					OrderKey:       "hosts_count",
					OrderDirection: fleet.OrderDescending,
				},
			},
		},
		{
			name: "Team 1 list - first page, sorted by host count",
			opts: fleet.VulnListOptions{
				IsEE:   true,
				TeamID: ptr.Uint(1),
				ListOptions: fleet.ListOptions{
					Page:           0,
					PerPage:        20,
					OrderKey:       "hosts_count",
					OrderDirection: fleet.OrderDescending,
				},
			},
		},
		{
			name: "Team 1 list - with exploit filter",
			opts: fleet.VulnListOptions{
				IsEE:         true,
				TeamID:       ptr.Uint(1),
				KnownExploit: true,
				ListOptions: fleet.ListOptions{
					Page:           0,
					PerPage:        20,
					OrderKey:       "hosts_count",
					OrderDirection: fleet.OrderDescending,
				},
			},
		},
		{
			name: "Global list - with CVE search",
			opts: fleet.VulnListOptions{
				IsEE: true,
				ListOptions: fleet.ListOptions{
					Page:           0,
					PerPage:        20,
					MatchQuery:     "2023",
					OrderKey:       "hosts_count",
					OrderDirection: fleet.OrderDescending,
				},
			},
		},
		{
			name: "Global list - second page",
			opts: fleet.VulnListOptions{
				IsEE: true,
				ListOptions: fleet.ListOptions{
					Page:           1,
					PerPage:        20,
					OrderKey:       "hosts_count",
					OrderDirection: fleet.OrderDescending,
				},
			},
		},
		{
			name: "Free version - global list",
			opts: fleet.VulnListOptions{
				IsEE: false,
				ListOptions: fleet.ListOptions{
					Page:           0,
					PerPage:        20,
					OrderKey:       "hosts_count",
					OrderDirection: fleet.OrderDescending,
				},
			},
		},
	}

	// Run performance tests
	for _, tc := range testCases {
		t.Run(tc.name, func(t *testing.T) {
			// Warm up the query cache
			_, _, err := ds.ListVulnerabilities(ctx, tc.opts)
			require.NoError(t, err)

			// Measure query performance
			const iterations = 5
			var totalDuration time.Duration

			for i := 0; i < iterations; i++ {
				start := time.Now()
				vulns, meta, err := ds.ListVulnerabilities(ctx, tc.opts)
				duration := time.Since(start)
				totalDuration += duration

				require.NoError(t, err)
				require.NotNil(t, meta)
				require.NotEmpty(t, vulns, "expected vulnerabilities to be returned")

				if i == 0 {
					t.Logf("  First run: %v (returned %d results)", duration, len(vulns))
				}
			}

			avgDuration := totalDuration / iterations
			t.Logf("  Average of %d runs: %v", iterations, avgDuration)

			// Performance assertions
			// These thresholds represent the current performance problem
			// After optimization, these should be reduced significantly
			if avgDuration > 2*time.Second {
				t.Logf("  ⚠️  WARNING: Query took %v (>2s) - performance issue detected", avgDuration)
			} else if avgDuration > 500*time.Millisecond {
				t.Logf("  ⚠️  SLOW: Query took %v (>500ms)", avgDuration)
			} else {
				t.Logf("  ✓ GOOD: Query took %v (<500ms)", avgDuration)
			}
		})
	}

	// Test count query performance
	t.Run("Count vulnerabilities performance", func(t *testing.T) {
		opts := fleet.VulnListOptions{
			IsEE: true,
		}

		// Warm up
		_, err := ds.CountVulnerabilities(ctx, opts)
		require.NoError(t, err)

		// Measure
		const iterations = 5
		var totalDuration time.Duration

		for i := 0; i < iterations; i++ {
			start := time.Now()
			count, err := ds.CountVulnerabilities(ctx, opts)
			duration := time.Since(start)
			totalDuration += duration

			require.NoError(t, err)
			require.Greater(t, count, uint(0))

			if i == 0 {
				t.Logf("  First run: %v (count=%d)", duration, count)
			}
		}

		avgDuration := totalDuration / iterations
		t.Logf("  Average of %d runs: %v", iterations, avgDuration)

		if avgDuration > 2*time.Second {
			t.Logf("  ⚠️  WARNING: Count query took %v (>2s)", avgDuration)
		} else if avgDuration > 500*time.Millisecond {
			t.Logf("  ⚠️  SLOW: Count query took %v (>500ms)", avgDuration)
		} else {
			t.Logf("  ✓ GOOD: Count query took %v (<500ms)", avgDuration)
		}
	})
}

// BenchmarkListVulnerabilities provides benchmark results for ListVulnerabilities.
// Run with:
//
//	go test -bench=BenchmarkListVulnerabilities -benchmem -run=^$ ./server/datastore/mysql
func BenchmarkListVulnerabilities(b *testing.B) {
	ds := CreateMySQLDSForBenchmark(b)
	defer TruncateTables(b, ds)

	ctx := context.Background()

	// Setup test data
	setupPerformanceTestData(b, ds)

	b.ResetTimer()

	// Benchmark the most common query pattern
	opts := fleet.VulnListOptions{
		IsEE: true,
		ListOptions: fleet.ListOptions{
			Page:           0,
			PerPage:        20,
			OrderKey:       "hosts_count",
			OrderDirection: fleet.OrderDescending,
		},
	}

	for i := 0; i < b.N; i++ {
		_, _, err := ds.ListVulnerabilities(ctx, opts)
		if err != nil {
			b.Fatal(err)
		}
	}
}

// BenchmarkListVulnerabilitiesWithTeam benchmarks team-specific queries
func BenchmarkListVulnerabilitiesWithTeam(b *testing.B) {
	ds := CreateMySQLDSForBenchmark(b)
	defer TruncateTables(b, ds)

	ctx := context.Background()
	setupPerformanceTestData(b, ds)

	b.ResetTimer()

	opts := fleet.VulnListOptions{
		IsEE:   true,
		TeamID: ptr.Uint(1),
		ListOptions: fleet.ListOptions{
			Page:           0,
			PerPage:        20,
			OrderKey:       "hosts_count",
			OrderDirection: fleet.OrderDescending,
		},
	}

	for i := 0; i < b.N; i++ {
		_, _, err := ds.ListVulnerabilities(ctx, opts)
		if err != nil {
			b.Fatal(err)
		}
	}
}

// BenchmarkCountVulnerabilities benchmarks the count query
func BenchmarkCountVulnerabilities(b *testing.B) {
	ds := CreateMySQLDSForBenchmark(b)
	defer TruncateTables(b, ds)

	ctx := context.Background()
	setupPerformanceTestData(b, ds)

	b.ResetTimer()

	opts := fleet.VulnListOptions{
		IsEE: true,
	}

	for i := 0; i < b.N; i++ {
		_, err := ds.CountVulnerabilities(ctx, opts)
		if err != nil {
			b.Fatal(err)
		}
	}
}

// setupPerformanceTestData creates a realistic dataset that mimics production
// This creates:
// - ~80,000+ unique CVEs (matching production scale)
// - ~73,000 software_cve entries
// - ~35,000 operating_system_vulnerabilities entries
// - Multiple teams
// - Various host counts per vulnerability
//
// Note: This will take several minutes to run but will replicate production performance issues
func setupPerformanceTestData(t testing.TB, ds *Datastore) {
	ctx := context.Background()

	// Create 100 hosts across different teams and OS types (doubled from 50)
	// More hosts = more realistic host count distributions
	hosts := make([]*fleet.Host, 100)
	for i := 0; i < 100; i++ {
		hosts[i] = test.NewHost(t, ds, fmt.Sprintf("host%d", i),
			fmt.Sprintf("192.168.1.%d", i%255+1), // Handle more than 255 hosts
			fmt.Sprintf("key%d", i),
			fmt.Sprintf("uuid%d", i),
			time.Now())
	}

	// Create 3 teams
	team1, err := ds.NewTeam(ctx, &fleet.Team{Name: "Engineering"})
	require.NoError(t, err)

	team2, err := ds.NewTeam(ctx, &fleet.Team{Name: "Sales"})
	require.NoError(t, err)

	team3, err := ds.NewTeam(ctx, &fleet.Team{Name: "Support"})
	require.NoError(t, err)

	// Distribute hosts across teams
	// 40 hosts in team1, 30 in team2, 20 in team3, 10 with no team
	err = ds.AddHostsToTeam(ctx, fleet.NewAddHostsToTeamParams(&team1.ID, getHostIDs(hosts[0:40])))
	require.NoError(t, err)

	err = ds.AddHostsToTeam(ctx, fleet.NewAddHostsToTeamParams(&team2.ID, getHostIDs(hosts[40:70])))
	require.NoError(t, err)

	err = ds.AddHostsToTeam(ctx, fleet.NewAddHostsToTeamParams(&team3.ID, getHostIDs(hosts[70:90])))
	require.NoError(t, err)

	// Set up OS versions (Windows, macOS, Ubuntu)
	windowsOS := fleet.OperatingSystem{
		Name:     "Microsoft Windows 11 Enterprise",
		Version:  "10.0.22621.2715",
		Arch:     "x86_64",
		Platform: "windows",
	}

	macOS := fleet.OperatingSystem{
		Name:     "macOS",
		Version:  "14.1.2",
		Arch:     "arm64",
		Platform: "darwin",
	}

	ubuntuOS := fleet.OperatingSystem{
		Name:     "Ubuntu",
		Version:  "22.04",
		Arch:     "x86_64",
		Platform: "ubuntu",
	}

	// Assign OS to hosts: 50 Windows, 30 macOS, 20 Ubuntu
	for i := 0; i < 50; i++ {
		err = ds.UpdateHostOperatingSystem(ctx, hosts[i].ID, windowsOS)
		require.NoError(t, err)
	}
	for i := 50; i < 80; i++ {
		err = ds.UpdateHostOperatingSystem(ctx, hosts[i].ID, macOS)
		require.NoError(t, err)
	}
	for i := 80; i < 100; i++ {
		err = ds.UpdateHostOperatingSystem(ctx, hosts[i].ID, ubuntuOS)
		require.NoError(t, err)
	}

	err = ds.UpdateOSVersions(ctx)
	require.NoError(t, err)

	// Create realistic CVE distribution matching production scale
	// In production, we see:
	// - ~73,000 software CVEs
	// - ~35,000 OS CVEs
	// - Many CVEs overlap between software and OS
	// - ~80,000 unique CVEs after deduplication

	// We now create production-scale data:
	// - 50,000 software CVEs (70% of production)
	// - 30,000 OS CVEs (85% of production)
	// - Some overlap to create ~80,000 total CVEs after UNION
	// This should replicate the 500-1000ms+ query times seen in production

	t.Log("Creating software vulnerabilities... (this will take a few minutes)")
	createSoftwareVulnerabilities(t, ds, hosts[:60], 50000)

	t.Log("Creating OS vulnerabilities...")
	createOSVulnerabilities(t, ds, 30000)

	t.Log("Creating CVE metadata...")
	createCVEMetadata(t, ds, 80000)

	t.Log("Updating vulnerability host counts...")
	err = ds.UpdateVulnerabilityHostCounts(ctx, 10)
	require.NoError(t, err)

	t.Log("Setup complete - ready for performance testing")
}

// createSoftwareVulnerabilities creates software entries and their CVEs
func createSoftwareVulnerabilities(t testing.TB, ds *Datastore, hosts []*fleet.Host, numCVEs int) {
	ctx := context.Background()

	// Create more software packages to better distribute CVEs
	softwarePackages := []fleet.Software{
		{Name: "Chrome", Version: "120.0.1", Source: "programs"},
		{Name: "Firefox", Version: "121.0", Source: "programs"},
		{Name: "Node.js", Version: "18.19.0", Source: "programs"},
		{Name: "Python", Version: "3.11.7", Source: "programs"},
		{Name: "Docker", Version: "24.0.7", Source: "programs"},
		{Name: "nginx", Version: "1.24.0", Source: "deb_packages"},
		{Name: "postgresql", Version: "15.5", Source: "deb_packages"},
		{Name: "redis", Version: "7.2.3", Source: "deb_packages"},
		{Name: "mysql", Version: "8.0.35", Source: "deb_packages"},
		{Name: "git", Version: "2.43.0", Source: "deb_packages"},
		{Name: "openssl", Version: "3.0.12", Source: "deb_packages"},
		{Name: "curl", Version: "8.5.0", Source: "deb_packages"},
		{Name: "vim", Version: "9.0", Source: "deb_packages"},
		{Name: "apache2", Version: "2.4.58", Source: "deb_packages"},
		{Name: "php", Version: "8.2.14", Source: "deb_packages"},
	}

	// Install software on hosts
	for i, host := range hosts {
		// Each host gets 5-8 software packages
		numPackages := 5 + (i % 4)
		if numPackages > len(softwarePackages) {
			numPackages = len(softwarePackages)
		}
		hostSoftware := softwarePackages[:numPackages]
		_, err := ds.UpdateHostSoftware(ctx, host.ID, hostSoftware)
		require.NoError(t, err)
	}

	err := ds.SyncHostsSoftware(ctx, time.Now())
	require.NoError(t, err)

	// Create CVEs for software (distributed across 15 software IDs)
	// Each software gets many CVEs to simulate real-world vulnerability distribution
	cvesPerSoftware := numCVEs / 15
	t.Logf("  Creating %d CVEs per software package (15 packages)", cvesPerSoftware)

	for softwareID := uint(1); softwareID <= 15; softwareID++ {
		// Insert CVEs in batches for better performance
		batchSize := 1000
		for batchStart := 0; batchStart < cvesPerSoftware; batchStart += batchSize {
			batchEnd := batchStart + batchSize
			if batchEnd > cvesPerSoftware {
				batchEnd = cvesPerSoftware
			}

			for i := batchStart; i < batchEnd; i++ {
				cveNum := int(softwareID-1)*cvesPerSoftware + i
				// Use wider CVE number range to avoid duplicates
				cve := fmt.Sprintf("CVE-2023-%05d", cveNum)

				_, err := ds.InsertSoftwareVulnerability(ctx, fleet.SoftwareVulnerability{
					SoftwareID: softwareID,
					CVE:        cve,
				}, fleet.NVDSource)
				require.NoError(t, err)
			}
		}

		if softwareID%5 == 0 {
			t.Logf("  Progress: %d/%d software packages completed", softwareID, 15)
		}
	}
}

// createOSVulnerabilities creates OS vulnerabilities
func createOSVulnerabilities(t testing.TB, ds *Datastore, numCVEs int) {
	ctx := context.Background()

	// Create CVEs for each OS type
	// OS ID 1 = Windows, 2 = macOS, 3 = Ubuntu
	osIDs := []uint{1, 2, 3}
	cvesPerOS := numCVEs / len(osIDs)

	t.Logf("  Creating %d CVEs per OS type (3 OS types)", cvesPerOS)

	for _, osID := range osIDs {
		// Insert in batches to avoid memory issues with large slices
		batchSize := 5000
		totalBatches := (cvesPerOS + batchSize - 1) / batchSize

		for batchNum := 0; batchNum < totalBatches; batchNum++ {
			batchStart := batchNum * batchSize
			batchEnd := batchStart + batchSize
			if batchEnd > cvesPerOS {
				batchEnd = cvesPerOS
			}

			vulns := make([]fleet.OSVulnerability, batchEnd-batchStart)
			for i := 0; i < len(vulns); i++ {
				actualIndex := batchStart + i
				// Use different year to avoid overlap with software CVEs
				// (but still create some overlap)
				cveNum := int(osID-1)*cvesPerOS + actualIndex
				cve := fmt.Sprintf("CVE-2022-%05d", cveNum)

				// 10% overlap with software CVEs for realism
				if actualIndex%10 == 0 {
					cve = fmt.Sprintf("CVE-2023-%05d", cveNum)
				}

				vulns[i] = fleet.OSVulnerability{
					OSID: osID,
					CVE:  cve,
				}
			}

			_, err := ds.InsertOSVulnerabilities(ctx, vulns, fleet.MSRCSource)
			require.NoError(t, err)

			if (batchNum+1)%2 == 0 || batchNum == totalBatches-1 {
				t.Logf("  Progress: OS %d - batch %d/%d completed", osID, batchNum+1, totalBatches)
			}
		}
	}
}

// createCVEMetadata creates CVE metadata entries
func createCVEMetadata(t testing.TB, ds *Datastore, numCVEs int) {
	ctx := context.Background()
	mockTime := time.Date(2023, 1, 1, 0, 0, 0, 0, time.UTC)

	// Create metadata in batches of 500 for better performance
	batchSize := 500
	totalBatches := (numCVEs + batchSize - 1) / batchSize
	t.Logf("  Creating CVE metadata in %d batches", totalBatches)

	for start := 0; start < numCVEs; start += batchSize {
		end := start + batchSize
		if end > numCVEs {
			end = numCVEs
		}

		batch := make([]fleet.CVEMeta, end-start)
		for i := start; i < end; i++ {
			// Alternate between 2022 and 2023 CVEs to match our created vulnerabilities
			year := 2023
			if i >= 50000 {
				year = 2022
			}

			// Use 5-digit format to match our CVE creation
			cve := fmt.Sprintf("CVE-%d-%05d", year, i%100000)

			// 30% have CISA known exploit
			cisaExploit := (i % 10) < 3

			batch[i-start] = fleet.CVEMeta{
				CVE:              cve,
				CVSSScore:        ptr.Float64(5.0 + float64(i%50)/10.0),
				EPSSProbability:  ptr.Float64(float64(i%100) / 100.0),
				CISAKnownExploit: ptr.Bool(cisaExploit),
				Published:        ptr.Time(mockTime.Add(time.Duration(i) * time.Hour)),
				Description:      fmt.Sprintf("Test vulnerability %s", cve),
			}
		}

		err := ds.InsertCVEMeta(ctx, batch)
		require.NoError(t, err)

		// Report progress every 10 batches
		batchNum := (start / batchSize) + 1
		if batchNum%10 == 0 || batchNum == totalBatches {
			t.Logf("  Progress: %d/%d batches completed (%d CVEs)", batchNum, totalBatches, end)
		}
	}
}

// getHostIDs extracts host IDs from a slice of hosts
func getHostIDs(hosts []*fleet.Host) []uint {
	ids := make([]uint, len(hosts))
	for i, h := range hosts {
		ids[i] = h.ID
	}
	return ids
}

// CreateMySQLDSForBenchmark creates a datastore for benchmarking
func CreateMySQLDSForBenchmark(b *testing.B) *Datastore {
	return CreateMySQLDS(b)
}
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Performance Improvements**
* Faster loading of the vulnerabilities list via optimized database
queries for the vulnerabilities API endpoint.

* **Bug Fixes**
* More accurate “created at” timestamps for vulnerabilities, improving
sorting and consistency.
* More consistent source attribution for vulnerabilities when multiple
sources are available.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-10-17 09:57:47 -05:00
RachelElysia
1ef91fe4e3
Feature: Script only package e2e followup (#34271)
Co-authored-by: Carlo DiCelico <carlo@fleetdm.com>
2025-10-17 10:54:00 -04:00
Jordan Montgomery
927fd1d240
Fix android policy non-compliance errors (#34430)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34422 

Unreleased bug in 4.75.0 Android Config Profile feature. No changes file
as such

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-10-17 10:27:24 -04:00
Scott Gress
61970118e9
Stop setup experience on software install failure (#34173)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #33173
**Related issue:** Resolves #33111 

# Details

This is the remaining work to implement the "Stop the setup experience
when required software fails to install" feature. This didn't turn out
to be quite as straightforward as expected so I ended up doing a bit of
design-by-code and expect some feedback on the approach. I tried to make
it as low-touch as possible. The general design is:

1. In the `maybeUpdateSetupExperienceStatus` function which is called in
various places when a setup experience step is marked as completed, call
a new `maybeCancelPendingSetupExperienceSteps` function if the setup
step was marked as failed. Similarly call
`maybeCancelPendingSetupExperienceSteps` if a VPP app install fails to
enqueue.
2. In `maybeCancelPendingSetupExperienceSteps`, check whether the
specified host is MacOS and whether the "RequireAllSoftwareMacOS" flag
is set in the team (or global) config. If so, mark the remaining setup
experience items as canceled and cancel any upcoming activities related
to those steps.
3. On the front-end, if the `require_all_software_macos` is set and a
software step is marked as failed, show a new failure page indicating
that setup has failed and showing details of the failed software.
4. On the agent side, when checking setup experience status, send a
`reset_after_failure` flag _only the first time_. If this flag is set,
then the code in the `/orbit/setup_experience/status` handler will clear
and re-queue any failed setup experience steps (but leave successful
steps to avoid re-installing already-installed software). This
facilitates re-starting the setup experience when the host is rebooted.

I also updated the way that software (packages and VPP) is queued up for
the setup experience to be ordered alphabetically, to make it easier to
test _and_ because this is a desired outcome for a future story. Since
the order is not deterministic now, this update shouldn't cause any
problems (aside from a couple of test updates), but I'm ok taking it out
if desired.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [X] Added/updated automated tests
* Added a new integration test for software packages, testing that a
failed software package causes the rest of the setup experience to be
marked as failed when `require_all_software_macos` is set, and testing
that the "reset after failure" code works.
* Added a new integration test for VPP packages, testing that a failed
VPP enqueue causes the same halting of the setup experience.
I _don't_ have test for a failure _during_ a VPP install. It should go
through the same code path as the software package failure, so it's not
a huge gap.

- [ ] QA'd all new/changed functionality manually
Working on it 

## fleetd/orbit/Fleet Desktop

- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [X] Verified that fleetd runs on macOS, Linux and Windows


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
- Configurable option to halt macOS device setup if any software install
fails.
- Device setup page now shows a clear “Device setup failed” state with
expandable error details when all software is required on macOS.
- Improvements
- Setup status now includes per-step error messages for better
troubleshooting.
- Pending setup steps are automatically canceled after a failure when
applicable, with support to reset and retry the setup flow as
configured.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-10-17 08:38:53 -05:00
Ian Littman
24ceeee0b4
Add CVE exclusion for Dota when we don't report the version number correctly (#34384)
Resolves #34323. Bit heavy-handed but our osquery ingestion doesn't pick
up the version number right now so this is as good as we can do, and if
we _do_ start picking up the version number this fix is surgical enough
that it won't create a false negative.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
2025-10-16 14:35:56 -05:00
Magnus Jensen
4609bca8cf
AP: Validate top-level keys in android profile upload (#34360)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34334 



# Checklist for submitter

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-10-16 14:06:05 -03:00
Ian Littman
e48d8033a9
Map manjaro-arm platform (#34357)
Resolves #34318. Thx @jmwatts for the QA on this!

- [x] QA'd all new/changed functionality manually
2025-10-16 11:00:05 -05:00
Jordan Montgomery
d8d25e6297
Clear profiles on Android host unenroll (#34343)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34335

No changes file as this is an unreleased bug in 4.75.0 and covered by
initial feature changes file

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-10-16 11:06:16 -04:00
Jordan Montgomery
abaf15ff51
Do not return MDM=off android hosts from reconciler (#34304)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34299 

Unreleased bug in Android Config profiles 4.75.0 feature. No changes
file as such. I'm not entirely sure how to cause this as I was unable to
repro it locally, there may be a timing issue or something, so I haven't
fully QA'd manually. QA was limited to verifying basic reconciler
functionality

Also updated Website endpoint to not throw a 5XX since we expect this to
occasionally happen

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [ ] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-10-16 09:59:17 -04:00
Carlo
50ff9249c4
Clear unsupported params for script packages (#34297)
Implements #33752. Clears unsupported parameters for `.sh` and `.ps1` installers, plus tests.
2025-10-15 20:43:58 -04:00
Magnus Jensen
717151f2ec
LM: fix missing error for manually enrolled ios and ipados when locking (#34216)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34209 



# Checklist for submitter

If some of the following don't apply, delete the relevant line.


## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-10-15 17:24:49 -03:00
Magnus Jensen
8c4b5f9371
LM: Fix deleted iOS/iPadOS checking in does not update lost mode status (#34250)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34203 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
2025-10-15 17:24:40 -03:00
Magnus Jensen
3e7fde5fef
remove println and fix datastore mock generator (#34305)
Small fixes, that was missed in #34292
2025-10-15 17:22:51 -03:00
Magnus Jensen
038c62aa0e
Android Unenroll: fix multiple unenroll activities (#34292)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34285 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-10-15 16:26:38 -03:00
Jordan Montgomery
0c98bce37e
Set proxy auth token in Android profile reconciler (#34294)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34261 

Longer term we should rework the calls that create the Android API
client to do this but this small fix will suffice for now to unblock QA

No changes file as this is unreleased bugfix into 4.75.0 and changes
covered by main story ticket

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
2025-10-15 15:14:31 -04:00
Scott Gress
45a8749d1a
Use host IDs instead of host names when doing generate-gitops for manual labels (#34254)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34225

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually
Did a `generate-gitops` for a manual label, noted the correct IDs were
output for hosts. Use `gitops` to re-apply the label, saw the label
membership was applied correctly.

For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
2025-10-15 13:31:40 -05:00
Scott Gress
ca61532d66
Fix issue where incorrect host gets added to label (#34237)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34236 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [X] Added/updated automated tests
Added new integration test for adding label w/ GitOps, with this
specific case tested. It fails on `main` and passes on this branch.
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
2025-10-15 13:24:17 -05:00
Ian Littman
7587e06252
Tweak Poly Lens Desktop software name on ingestion (#34287)
To match revision in #34159 spec spreadsheet.

- [x] Added/updated automated tests
2025-10-15 11:16:05 -05:00
Victor Lyuboslavsky
b9520f6b14
Changed the default name of Host Identity CA to 'Fleet Host Identity CA' to avoid conflict with Fleet's Apple MDM CA. (#34219)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #34217

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] QA'd all new/changed functionality manually
2025-10-15 09:52:45 -05:00
Ian Littman
acff7b7343
Add software name cleanup on ingestion for various items where bundle executable isn't well-named (#34232)
Fixes #34159. Split from CPE translation fixes so this can be merged
into `main` pre-QA.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests

- [ ] QA'd all new/changed functionality manually
2025-10-15 09:50:52 -05:00
Gabriel Hernandez
6e391178f6
send back correct platform in mdm unenroll activity when unenrolling android (#34242)
**Related issue:** Resolves #33807

sends back the correct platform on the activity details for the mdm
unenroll activitiy for android
2025-10-15 13:19:51 +01:00
Lucas Manuel Rodriguez
15518d2893
Optimize software title reconciliation in vulnerabilities job (#34146)
Resolves #34055

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Optimized software title reconciliation used during vulnerability
processing, improving scan performance and reducing database load. More
efficient cleanup of orphaned titles and updates to title names.
* **Tests**
  * Corrected a test name typo for clarity.
* Streamlined MDM integration test by removing redundant title
recreation steps.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-10-14 17:36:45 -05:00
Konstantin Sykulev
bb4b62bd0f
Adding name to software checksum for mac software (#34097)
**Related issue:** Resolves #28788

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually

## Database migrations

- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* macOS app checksums now include the app name, improving grouping,
deduplication, and preventing mis-linking or duplicate entries when
multiple names share a bundle ID.
* More stable title handling when bundle IDs are missing, reducing
unintended renames and mismatches.

* **Tests**
* Re-enabled related host-software tests and added a
longest-common-prefix test to validate name reconciliation.

* **Chores**
* Database migration added to recalculate checksums for affected macOS
app records.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-10-14 17:36:34 -05:00
Jonathan Katz
c5ad64056c
32751 Change installer add/edit endpoints to read from writer db instead of reader (#33820)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #32751 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [ ] QA'd all new/changed functionality manually

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
2025-10-14 16:59:01 -04:00
Magnus Jensen
d99afbe9aa
Better api error for EUA manual macOS profile download (#34101)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** Resolves #33447 

User error:
<img width="1020" height="213" alt="image"
src="https://github.com/user-attachments/assets/daa90fa3-aa43-4734-9f29-4744ca6fb3e2"
/>

As far as I could tell it was not possible to customize the error
message shown when installing the failing profile.
We have some code that mentions iOS/iPadOS and after replicating, it
seems it does not work the same for macOS.


# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests

- [x] QA'd all new/changed functionality manually

---------

Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2025-10-14 16:33:12 -03:00
Carlo
f42e45eeec
Hotfix 34099 merge (#34191)
Fixes main for bad merge related to #34099 causing [build
failures](https://github.com/fleetdm/fleet/actions/runs/18499765354/job/52713214491?pr=34186#step:14:182).
2025-10-14 11:08:44 -04:00
Tim Lee
2b18caaee1
Add Jetbrains plugins (#34024) 2025-10-14 09:01:45 -06:00