Commit graph

450 commits

Author SHA1 Message Date
Lucas Manuel Rodriguez
330a708392
Changes to not rely on Fleet Desktop for Linux setup experience (#33018)
For #32788.

## Testing

- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually

## fleetd/orbit/Fleet Desktop

- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [X] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
- Enhanced Linux setup experience: persists status on disk, resumes
automatically, and completes when software/scripts finish.
- Opens the “My Device” page only when desktop is enabled, using a
user-aware launcher on Linux.
- Linux setup status now focuses on software progress for faster,
clearer feedback.

- Bug Fixes
- Corrected auth/MDM checks: macOS requires Apple MDM; Linux no longer
blocked by MDM configuration on shared endpoints.
- Improved reliability and logging around software installation and
temporary directory cleanup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-09-16 13:26:00 -03:00
Jordan Montgomery
572536d466
Skip setup experience during AxM based migrations (#32822)
Fixes #32096

The gist of the fix is that when syncing devices from DEP we save the
migration deadline to our host_dep_assignments table. The next
enrollment, which we assume should be the migration, looks at
host_dep_assignments, sees that mdm_migration_deadline is non-Null and
mdm_migration_completed is NULL, and uses that as the signal that a
migration is in progress and skips enqueuing setup experience items. It
then marks the migration as complete which sets mdm_migration_completed
= mdm_migration_deadline. Once this is set setup experience will run as
normal unless mdm_migration_completed gets set to NULL and/or
mdm_migration_deadline gets set to a value in the future(which e.g.
would happen if the customer assigned to another MDM server then
assigned to migrate to fleet again)

DB test failure is expected here because it won't like the migration
timestamp but that is a necessary failure because this fix is going to
be backported into 4.73

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes

## Testing

- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [x] Alerted the release DRI if additional load testing is needed

## Database migrations

- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* New Features
  * Tracks and stores Apple DEP MDM migration deadlines per device/host.
  * Detects “migration in progress” during DEP sync and check-in.
* Automatically marks migration complete and skips Setup Assistant items
while migration is in progress to prevent conflicts.

* Bug Fixes
* Improved DEP compatibility by updating the protocol version and
User-Agent used for Apple’s APIs, reducing the chance of blocked or
rejected requests.

* Migrations
* Adds fields to support migration deadlines and completion status (no
action required).

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
2025-09-11 09:40:40 -04:00
Magnus Jensen
4c59270795
Update error messages of MDM not being configured with the respective platform (#32800)
Just stumbled upon this and it was not clear what platform it was
failing on, so I just updated the errors to include either Apple or
Windows based on the platform.
2025-09-11 10:17:31 +03:00
Lucas Manuel Rodriguez
33a15831c0
Add missing platform_like during orbit enrollment (#32671)
#30877

We need to send `platform_like` during orbit enrollment for proper setup
experience for Linux

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [X] Added/updated automated tests

- [X] QA'd all new/changed functionality manually

## fleetd/orbit/Fleet Desktop

- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
2025-09-05 16:05:19 -03:00
Lucas Manuel Rodriguez
29475ab55e
API endpoints for Linux setup experience (#32493)
For #32040.

---

Backend changes to unblock the development of the orbit and frontend
changes.

New GET and PUT APIs for setting/getting software for Linux Setup
Experience:
```
curl -k -X GET -H "Authorization: Bearer $TEST_TOKEN" https://localhost:8080/api/latest/fleet/setup_experience/linux/software?team_id=8&per_page=3000
curl -k -X PUT -H "Authorization: Bearer $TEST_TOKEN" https://localhost:8080/api/latest/fleet/setup_experience/linux/software -d '{"team_id":8,"software_title_ids":[3000, 3001, 3007]}'
```

New setup_experience/init API called by orbit to trigger the Linux setup
experience on the device:
```
curl -v -k -X POST -H "Content-Type: application/json" "https://localhost:8080/api/fleet/orbit/setup_experience/init" -d '{"orbit_node_key": "ynYEtFsvv9xZ7rX619UE8of1I28H+GCj"}'
```

Get status API to call on "My device":
```
curl -v -k -X POST "https://localhost:8080/api/latest/fleet/device/7d940b6e-130a-493b-b58a-2b6e9f9f8bfc/setup_experience/status"
```

---

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [x] QA'd all new/changed functionality manually

## New Fleet configuration settings

- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
  - Added Linux support for Setup Experience alongside macOS.
- Introduced platform-specific admin APIs to configure and retrieve
Setup Experience software (macOS/Linux).
- Added device API to report Setup Experience status and an Orbit API to
initialize Setup Experience on non-macOS devices.
- Setup Experience now gates policy queries on Linux until setup is
complete.
- New activity log entry when Setup Experience software is edited
(includes platform and team).

- Documentation
- Updated audit logs reference to include the new “edited setup
experience software” event.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-09-04 12:58:47 -03:00
Victor Lyuboslavsky
84e45f6fa1
OpenTelemetry minor improvements (#32324)
Fixes #32313

  OpenTelemetry Tracing

- Added tracing to async task collectors: FlushHostsLastSeen,
collectHostsLastSeen, collectLabelQueryExecutions,
collectPolicyQueryExecutions, collectScheduledQueryStats
- Updated HTTP middleware to use OTEL semantic convention for span names
({method} {route})
  - Added OTELEnabled() helper to FleetConfig

  Optimizations

- Reduced OTEL batch size from 512 to 256 spans to prevent gRPC message
size errors
  - Enabled gzip compression for trace exports

NOTE: I tried to improve OTEL instrumentation for cron jobs, but it got
too complicated due to goroutines in `schedule.go` so that effort should
be separate. We do have SQL instrumentation for cron jobs, but we are
missing root spans for cron jobs as a whole.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [x] QA'd all new/changed functionality manually


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Expanded OpenTelemetry tracing for async tasks (host last seen, label
membership, policy membership, scheduled query stats) to provide richer
observability.
* More descriptive HTTP span names using “METHOD /route” for clearer
trace analysis.

* **Bug Fixes**
* Improved OTLP gRPC exporter reliability by enabling gzip compression
and reducing export batch size, mitigating intermittent gRPC errors.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-28 19:32:46 -05:00
Carlo
8212180819
Extract Android disk storage data (#32133)
Implements Android storage data extraction for issue #27080.
2025-08-22 12:27:15 -04:00
Juan Fernandez
9ec63e37a2
Consider team settings when getting detail queries (#32141)
For #28133.

This fixes a bug that prevented TPM PIN detail queries to be included if
the required settings are only set at the Team level.
2025-08-22 11:14:32 -04:00
Lucas Manuel Rodriguez
53b7a0628a
Prevent IT admins from deleting a secret variable in use (#32161)
#31977

Tested with:
224 scripts, 102 Apple configuration profiles, 105 Apple declarations,
37 Windows configuration profiles. Scattered in 7 teams.

No variables on profiles/scripts:
- scan scripts: 22.079416ms
- scan apple profiles: 1.865083ms
- scan apple declarations: 683.75µs
- scan windows profiles: 531.167µs

A variable "X" on with profiles/scripts that have variable "Y":
- scan scripts: 26.305459ms
- scan apple profiles: 1.841708ms
- scan apple declarations: 756.917µs
- scan windows profiles: 571.875µs

---

## Testing

- [X] Added/updated automated tests

- [X] QA'd all new/changed functionality manually

For unreleased bug fixes in a release candidate, one of:

- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [X] Alerted the release DRI if additional load testing is needed

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- New Features
- Prevents deleting a secret that’s still in use. The system now checks
scripts, Apple/Windows MDM profiles, and Apple declarations across teams
before allowing deletion.
- The delete dialog now shows a clear message indicating which item (and
team) is using the secret, replacing the previous generic error.
- Deletion succeeds once all references are removed, ensuring safer
secret management.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-22 11:22:37 -03:00
Juan Fernandez
51a5b6166a
Refactor failing policies total on Host endpoint (#31906)
For #29795

Refactored the way we compute the number of failing policies to avoid
discrepancies due to either read replica delays or due to async nature
of the failing policy computation stored in host issues.
2025-08-19 13:39:32 -04:00
Jonathan Katz
3388740f0e
4498 empty software (#31940)
Fixes: #4498 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- Added unit test 
- Changed existing unit tests to accept empty array instead of null

- [x] QA'd all new/changed functionality manually
- Tested that Fleet UI > host details, returns `software: []` instead of
nothing.
- Tested that with exclude_software=false, software returns the full
array for host.

---------

Co-authored-by: Anthony Maxwell <133805840+Illbjorn@users.noreply.github.com>
2025-08-19 10:38:53 -04:00
Lucas Manuel Rodriguez
58233817f0
Add backend APIs for adding, deleting and listing secret variables (#31936)
For #31055.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

## Testing

- [X] Added/updated automated tests

- [x] QA'd all new/changed functionality manually
2025-08-14 19:33:47 -03:00
Victor Lyuboslavsky
a0e9d88e0d
Updated SQL modes in tests to match production. (#31445)
Fixes #31444 

The changes are primarily in tests. The only changes in production code
are a couple validations/checks for invalid values in:
- mysql/apple_mdm.go
- mysql/hosts.go
- mysql/queries.go

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

## Testing

- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved handling of timestamp and default values across various
features to prevent database errors and warnings.
* Enhanced validation and data consistency for Apple Business Manager
tokens and MDM profiles.
* Updated test data and logic to comply with stricter database
constraints and realistic scenarios, including date handling and field
lengths.

* **Chores**
* Updated test setups to reflect schema changes, improve data integrity,
and avoid future compatibility issues.
  * Standardized SQL mode and timestamp usage in test environments.
* Refined test data for VPP apps, software installers, and device
enrollments for better reliability.

* **Tests**
* Expanded and updated tests to cover new fields, stricter validation,
and more accurate simulation of real-world conditions.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-03 08:18:13 +02:00
Jordan Montgomery
fea2b40cd8
BMAA: Add personal enrollment type including aggregate counts (#31091)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [ ] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [ ] Manual QA for all new/changed functionality
2025-07-22 17:24:19 -04:00
Juan Fernandez
fdfef5adf1
30311: Fix race condition in test (#30903)
For #30311 

Refactored `AddHostsToTeam` so that batch size can be specified via a
parameter and not a global variable.
2025-07-17 10:20:49 -04:00
Victor Lyuboslavsky
836cc044d2
Fleet server verifies HTTP signature (#30825)
Fixes #30473 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added support for TPM-backed host identity certificates enabling
hardware-backed HTTP signature authentication for hosts.
* Introduced HTTP signature verification middleware for API requests,
applied conditionally for premium licenses.
* Hosts presenting identity certificates must authenticate with matching
HTTP message signatures during enrollment and authentication.
* Added SCEP-based certificate issuance for secure host identity
management.
* Updated enrollment endpoints to use standardized request/response
contract types.

* **Bug Fixes**
* Enhanced authentication logic to verify consistency between host
identity certificates and host records, preventing duplicate or
mismatched identities.

* **Chores**
* Updated dependencies and test infrastructure to support HTTP signature
verification and host identity certificate workflows.
* Added comprehensive integration and datastore tests for host identity
certificate issuance, storage, and authentication.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-16 20:08:27 +02:00
Victor Lyuboslavsky
85a98d83dd
Refactor EnrollOrbit/EnrollHost (#30872)
Fixes #30473 

Refactore Datastore.EnrollHost and Datastore.EnrollOrbit methods to use
functional options. Doing this refactor before adding new options to
those methods. This should make the code more maintainable and easier to
understand.

No functional changes here. Just refactoring.

# Checklist for submitter

- [x] Added/updated automated tests


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Streamlined host and Orbit enrollment methods to use a flexible
options-based pattern instead of fixed parameter lists.
* Updated related tests and service logic to use the new options
approach, improving clarity and extensibility for enrollment operations.

* **New Features**
* Introduced configuration options for host and Orbit enrollment,
allowing more explicit and customizable parameter setting during
enrollment.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-15 17:22:02 -03:00
Lucas Manuel Rodriguez
502fb0c5cc
Support host vital labels with department without groups (#30598)
Fix for #23899.

I found this while preparing the demo (I had a user with department and
no groups).
Assigning to @getvictor because Scott is OOO.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved host filtering for Identity Provider (IdP) group labels to
include hosts without corresponding group entries.

* **Tests**
* Enhanced label tests to cover IdP department labels, including
scenarios where users have no groups.
* Added new subtests to verify correct host inclusion and label counts
for department-based labels.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-07 16:03:53 -03:00
Lucas Manuel Rodriguez
c69d56ed64
Replace home-made SAML implementation with https://github.com/crewjam/saml (#28486)
For https://github.com/fleetdm/confidential/issues/9931.


[Here](ec3e8edbdc/docs/Contributing/Testing-and-local-development.md (L339))'s
how to test SAML locally with SimpleSAML.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Improved SSO and SAML integration with enhanced session management
using secure cookies.
  * Added support for IdP-initiated login flows.
* Introduced new tests covering SSO login flows, metadata handling, and
error scenarios.

* **Bug Fixes**
* Enhanced validation and error handling for invalid or tampered SAML
responses.
  * Fixed session cookie handling during SSO and Apple MDM SSO flows.

* **Refactor**
* Replaced custom SAML implementation with the crewjam/saml library for
improved reliability.
  * Simplified SAML metadata parsing and session store management.
  * Streamlined SSO authorization request and response processing.
  * Removed deprecated fields and redundant code related to SSO.

* **Documentation**
* Updated testing and local development docs with clearer instructions
for SSO and IdP-initiated login.

* **Chores**
  * Upgraded dependencies including crewjam/saml and related packages.
* Cleaned up tests and configuration by removing deprecated fields and
unused imports.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-07 15:13:46 -03:00
Scott Gress
af2de5bc42
Add support for host vitals labels (#30278)
# Details

This PR adds support for a new label membership type, `host_vitals`.
Membership for these labels is based on a database query created from
user-supplied criteria. In this first iteration, the allowed criteria
are very simple: a label can specify either an IdP group or IdP
department, and hosts with linked users with a matching group or
department.

Groundwork is laid here for more complex host vitals queries, including
`and` and `or` logic, different data types and different kinds of vitals
(rather than just the "foreign" vitals of which IdP is an example).

Note that this PR does _not_ include the cron job that will trigger
membership updating, and it doesn't include ; for sake of simplicity in
review that will be done in a follow-on PR.

## Basic flow

### Creating a host vitals label

1. A new label is created via the API / GitOps with membership type
`host_vitals` and a `criteria` property that's a JSON blob. Currently
the JSON can only contain `vital` and `value` keys (and must contain
those keys)
2. The server validates that the specified `vital` exists in our [set of
known host
vitals](https://github.com/fleetdm/fleet/pull/30278/files#diff-b6d4c48f2624b82c2567b2b88db1de51c6b152eeb261d40acfd5b63a890839b7R418-R436).
3. The server validates that the [criteria can be parsed into a
query](https://github.com/fleetdm/fleet/pull/30278/files?diff=unified&w=1#diff-4ac4cfba8bed490e8ef125a0556f5417156f805017bfe93c6e2c61aa94ba8a8cR81-R86).
This also happens during GitOps dry run.
4. The label is saved (criteria is saved as JSON in the db)

### Updating membership for a host vitals label

1. The label's criteria is used to generate a query to run on the
_Fleet_ db.
1. For each vital criteria, check the vital type. Currently only foreign
vitals are supported.
   2. For foreign vitals, add its group to a set we keep track of.
3. Add a `WHERE` clause section for the vital and value, e.g.
`end_user_idp_groups = ?`
4. Once we have all the `WHERE` clauses, create the query as `SELECT %s
FROM %s` + any joins contributed by foreign vitals groups + `WHERE ` +
all the `WHERE` clauses we just calculated. The `%s` provide some
flexibility if we want to use these queries in other contexts.
2. Delete all existing label members
3. Do an `INSERT...SELECT` using the query we calculated from the label
criteria. The query will be `SELECT <label id> as label_id, hosts.id
FROM hosts JOIN ...`

## Future work

### Domestic vitals

These can be anything that we already store in the `hosts` table.
Domestic vitals won't add any `JOIN`s to the calculated label query, and
will simply be e.g. `hosts.hostname = ?`

### Custom vitals

We currently support an `additional_queries` config that will cause
other queries to run on hosts. The data returned from these queries is
stored in a `hosts_additional` table as a JSON blob. We can use MySQL
JSON functions to match values in this data, e.g.
`JSON_EXTRACT(host_additional, `$.some_custom_vital`) = ?`

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
> I'll add the changelog item when I add the cron job PR
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
2025-06-30 09:58:58 -05:00
Juan Fernandez
81abc49786
28224: Added missing property to hosts/identifier/:id endpoint (#30097)
For #28224 

Added missing team_name property on /api/v1/fleet/hosts/identifier/:id endpoint.
2025-06-23 13:01:33 -04:00
RachelElysia
1d2b71857b
Host API: Return empty array instead of 404 for software filter not found (#30045) 2025-06-17 09:20:09 -04:00
Lucas Manuel Rodriguez
1c5700a8c4
Microsoft Compliance Partner backend changes (#29540)
For #27042.

Ready for review, just missing integration tests that I will be writing
today.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com>
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-06-11 14:22:46 -03:00
Martin Angers
604464f60f
DCLK: Ingest login-keychain certificates for macOS (#29555) 2025-06-11 11:13:44 -04:00
Dante Catalfamo
12851f5679
Check content-type header when parsing cross-origin JSON (#29497) 2025-05-29 15:26:55 -04:00
Dante Catalfamo
3d60a28bce
Prevent user invite race condition (#29559) 2025-05-29 15:26:02 -04:00
Scott Gress
2e89780c6d
add HostIDs to label API (#29185)
for #27701

# Checklist for submitter

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Manual QA for all new/changed functionality

## Details

This PR fixes an issue where adding or removing a single host on a
manual label in the UI had unexpected results when that host's serial
number was not unique. When adding the host, all other hosts with the
same serial # would be added. When removing the host, _no_ hosts would
be removed unless _all_ the hosts with the matching serial # were
removed. The fix here is to introduce a new API param `host_ids` which
allows sending explicit Fleet host IDs to the add/update label APIs.
These are guaranteed to be unique.

## Testing

* Added new automated tests for the `NewLabel` and `ModifyLabel`
services
* Manually tested adding and modifying labels using hosts with duplicate
serials (I manually updated serials in my local db to get duplicates)

## Notes

* The existing `hosts` param is preserved (and tested) since API-only
users may rely on it.
* A separate API docs PR will be opened.
2025-05-22 08:20:35 -05:00
Konstantin Sykulev
a42167462f
Added SHA256 hash from mac apps on install paths (#29280)
https://github.com/fleetdm/fleet/issues/25545

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2025-05-20 23:38:59 -05:00
jacobshandling
b4a1042d3e
Re-calculate host failing policy and total issue counts whenever GET ing that host (#29109)
## For #27085 


![ezgif-4c995b0462ebed](https://github.com/user-attachments/assets/a1c1b2d1-c585-42c9-9db7-b45f4853e90b)


- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-05-20 13:41:38 -07:00
Martin Angers
7f5fc14f59
Immediately ask for a host refetch when a host re-enrolls and reuses an existing host row (#29081) 2025-05-14 09:38:53 -04:00
Jordan Montgomery
e514fc4881
Custom SCEP renewal (#28616)
For https://github.com/fleetdm/fleet/issues/27984

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-30 15:31:45 -04:00
Sarah Gillespie
a21f8c43a7
Update error message for VerifyMDMWindowsConfigured (#28646) 2025-04-29 14:11:31 -05:00
Victor Lyuboslavsky
df71307eeb
Added scim/details endpoint (#28007)
For #27281

This PR adds `/api/{version}/fleet/scim/details` endpoint, along with
some frontend fixes.

# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-10 14:08:45 -05:00
Victor Lyuboslavsky
3d0025c570
SCIM + host integration (#27880)
For #27284

This PR:
- Adds SCIM as a fallback for username during macOS end user
authentication during setup experience
- Adds SCIM/endUsers details to host details

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-08 09:35:06 -05:00
Dante Catalfamo
b49d131a05
Parse top-level json array when request arrives, before service func (#26665)
#24390
2025-03-10 13:27:17 -04:00
Victor Lyuboslavsky
faeb43f832
Scaffold for new integration tests. (#26747)
For #26219

Created a scaffold for server/service integration tests going forward.
- separate package
- no dependency on testify Suite
2025-03-04 12:04:25 -06:00
gillespi314
c5386b1290 Merge branch 'main' into feat-23235-host-certificates 2025-02-26 12:43:19 -06:00
Martin Angers
d903cf9081
Android: add test to verify that filtering hosts by android platform is supported (#26613) 2025-02-26 11:31:19 -05:00
Dante Catalfamo
347ab3955a
Always allow passwords for users (#26334)
For #25834
2025-02-25 16:27:58 -05:00
Martin Angers
f5cf6a3c6f
Add validation extension to the make decoder flow, validate order_key (#26567) 2025-02-25 10:42:17 -05:00
Martin Angers
8477856886 FIx conflicts 2025-02-24 14:28:34 -05:00
Martin Angers
a141830f76
CHV: implement paginated list certificates endpoints (#26554) 2025-02-24 12:52:39 -05:00
jacobshandling
c22f575150
Add gitops app config fields, API access, activities, tests (#26282)
## For #26230 

- Add `gitops` settings to app config
- GET and PATCH endpoint functionality to retrieve and modify these
settings
- generate activities for enabling and disabling GitOps mode
- Premium only
- Update tests

<img width="1355" alt="Screenshot 2025-02-17 at 9 22 13 AM"
src="https://github.com/user-attachments/assets/e03d7fd6-8795-4df2-9c8f-ffbcabd2d212"
/>

<img width="1575" alt="Screenshot 2025-02-17 at 9 24 33 AM"
src="https://github.com/user-attachments/assets/5bf963d4-6fb7-4586-8f6f-9e605c25bf06"
/>

## Checklist for submitter

- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-02-20 08:40:46 -08:00
Victor Lyuboslavsky
4b007e20e1
Android scaffold (#26274)
Android scaffold code and refactorings
- Android packages intended to be decoupled from other Fleet code

Video explaining the PR: https://www.youtube.com/watch?v=cza-35Z9Wxk

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-13 14:32:19 -06:00
Martin Angers
606df3f349
Upcoming Activities feature branch (#25450) 2025-02-11 14:53:11 -05:00
jacobshandling
56b0f0bd61
Add missing condition for disable SSO error state, update integration test to cover the situation (#25960)
## Follow-up for https://github.com/fleetdm/fleet/pull/25843


- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-02-03 11:36:45 -08:00
jacobshandling
8d9ca0eabf
Require a password when changing a user from SSO to password-based authentication (#25843)
## For #24754

- Backend:
- Return an error when a PATCH attempts to update a user's
authentication from SSO to password but doesn't include a password
  - Add checks to integration test.
- Frontend:
- Form error when attempting to switch a user who is currently
SSO-authed to password without a password
- Refactor upstream inherited errors to allow for disabling the form
submission button when errors are present
  - Other improvements to user form validation

**[UI
Demo](https://drive.google.com/file/d/1-BIzCpqu0zjYHf7zxiZL_7kVoE2sLwtx/view?usp=sharing)**
**[API
Demo](https://drive.google.com/file/d/19lQ7Pvfmq3MwEjHw0_r9IoxVuNaSNwGb/view?usp=sharing)**

<img width="994" alt="Screenshot 2025-01-28 at 3 38 11 PM"
src="https://github.com/user-attachments/assets/304f8def-2656-43f7-97e5-8be1fc679814"
/>

<img width="660" alt="Screenshot 2025-01-28 at 3 39 41 PM"
src="https://github.com/user-attachments/assets/77283520-b313-4743-96df-06c55e573496"
/>


- [x] Changes file added for user-visible changes in `changes/`
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-01-30 09:32:11 -08:00
jacobshandling
f93b869f26
Update label membership by host IDs directly (#25687)
## For #25261 

<img width="826" alt="Screenshot 2025-01-23 at 11 07 19 AM"
src="https://github.com/user-attachments/assets/3a2f5d75-c0bf-445a-80dc-976914ff434e"
/>

### [Demo
video](https://drive.google.com/file/d/1ZFcrizkZ6zNODnTXjRC1f-Oeght5zOP4/view?usp=sharing)
- [x] Changes file added for user-visible changes in `changes/`, 
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-01-23 12:38:57 -08:00
Dante Catalfamo
347c65b5c6
Add options to populate users and labels on list hosts endpoint (#25621)
#22464

---------

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
2025-01-22 11:17:26 -05:00
Dante Catalfamo
39466cb644
Use webhooks settings from gitops even when empty (#25347)
#24958

---------

Co-authored-by: Scott Gress <scottmgress@gmail.com>
2025-01-15 11:31:48 -05:00