Commit graph

448 commits

Author SHA1 Message Date
Lucas Manuel Rodriguez
e44f454158
Reorder migrations for cherry-pick into v4.69.0 (#29927)
Re-ordering migrations for cherry-picking #27042 into v4.69.0.

Migrations in `main`:
```
[...]
20250519161614_AddExecutableSha256HostSoftwareInstalledPaths.go
20250520153848_AddCancelledStatusToSetupExperienceStatusResults.go
20250528115932_AddInviteIDToUsersTable.go
20250529102706_SetMediumTextColumns.go
20250603105558_AlterLegacyEnrollRefsUUIDSize.go
20250609102714_AddHostCertificateSourcesTable.go
20250609120000_MicrosoftCompliancePartner.go
````

Migrations in rc-minor-fleet-v4.69.0:
```
[...]
20250519161614_AddExecutableSha256HostSoftwareInstalledPaths.go
```

So we have to move this one between
`20250519161614_AddExecutableSha256HostSoftwareInstalledPaths.go` and
`20250520153848_AddCancelledStatusToSetupExperienceStatusResults.go`.
2025-06-11 15:05:00 -03:00
Lucas Manuel Rodriguez
1c5700a8c4
Microsoft Compliance Partner backend changes (#29540)
For #27042.

Ready for review, just missing integration tests that I will be writing
today.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com>
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-06-11 14:22:46 -03:00
Martin Angers
604464f60f
DCLK: Ingest login-keychain certificates for macOS (#29555) 2025-06-11 11:13:44 -04:00
Martin Angers
33454fc6e9
Bugfix: wiped ADE-enrolled iOS device remains wiped on re-enroll (#29715) 2025-06-09 10:18:03 -04:00
Jahziel Villasana-Espinoza
9d2b07f76f
add a test that checks collation on new migrations (#29309)
> closes #26403

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-05-29 17:00:30 -04:00
Dante Catalfamo
3d60a28bce
Prevent user invite race condition (#29559) 2025-05-29 15:26:02 -04:00
Martin Angers
aebf0bfc07
DCLK: Add source column to host_certificates (#29528) 2025-05-28 11:42:11 -04:00
Jordan Montgomery
612d3de968
Mark setup experience installs as "cancelled" and later fail them when certain bulk actions happen (#29355)
Still adding tests but wanted to get this up for review of the overall
"shape" of the fix

When certain things happen like installer updates we delete pending
upcoming_activities(UA) and host_software_install(HSI) entries and need
to mark setup_experience_status_results(SESR) cancelled. When this
happens if that UA/HSI are being depended on by setup experience we need
to make sure that that setup experience result eventually gets marked
failed.

I kind of went back and forth a few times on how best to do this and
avoid race conditions. One thing I tried was looking at existence of the
UA/HSI but found that naively just trying to look at that in relation to
the SESR entry seemed to have a few race conditions that were hard to
resolve. There are a few possible states here we need to account for
such as:

un-activated, totally not yet running software install cancelled
activated but not yet running on the host software install cancelled
activated and running on the host software install cancelled before
results are completely reported back

What I eventually came around to was essentially that we want to mark
the SESR cancelled in the same transaction we delete the HSI/UA in. We
then finalize it by marking it failed and sending the activity the next
time the host fetches setupm experience results. The new cancelled
status never leaves fleet. This is a bit ugly but in my testing avoided
the race conditions and works well.

Note that to actually avoid setup experience hanging entirely we still
need to fix #29357 which encompasses several cases where the unified
queue can get completely stuck for a host

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
- [ ] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-27 16:52:51 -04:00
Konstantin Sykulev
a42167462f
Added SHA256 hash from mac apps on install paths (#29280)
https://github.com/fleetdm/fleet/issues/25545

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2025-05-20 23:38:59 -05:00
Sarah Gillespie
aea4406b4f
Improve MDM device-to-user mapping for Apple devices (#29239) 2025-05-19 13:29:46 -05:00
Victor Lyuboslavsky
bfad93a1f0
Fixing issues with Apple DDM profile status (#29059)
For #27979 

This PR fixes Apple declarations issues:
- P2 issue with hashing the declaration token
- When declaration items are requested, mark any outstanding "remove"
operations as pending. This prevents "remove" operations from being
stuck in pending in some cases because they were actually already
processed.
- When updating verification status, don't update "remove" operations --
we don't update their status and we just delete them. This prevents the
issue where a "remove" operation got the updated status and the
"install" operation got stuck in verifying forever.
- when adding a declaration that has a matching remove outstanding, mark
the declaration verified. This prevents "install" operations from being
stuck in pending/verifying. Why? Because there is nothing for the host
to do if the same declaration was removed and then immediately added
back.
- migration to delete "remove" operations with non-nil and non-pending
status. These are the only legal statuses for remove operations.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-15 13:05:25 -05:00
Jordan Montgomery
cb29a5d751
Add timestamp to host_mdm_apple_profiles for variable update times (#29074)
For #28970 

Adds a timestamp to the host-apple profiles table so that we know when
the profile should have been installed or reinstalled based on the last
date the variables were updated. This overrides the usual date which is
just based on profile upload time. It's needed because of how
variable-based and specifically CA profiles are used to renew
certificates - the checksum never changes but we need to verify that the
profile was updated after the expected time.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-13 17:22:27 -04:00
Victor Lyuboslavsky
0ae58c02ca
Fixes and optimizations for Entra ID (#28938)
For #28196

This PR depends on PR https://github.com/fleetdm/fleet/pull/28832

This PR adds support for excludeAttributes=members, which is being used
by Microsoft Entra ID.

This PR modifies the primary key of host_scim_user table to be host_id.
This should have been done initially and has added accidental complexity
and maintainability challenges, so we are doing it now. This means a
host can have a maximum of 1 SCIM user associated with it. A SCIM user,
on the other hand, can be associated with multiple hosts.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-08 13:46:16 -05:00
Jahziel Villasana-Espinoza
db5444d6cd
software categories: backend (#28479)
> For #28138 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-05-02 11:41:26 -04:00
Martin Angers
95b80482ba
IdP vars: store profile-variable associations when new profile is created/updated (#28645) 2025-04-30 16:03:23 -04:00
Jordan Montgomery
e514fc4881
Custom SCEP renewal (#28616)
For https://github.com/fleetdm/fleet/issues/27984

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-30 15:31:45 -04:00
Dante Catalfamo
1ab7bdc923
Bulk script execution backend (#28299)
#28158
2025-04-30 12:54:46 -04:00
Martin Angers
2673b83dee
IdP: resend profiles when IdP values change (#28436) 2025-04-29 14:35:37 -04:00
Victor Lyuboslavsky
510a9bbc44
Added macos_setup.manual_agent_install to global/team config (#28419)
For #26071 and #26089

Added `macos_setup.manual_agent_install` boolean option. No validation
(pushed to another story due to complications caused by bug #28497)

Tests are failing due to vulnerability issues that are not related to
this feature. All tests were passing earlier.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-24 15:18:01 -05:00
Jordan Montgomery
862739292e
Renewal of DigiCert certificates on macOS (#28449)
Adds renewal of Digicert certificates:
https://github.com/fleetdm/fleet/issues/26553 . Does not attempt to
renew custom SCEP or NDES. Also we aren't actually calling the DigiCert
renewal endpoint at this time because we don't believe we need to and we
can't necessarily do that as we weren't previously storing the serial
number however this change adds storage of the serial number.


# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
2025-04-24 08:35:15 -04:00
Jahziel Villasana-Espinoza
fa8c087abf
fix: change how macOS software names are calculated to avoid erroneous duplicates (#28037)
> For #24087 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-11 19:19:07 -04:00
Victor Lyuboslavsky
df71307eeb
Added scim/details endpoint (#28007)
For #27281

This PR adds `/api/{version}/fleet/scim/details` endpoint, along with
some frontend fixes.

# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-10 14:08:45 -05:00
Sarah Gillespie
63a295151a
Update migration numbering to account for cherry-pick in v4.66 (#27901)
For #27900
2025-04-04 14:29:05 -05:00
Dante Catalfamo
37932e0f97
Custom targets for policies - backend (#27575)
#27277
2025-04-02 12:36:03 -04:00
Martin Angers
69fcda9686
Cancel upcoming activities: DB schema and backend (#27710) 2025-04-01 14:08:56 -04:00
Victor Lyuboslavsky
2198fd8d65
Add SCIM Users (#27551)
For #27287

Video explaining the PR: https://www.youtube.com/watch?v=ZHgFUAvrPEI

This PR adds SCIM Users support for Okta. The goal is to first add
Users/Groups support so that the remaining backend SCIM work can be done
in parallel.

This PR does not include the following, which will be added in later PRs
- Changes file
- Groups support for Okta
- Full support for Entra ID
- Integration tests

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-04-01 11:02:24 -05:00
Sarah Gillespie
8d17956f7b
Skip bootstrap package and other setup items when renewing Apple MDM enrollment profiles (#27560) 2025-03-28 16:33:22 -05:00
Martin Angers
78c0d67e78
Bugfix: re-create deleted iOS/iPadOS host entries in Fleet if it checks in again via MDM (#27231) 2025-03-26 09:33:38 -04:00
Ian Littman
4a1e5340f0
Switch Fleet-maintained apps to use manifest-based structure (#27201)
For #26082.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-20 21:21:56 -05:00
Scott Gress
7eebc22693
Add author ID to labels (#27055)
For #27035 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

## Details

This PR adds an `author_id` column to the `labels` table, and adds the
associated properties to the `Label` and `LabelSpec` types. When a new
label is created via the UI or API, an author ID is set on the label if
one can be inferred from the context. Otherwise, the author ID is set to
`null`.

## Authz and Automated testing

Additional backend authorization logic is introduced in a follow-on PR,
https://github.com/fleetdm/fleet/pull/27089, because rconciling all of
the test updates between this PR and
https://github.com/fleetdm/fleet/pull/27038 was getting complicated.

## Manual Testing

* Tested in the UI by creating a new label on the Hosts page
* Tested via Gitops by merging this branch with
https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops`
with a global config with `labels:` in it.
2025-03-20 16:05:16 -05:00
Victor Lyuboslavsky
93bdb437ac
Resend Windows profiles on change (#27308)
For #25030 

This PR includes the bug fix and tests.

It also includes the `secrets_updated_at` columns needed for story
#27351. These columns are currently unused and always NULL.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-20 14:43:04 -05:00
Victor Lyuboslavsky
6b7d232522
Additional CA validation (#27169)
For #26623

- Updated `github.com/groob/plist` to `github.com/micromdm/plist` -- it
was renamed
- Added validation that restricts DigiCert Fleet variables to
`com.apple.security.pkcs12` payloads plus additional restrictions
- Added validation that restricts Custom SCEP Fleet variables to
`com.apple.security.scep` payloads plus additional restrictions
- Enabled multiple CAs (Fleet variables) to be present in an Apple MDM
profile. But each CA can only be used once. For example, we can have
DigiCert CA and Custom SCEP CA in one Apple profile.

# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-19 08:27:55 -05:00
Victor Lyuboslavsky
131a52695b
Custom SCEP integration (#27121)
For #26623 

This PR enables deploying an Apple configuration profile with Fleet
proxying a custom SCEP server.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-03-17 10:59:07 -05:00
Dante Catalfamo
d5e9153cea
Add LabelsIncludeAny to queries endpoint (#26823)
#16413 

---------

Co-authored-by: Scott Gress <scottmgress@gmail.com>
2025-03-11 09:45:01 -04:00
Victor Lyuboslavsky
9deabe8f70
DigiCert config (encrypt API token) (#26858)
For #26603

This PR includes:

- DB migration for DigiCert
- new datastore methods and tests
- saving/deleting/updating of encrypted DigiCert API tokens
- Integration test for DigiCert configs

This PR does not include the following:

- Making an HTTP call to DigiCert API for validation (in later PR)
- Changes file (in later PR)

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-03-06 12:47:57 -06:00
Victor Lyuboslavsky
b976956b24
Add DigiCert configs. (#26780)
For #26603

This PR includes:
- DigiCert configs
- Some config code for custom SCEP, but not fully functional
- Validation tests for DigiCert Configs
- DigiCert activities (manually tested)

This PR does not include the following:
- Encryption of DigiCert API tokens -- this requires DB migration (in
next PR)
- Making an HTTP call to DigiCert API for validation (in later PR)
- Integration tests (in later PR)
- Changes file (in later PR)

# Checklist for submitter

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-03-05 15:02:26 -06:00
Victor Lyuboslavsky
67b72764c5
Added Android activity and better handling of deleted users. (#26640)
For #26218

- Added `users_deleted` table to track user actions if the user was
actually deleted.
- Added enable/disable Android MDM activities

Note: I could not auto-generate fleet.Service mock because it has issues
with methods that don't return anything. I ended up using testify mock
instead.

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-27 14:19:15 -06:00
gillespi314
b2bf148b65 Update migration timestamp 2025-02-26 12:49:43 -06:00
gillespi314
c5386b1290 Merge branch 'main' into feat-23235-host-certificates 2025-02-26 12:43:19 -06:00
Martin Angers
d249aa888b
Android: add 'Android' builtin label, add new Android hosts to 'All hosts' and this label (#26585) 2025-02-25 14:44:48 -05:00
Martin Angers
0adf67e538 Fix conflicts 2025-02-25 14:39:35 -05:00
Jahziel Villasana-Espinoza
dfc58d9600
fix: add index to os cve column (#26576)
> for #26178

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-02-24 21:59:56 -05:00
Victor Lyuboslavsky
bf7917b503
Basic Android enroll functionality (#26386)
For #26219 

Video explaining the PR: https://www.youtube.com/watch?v=t2hmpE7N-50

This PR includes:
- applying a default/dummy policy to Android enterprise
- Setting up PubSub topic and subscription to point at Fleet Server
- Generating device enrollment token
- Enrolling a device

Although some tests were added, this PR is mostly manually tested.
Integration tests are an outstanding task.

# Checklist for submitter

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-24 14:31:21 -06:00
Martin Angers
8477856886 FIx conflicts 2025-02-24 14:28:34 -05:00
Ian Littman
f6f540b74e
Add created_at/updated_at timestamps to VPP apps teams table, return as added_at (#26442)
For #23744.

Splitting into another PR: query to pull more accurate timestamps from
activity feed

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-02-21 15:07:52 -06:00
jacobshandling
c22f575150
Add gitops app config fields, API access, activities, tests (#26282)
## For #26230 

- Add `gitops` settings to app config
- GET and PATCH endpoint functionality to retrieve and modify these
settings
- generate activities for enabling and disabling GitOps mode
- Premium only
- Update tests

<img width="1355" alt="Screenshot 2025-02-17 at 9 22 13 AM"
src="https://github.com/user-attachments/assets/e03d7fd6-8795-4df2-9c8f-ffbcabd2d212"
/>

<img width="1575" alt="Screenshot 2025-02-17 at 9 24 33 AM"
src="https://github.com/user-attachments/assets/5bf963d4-6fb7-4586-8f6f-9e605c25bf06"
/>

## Checklist for submitter

- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-02-20 08:40:46 -08:00
Martin Angers
6ed89bd099
Unified Queue: add fleet initiated flag to past activities (#26451) 2025-02-19 14:55:20 -05:00
Martin Angers
6049b3919f
Unified Queue: add DB migration for existing pending activities (#26413) 2025-02-19 08:29:14 -05:00
Sarah Gillespie
7b6e212003
Add datastore methods for host certificates feature (#26416) 2025-02-18 17:49:02 -06:00
Victor Lyuboslavsky
fc33df1981
Basic Android MDM on/off functionality (#26309)
For #26218 

Basic Android MDM on/off backend functionality. Manually tested.

The following env vars must be set:
```
FLEET_DEV_ANDROID_ENABLED=1
FLEET_DEV_ANDROID_SERVICE_CREDENTIALS=$(cat credentials.json)
FLEET_DEV_ANDROID_PUBSUB_TOPIC=projects/your-project/topics/your-topic
```

I picked https://github.com/go-json-experiment/json as the JSON library,
which seems like the safest option.
- will become json/v2 at some point
- currently used in production by other companies, like Tailscale
- well-maintained
- Some context here: https://github.com/fleetdm/fleet/issues/25512

Plan for next work:
- refactoring from 1st PR
- add pubsub with device enroll -> spec proxy for fleetdm.com
- come back to this sub-task to add tests and finish TODOs

# Checklist for submitter

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-02-18 09:43:11 -06:00