Commit graph

3504 commits

Author SHA1 Message Date
Luke Heath
7380919dc3
Organize contributor docs and establish ADR process and template (#29101) 2025-05-17 15:03:52 -05:00
Ian Littman
21006a1bd7
Batch host_software inserts in macOS names migration to improve performance for large host counts (#29238)
For https://github.com/fleetdm/confidential/issues/10596

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2025-05-17 12:42:01 -05:00
Jordan Montgomery
38811da1c0
Hold off on policy queries until after setup experience (#29159)
For #28205 

During setup experience customers often install all or most of the
software that would otherwise be installed based on the results of
policy queries. If we run policy queries during setup experience we end
up trying to install some software twice which, at best, leads to
confusing activities listed for the host. With these changes we will not
run policy queries on macOS hosts until after the host has exited setup
experience, at which point we should be able to avoid duplicate installs

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-05-16 14:56:27 -04:00
Gabriel Hernandez
9a32be0540
enable fleet secret variables in the macos setup script in gitops (#29164)
For #28215

Allows users to use fleet secret variables for macos setup script for
gitops.


- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-05-16 19:05:33 +01:00
Jahziel Villasana-Espinoza
8aa268b1b5
prevent panic if installer is removed from s3 (#29189)
> for #28815

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-16 11:27:13 -04:00
Victor Lyuboslavsky
bfad93a1f0
Fixing issues with Apple DDM profile status (#29059)
For #27979 

This PR fixes Apple declarations issues:
- P2 issue with hashing the declaration token
- When declaration items are requested, mark any outstanding "remove"
operations as pending. This prevents "remove" operations from being
stuck in pending in some cases because they were actually already
processed.
- When updating verification status, don't update "remove" operations --
we don't update their status and we just delete them. This prevents the
issue where a "remove" operation got the updated status and the
"install" operation got stuck in verifying forever.
- when adding a declaration that has a matching remove outstanding, mark
the declaration verified. This prevents "install" operations from being
stuck in pending/verifying. Why? Because there is nothing for the host
to do if the same declaration was removed and then immediately added
back.
- migration to delete "remove" operations with non-nil and non-pending
status. These are the only legal statuses for remove operations.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-15 13:05:25 -05:00
Victor Lyuboslavsky
890042d27a
Force enrollment profile sync when an Apple device was added to ABM. (#29147)
For #27854

I was able to reproduce the issue by simply unassigning device from an
MDM server, and then assigning back. Once assigned back, Fleet did not
resend the profile to ABM, and device was not able to enroll into MDM.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-05-15 12:29:38 -05:00
Ian Littman
5ace1e621d
Skip vulnerability checks on Docker DX VSCode plugin to avoid false positives due to overly broad Docker CPEs (#29156)
For #28983. No changes file since this is being released directly to the
vulns feed.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-05-15 09:16:33 -05:00
Gabriel Hernandez
63b2e19630
allow turning off mdm for iphone and ipad hosts (#29087)
For [#23784](https://github.com/fleetdm/fleet/issues/23784)

This adds the "turn off mdm" option don't he host details page for
iPhone and iPad devices.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [ ] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
2025-05-15 12:38:07 +01:00
Martin Angers
7f5fc14f59
Immediately ask for a host refetch when a host re-enrolls and reuses an existing host row (#29081) 2025-05-14 09:38:53 -04:00
Juan Fernandez
3f298ac218
Error when deleting non managed GitOps labels #28440 (#29067)
Fixes #28440 

When running GitOps, delete any non-managed labels as the last step to avoid any DB constraint issues.
2025-05-13 20:16:16 -04:00
Konstantin Sykulev
1797120639
Non fleet installed software properly filtered by labels (#29116)
https://github.com/fleetdm/fleet/issues/28328

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-13 18:41:33 -05:00
Jordan Montgomery
cb29a5d751
Add timestamp to host_mdm_apple_profiles for variable update times (#29074)
For #28970 

Adds a timestamp to the host-apple profiles table so that we know when
the profile should have been installed or reinstalled based on the last
date the variables were updated. This overrides the usual date which is
just based on profile upload time. It's needed because of how
variable-based and specifically CA profiles are used to renew
certificates - the checksum never changes but we need to verify that the
profile was updated after the expected time.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-13 17:22:27 -04:00
Konstantin Sykulev
13cd8386e4
Mdm turn off, not install vpp apps are not available for install (#29068)
https://github.com/fleetdm/fleet/issues/29004

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-13 12:35:50 -05:00
Ian Littman
3edf684db1
Add backend for uninstalls in My device UI (#29035)
For #28846. Intentionally not limited to self-service/in-scope apps,
though we don't have any software listing changes in this PR to show
more titles in the self-service list.

QA plan is a bit light due to ticket being underspec'd. Can figure out
how we deal with that later.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-13 12:14:45 -05:00
Scott Gress
ecb1a1caca
Add retry when applying queries (#28951)
for #28642

> Note: this PR diff is easier to view [with whitespace
off](https://github.com/fleetdm/fleet/pull/28951/files?w=1).

## Details

This PR adds retry to the "Apply Queries" logic, in an attempt to
alleviate deadlock issues when applying queries via GitOps. The
`applyQueriesInTx` now uses `withRetryTxx` instead of starting a
transaction with `BeginTxx`. This requires some downstream updates to
`updateQueryLabels` and `updateQueryLabelsInTx`, see PR comments for
details.

This is a first (and hopefully only necessary) step to fixing the
deadlock issues. If needed, we have other steps we can take like
batching the query inserts and splitting the read/write in
saveHostPackStatsDB
(see
https://github.com/fleetdm/fleet/issues/28642#issuecomment-2845804689)

## Testing

I tested this manually using `fleetctl gitops` to apply queries with and
without labels. Existing automated tests for Apply Queries still pass.
2025-05-13 12:06:20 -05:00
Konstantin Sykulev
e134691da3
Handled installed vpp apps when mdm is turned off (#29063)
https://github.com/fleetdm/fleet/issues/28985

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-05-13 09:09:39 -05:00
Martin Angers
c96ccb7774
BRP: implement API endpoint for host status summary of single profile (#29039) 2025-05-13 08:49:08 -04:00
Jordan Montgomery
db3e3ff70b
Fix MDM last checkin and enrollment names to match API spec (#29073)
For #17710 

I focused too much on making sure we were returning the requested data
and got the actual property names wrong.

See https://github.com/fleetdm/fleet/pull/28940/files for proper names

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-05-13 08:17:11 -04:00
Konstantin Sykulev
ec9f6c3947
Handled fleet installed & inventoried vpp app versions (#29051)
https://github.com/fleetdm/fleet/issues/28985

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-12 22:20:33 -05:00
Katheryn Satterlee
9ea5ecde68
Add neon to Linux platform list (#28977)
Added `neon` to list of Linux platforms associated with hosts so that
Linux-specific detail queries and policies will be sent to hosts running
the XDE Neon operating system.

This does not guarantee full compatibility with Neon, but will improve
telemetry.

Resolves #28560 


# Checklist for submitter


If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [ ] Manual QA for all new/changed functionality
2025-05-12 17:37:21 -05:00
jacobshandling
2f67a87e53
API, Environment variable for bespoke functionality for customer-preston (#28893)
## For #28221

- Add support for env var / flag
- surface as `config.partnerships.enable_primo` if `true`, omit from
response otherwise

### when `FLEET_PARTNERSHIPS_ENABLE_PRIMO=true`:
![Screenshot 2025-05-06 at 6 14
01 PM](https://github.com/user-attachments/assets/f7a0c705-37e7-4491-ad8b-492f456ca924)

### when `FLEET_PARTNERSHIPS_ENABLE_PRIMO=false` or not set:
![Screenshot 2025-05-06 at 6 13
57 PM](https://github.com/user-attachments/assets/e8a88b15-43ef-4d56-ab6c-3fbe832b6ddd)


~Changes file added for user-visible changes in `changes/`~ in companion
frontend PR
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-05-12 13:04:18 -07:00
Ian Littman
4b9774d49e
Move fleet_maintained_app_id from title level to package level in software title list endpoint (#29033)
For #29032.

- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-12 12:37:09 -05:00
Konstantin Sykulev
6ec2506e5f
tests for vpp with app self service (#29029)
https://github.com/fleetdm/fleet/issues/28919

Looks like either #29007 or #28991 fixed this issue. But adding a test
to ensure

- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-12 09:44:05 -05:00
Konstantin Sykulev
e374972133
Ensured vpp apps return installed version if exists (#29007)
https://github.com/fleetdm/fleet/issues/28985

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-09 15:11:27 -05:00
Konstantin Sykulev
4d754aacbb
Fixed search with self service filter (#28991)
For https://github.com/fleetdm/fleet/issues/28880

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-09 14:52:11 -05:00
Jordan Montgomery
ca149a0cdb
Add host apple MDM timestamps to host detail responses (#28998)
For #17710 

Adds mdm_last_seen_at and mdm_last_enrolled_at to the host details
response for Apple platforms

Still testing with actual hardware to make sure the timestamps update
when expected

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
2025-05-09 14:18:48 -04:00
Victor Lyuboslavsky
3af36b9505
Updated SCEP error message per Figma. (#29003)
For NDES and Custom SCEP profile validation.

# Checklist for submitter

- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual testing
2025-05-09 12:10:27 -05:00
Gabriel Hernandez
340d63e0f5
fix issue with gitops not removing custom profiles when they were removed from default.yml file (#28904)
For [#27249](https://github.com/fleetdm/fleet/issues/27249)

fix issue where we custom profiles were not removed when they were
removed from default.yml and running gitops.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
2025-05-09 16:11:38 +01:00
Victor Lyuboslavsky
023be30cef
Final fixes after testing with Entra ID. (#28987)
For #28196 

Demo video of the full feature: https://youtu.be/7PM41LBsnig

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-09 09:27:23 -05:00
Victor Lyuboslavsky
0ae58c02ca
Fixes and optimizations for Entra ID (#28938)
For #28196

This PR depends on PR https://github.com/fleetdm/fleet/pull/28832

This PR adds support for excludeAttributes=members, which is being used
by Microsoft Entra ID.

This PR modifies the primary key of host_scim_user table to be host_id.
This should have been done initially and has added accidental complexity
and maintainability challenges, so we are doing it now. This means a
host can have a maximum of 1 SCIM user associated with it. A SCIM user,
on the other hand, can be associated with multiple hosts.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-08 13:46:16 -05:00
Victor Lyuboslavsky
6f9030ee3c
SCIM Entra ID support (#28832)
For #28196

This PR adds full patching for SCIM Users and Groups, and adds the
ability to filter Groups by displayName.

The changes have been tested with [Entra ID SCIM
Validator](67dfd91c0c/docs/Contributing/SCIM-integration.md (entra-id-integration))
and Okta SCIM 2.0 SPEC Test (to make sure we didn't break Okta).

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-08 13:02:49 -05:00
Jahziel Villasana-Espinoza
5e9eb4d384
fma gitops categories (#28943)
> For #28855

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
2025-05-07 21:12:12 -04:00
Konstantin Sykulev
a3ae1a6f91
Support for fleet maintained apps in gitops (#28751)
https://github.com/fleetdm/fleet/issues/24469

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-07 18:16:08 -05:00
Martin Angers
48de857dca
BRP: add batch-resend profile to hosts endpoint based on status (#28871) 2025-05-07 16:48:18 -04:00
Jahziel Villasana-Espinoza
5a8da97c5d
feat: software categories in gitops (#28863)
> For #28140

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-07 11:19:33 -04:00
Scott Gress
d716265641
Add "generate-gitops" command (#28555)
For #27476

# Checklist for submitter

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)

# Details

This PR adds a new command `generate-gitops` to the `fleetctl` tool. The
purpose of this command is to output GitOps-ready files that can then be
used with `fleetctl-gitops`.

The general usage of the command is:

```
fleectl generate-gitops --dir /path/to/dir/to/add/files/to
```

By default, the outputted files will not contain sensitive data, but
will instead add comments where the data needs to be replaced by a user.
In cases where sensitive data is redacted, the tool outputs warnings to
the user indicating which keys need to be updated.

The tool uses existing APIs to gather data for use in generating
configuration files. In some cases new API client methods needed to be
added to support the tool:

* ListConfigurationProfiles
* GetProfileContents
* GetScriptContents
* GetSoftwareTitleByID

Additionally, the response for the /api/latest/fleet/software/batch
endpoint was updated slightly to return `HashSHA256` for the software
installers. This allows policies that automatically install software to
refer to that software by hash.

Other options that we may or may not choose to document at this time:

* `--insecure`: outputs sensitive data in plaintext instead of leaving
comments
* `--print`: prints the output to stdout instead of writing files
* `--key`: outputs the value at a keypath to stdout, e.g. `--key
agent_options.config`
* `--team`: only generates config for the specified team name
* `--force`: overwrites files in the given directory (defaults to false,
which errors if the dir is not empty)

# Technical notes

The command is implemented using a `GenerateGitopsCommand` type which
holds some state (like a list of software and scripts encountered) as
well as a Fleet client instance (which may be a mock instance for tests)
and the CLI context (containing things like flags and output writers).
The actual "action" of the CLI command calls the `Run()` method of the
`GenerateGitopsCommand` var, which delegates most of the work to other
methods like `generateOrgSettings()`, `generateControls()`, etc.

Wherever possible, the subroutines use reflection to translate Go struct
fields into JSON property names. This guarantees that the correct keys
are written to config files, and protects against the unlikely event of
keys changing.

When sensitive data is encountered, the subroutines call `AddComment()`
to get a new token to add to the config files. These tokens are replaced
with comments like `# TODO - Add your enrollment secrets here` in the
final output.

# Known issues / TODOs:

* The `macos_setup` configuration is not output by this tool yet. More
planning is required for this. In the meantime, if the tool detects that
`macos_setup` is configured on the server, it outputs a key with an
invalid value and prints a warning to the user that they'll need to
configure it themselves.
* `yara_rules` are not output yet. The tool adds a warning that if you
have Yara rules (which you can only upload via GitOps right now) that
you'll have to migrate them manually. Supporting this will require a new
API that we'll have to discuss the authz for, so punting on it for now.
* Fleet maintained apps are not supported by GitOps yet (coming in
https://github.com/fleetdm/fleet/issues/24469). In the meantime, this
tool will output a `fleet_maintained_apps` key and trigger a warning,
and GitOps will fail if that key is present.

---------

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2025-05-06 15:25:44 -05:00
Konstantin Sykulev
b2c1ddaafa
Rewrite label scoping logic for listing host software, fix other ListHostSoftware bugs (#28592)
https://github.com/fleetdm/fleet/issues/28328

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-05-06 12:32:35 -05:00
Victor Lyuboslavsky
8ad9da71d6
Only allow distribution packages for bootstrap package (#28787)
For #27700 
When uploading bootstrap package for macOS setup experience, validate
that it is a Distribution package since that is required by Apple's
InstallEnterpriseApplication MDM command.


# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-06 11:02:13 -05:00
Martin Angers
4c9b955343
Bugfix: resend only affected profile with vars on a team (#28844) 2025-05-06 10:02:23 -04:00
Tim Lee
14967172a7
Downgrade sotware paths error (#28736) 2025-05-05 14:47:59 -06:00
Martin Angers
672ea70876
IdP vars: add integration test, address last PR comments (#28794) 2025-05-05 11:46:10 -04:00
Jahziel Villasana-Espinoza
db5444d6cd
software categories: backend (#28479)
> For #28138 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-05-02 11:41:26 -04:00
Ian Littman
99d03a61de
Add .tar.gz package support (#28339)
For #26692.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

Changes file included in FE PR.

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2025-05-02 09:07:35 -05:00
Dante Catalfamo
98e92aa9b4
Webhook logging destination (#28692)
#27445
2025-05-01 16:13:04 -04:00
jacobshandling
1f1ef3f8ef
Update Google Calendar event bodies and relevant previews in the Fleet UI (#28715)
## For #27458 

- Update Calendar events modal:
  - not-configured preview image
  - preview modal
- Update Google calendar event body

### In Google Calendar:
<img width="453" alt="Screenshot 2025-04-29 at 3 48 38 PM"
src="https://github.com/user-attachments/assets/6f7a7486-ab8d-448c-8e12-3ab9ac32b5ac"
/>

### In Fleet UI:
<img width="736" alt="Screenshot 2025-04-30 at 4 03 28 PM"
src="https://github.com/user-attachments/assets/5850f062-3ae9-4523-9c02-e2c52c3586c0"
/>


<img width="736" alt="Screenshot 2025-04-30 at 4 04 57 PM"
src="https://github.com/user-attachments/assets/6f6104a1-b3e7-4d40-8af0-5c264f93f2dc"
/>


- [x] Changes file added for user-visible changes in `changes/`, 
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-05-01 13:12:53 -07:00
Victor Lyuboslavsky
bef1506bde
NDES variable validation (#28734)
For #24878

Added validation per Figma.
Some validation questions may be outstanding in Slack. They can be
addressed in a subsequent PR.

# Checklist for submitter
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-01 14:40:06 -05:00
Dante Catalfamo
1815440a93
Add query IDs to query automation logs (#28570)
#27436

The `query_id` field is only added for responses where we have the query
in the database, to prevent unnecessary remarshaling during ingestion.
2025-05-01 15:21:30 -04:00
Victor Lyuboslavsky
d0d65b6dec
NDES cert renewal (#28712)
For #24880 

This includes a logic to gather the expiration date of managed NDES
certs and renewal of these certs. This PR includes some validation logic
(needed to not interfere with custom SCEP validation). The rest of
validation will be implemented as part of #24878

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
2025-05-01 12:16:45 -05:00
Jahziel Villasana-Espinoza
3c07f50cb9
prevent exe from being added to another team when installer already exists and no scripts provided (#28718)
> For #28558, part 2

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
2025-05-01 10:47:48 -04:00