<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40122
# Details
* Adds deprecation warnings to `fleetctl apply`
* Adds alias conflict errors (i.e. using both new and deprecated keys in
the same spec) to `fleetctl apply`
* Adds logic around all deprecated field warnings to check the topic
first
* Disables deprecation warnings by default for `fleet serve`, `fleetctl
gitops` and `fleetctl apply`
* Enables deprecation warnings for dogfood via env var
To turn on warnings:
* In `fleet serve`, use either
`--logging_enable_topics=deprecated-field-names` or the
`FLEET_LOGGING_ENABLE_TOPICS=deprecated-field-names` env var
* In `fleetctl gitops` / `fleetctl apply` use either
`--enable-log-topics=deprecated-field-names` or
`FLEET_ENABLE_LOG_TOPICS=deprecated-field-names`
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
tested in `fleetctl apply`, `fleet serve` and `fleet gitops` that
warnings are suppressed by default and added when the appropriate env
var or CLI option is used
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#40124
# Details
Implements the proposal in
https://docs.google.com/document/d/16qe6oVLKK25nA9GEIPR9Gw_IJ342_wlJRdnWEMmWdas/edit?tab=t.0#heading=h.nlw4agv1xs3g
Allows doing e.g.
```go
logger.WarnContext(logCtx, "The `team_id` param is deprecated, use `fleet_id` instead", "log_topic", "deprecated-field-names")
```
or
```go
if logging.TopicEnabled("deprecated-api-params") {
logging.WithLevel(ctx, slog.LevelWarn)
logging.WithExtras(
ctx,
"deprecated_param",
queryTagValue,
"deprecation_warning",
fmt.Sprintf("'%s' is deprecated, use '%s'", queryTagValue, renameTo),
)
}
```
Topics can be disabled at the app level, and enabled/disabled at the
command-line level.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
No logs have this in prod yet, but I added some manually in a branch and
verified that I could enable/disable them via CLI options and env vars,
including enabling topics that were disabled on the server. Tested for
both server and `fleetctl gitops`.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added per-topic logging control to enable or disable logging for
specific topics via configuration and CLI flags.
* Added context-aware logging methods (ErrorContext, WarnContext,
InfoContext, DebugContext) to support contextual logging.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #39344
# Details
This PR builds on the previous PR
(https://github.com/fleetdm/fleet/pull/39847) which added `renameto`
tags to certain API parameters to mark them as deprecated. How this is
used:
### In requests
* When decoding requests, log a warning if a `json` or `query` param is
used that has a `renameto` tag, e.g. if a `team_id` param is sent but
the related struct has `renameto:"fleet_id"` in it.
* If the `renamedto` version (e.g. `fleet_id`) is sent in the request,
rewrite it to the deprecated name so that it can be unmarshalled into
the struct
* If both versions are sent (e.g. `team_id` AND `fleet_id`), throw an
error and quit
* URLs with deprecated terms have new aliases using `WithAltPaths` --
warning on using old URLSs a TODO that will be handled in a subsequent
PR.
### In responses
* Output _both_ the deprecated and new names for fields that have
`renameto` tags, so that we don't break existing workflows expecting the
old keys. Uses a shared `DuplicateJSONKeys` to do the duplication.
* Most API responses are handled in `EncodeCommonResponse`. Exceptions
are activities, failing policy webhooks and the streaming "list hosts"
endpoints which call the function directly.
### In fleetctl
* Similar to requests, log warnings when deprecated keys are used and
rewrite the new keys internally so that they can be unmarshalled.
* For `fleetctl get` and `fleetctl generate-gitops`, _only_ output the
new names
* The set of keys to replace is hardcoded in `fleetctl` rather than
being dynamically generated as it is for API endpoints. Given the
mixture of typed and untyped data and the level of nesting, dynamic map
generation was very fragile and error-prone.
### Performance considerations
* The biggest performance hit is the addition of the JSON key rewriter
to the request pipeline. The rewriter buffers the entire request into
memory before eventually passing it to the decoder than unmarshals the
data into structs. I tried implementing this as a true streaming
rewriter but encountered issues where the request would hang if the
downstream reader (the decoder) encountered any errors. It's possible we
could implement this in a streaming fashion if we replace our [current
request
decoder](da43bf8371/server/service/endpoint_utils.go (L108))
with the v2 version, which is a bigger change requiring more thoughtful
discussion in the engineering team. As it stands, memory usage for
requests with deprecated fields will double while the request is being
decoded.
* The "alias rules" used to determine the old and new key names are
cached per struct type and for most endpoints are generated on server
start, so no performance impact is expected.
* Some `fleetctl` commands may have an extra unmarshal/marshal step but
as these are user-initiated and not performed in tight loops, the impact
should be minimal.
### TODO
* Log deprecation warnings when old URLs like "/fleet/teams" are used
* Update API fields that the front-end uses to avoid deprecation
warnings
* Update `fleetctl apply` to accept/return `kind: fleet` rather than
`kind: team`
* Find/update any fleet server config vars with old language
* Update all error messages that use old language
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
* Clicking around the front-end, no broken pages due to request
ingestion errors or bad responses
* Looking in network tab to verify that responses have both the old and
new keys
* Running `fleetctl generate-gitops` and verifying that the output looks
correct and can be ingested by `fleetctl gitops`
* Running `fleetctl get` and `fleetctl apply`
---------
Co-authored-by: kiloconnect[bot] <240665456+kiloconnect[bot]@users.noreply.github.com>
This PR changes 3 things.
1. Validate `admin_url` + all URLs for HTTPS/non-private
2. Add custom `DialContext` hook in fleethttp.NewClient(), this is
needed for DNS-rebinding protection at connection time
3. Validate Smallstep SCEP challenge endpoint
# **IMPORTANT**
There are two validations occurring.
1. `CheckURLForSSRF`
2. `SSRFDialContext`
## Why?
`CheckURLForSSRF` checks the hostname. It resolves DNS, validates the
ip, and then returns an error to the user. It protects certificate
authority create/update API endpoints. But then
`GetSmallstepSCEPChallenge` calls `http.NewRequest(http.MethodPost,
ca.ChallengeURL, ...)` with the original hostname
This is where `SSRFDialContext` comes into play. It fires when an actual
HTTP request is attempted. Meaning Fleet would first build the request,
encode the body, set up TLS, etc., before being blocked at the dial.
`CheckURLForSSRF` stops the operation before any of that work happens.
`SSRFDialContext` protects the actual challenge fetch that happens later
at enrollment time. They're not always called together. The dial-time
check is the only thing protecting the enrollment request and DNS
rebinding.
## Should we remove `CheckURLForSSRF`
This is debatable and I don't have a strong opinion. Removing
`CheckURLForSSRF` would still provide the same protection. However, it
would return a generic connection error from the HTTP client which would
make it slightly hard to diagnose why it is broken.
## What's next
I implemented this for certificate authorities. I am sure there are
other places in the code base that take user submitted urls and could
also use this check. That is outside the scope of this particular PR.
But worthy to investigate in the near future.
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Security**
* Added SSRF protections for validating external URLs and blocking
private/IP-metadata ranges; dev mode can bypass checks for local testing
* **New Features**
* Introduced an SSRF-protected HTTP transport and an option to supply a
custom transport per client
* **Tests**
* Added comprehensive tests covering SSRF validation, dialing behavior,
and resolution edge cases
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39272
Changes file already added on another subtask
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#39265
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
Currently in Fleet Device Management, there is no support for Apple
iPods.
Eventhough iPods are considered vintage by Apple already, we still use
them and I know that in various companies they are still used as a low
cost device within the company. (eg. shops/warehouses to look up stock
levels)
Currently, enrolling an iPod through ABM, results in the device being
recognised as a Mac device.
With this PR, I'd like to add support for iPods, similar functionality
as iPhones to Fleet, simply as iOS device, which works fine. Considering
that all commands are the same (if available) and considering iPods
aren't updated anymore, I don't think we need to explicitly mention it,
perhaps just in docs, and add them to a separate category than iPhones.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [ ] Added/updated automated tests
- I have not added automated tests since it'd basically be a 1:1 copy of
iPhone tests
- [x] Manual QA for all new/changed functionality
> Follows up on discussion from #27263 with @noahtalerman
Manual QA:
- adding an iPod in ABM results in the device being recognised as iOS
<img width="1754" alt="overview"
src="https://github.com/user-attachments/assets/7681c613-2b34-489a-8b94-10eff8977e19"
/>
<img width="1766" alt="detail-abm"
src="https://github.com/user-attachments/assets/f88c8e84-e55f-4c5f-8998-8b6697b57abc"
/>
- after enrolling the iPod through setup, it is correctly synced with
Fleet and all commands are possible. (tried Restart, Rename device, push
apps)
<img width="1766" alt="ipod-post-sync"
src="https://github.com/user-attachments/assets/7668942e-b110-4c38-a448-b6027419507c"
/>
- enrollment video (can be uploaded if needed)
- manual enrollment works fine too (using Enroll url)
---------
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves
https://github.com/fleetdm/confidential/issues/13775
Feature branch merging into main, so all code should be reviewed
---------
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Resolves#38484. This includes a CI job change to make sure we don't
introduce any more env vars that don't get proxied (and thus turned off
outside `--dev`).
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
Manual QA touched hot paths, but did _not_ manually test every
FLEET_DEV_* environment variable change.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Centralized dev-mode environment management for consistent FLEET_DEV_*
handling and test-friendly overrides.
* Dev-mode allows targeted overrides for certain dev-only configuration
when running with --dev.
* **Chores**
* Migrated environment access to the centralized dev-mode helper across
the codebase.
* Added CI checks to enforce proper usage of FLEET_DEV_* variables.
* **Documentation**
* Added guidance on dev-mode environment variable rules and overrides.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Victor Lyuboslavsky <2685025+getvictor@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#38777
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#37710
Adds some logic to try to use the name/version/identifier from a bundle
that exactly matches the bundle identifier from the root pkg-info
element, if found. The loop continues even if that is found since
packageIDSet needs to be built.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
Tested against all installers from the google drive folder, prisma
access browser is the only package with changed metadata.
Before:
```
File: Prisma Access Browser-144.6.10.59-b070a3e0.pkg
- Name: 'PrismaAccessBrowserSoftwareUpdateAgent'
- Bundle Identifier: 'com.paloaltonetworks.PrismaAccessBrowserUpdater.Agent'
- Package IDs: 'com.paloaltonetworks.PrismaAccessBrowserUpdater,com.paloaltonetworks.PrismaAccessBrowserUpdater.Agent,com.talon-sec.Work,com.talon-sec.Work.framework,com.talon-sec.Work.framework.AlertNotificationService,com.talon-sec.Work.helper,com.talon-sec.Work.helper.plugin,com.talon-sec.Work.helper.renderer,org.sparkle-project.Downloader,org.sparkle-project.InstallerLauncher,org.sparkle-project.Sparkle,org.sparkle-project.Sparkle.Updater'
- Version: 144.6.10
```
After:
```
File: Prisma Access Browser-144.6.10.59-b070a3e0.pkg
- Name: 'Prisma Access Browser'
- Bundle Identifier: 'com.talon-sec.Work'
- Package IDs: 'com.paloaltonetworks.PrismaAccessBrowserUpdater,com.paloaltonetworks.PrismaAccessBrowserUpdater.Agent,com.talon-sec.Work,com.talon-sec.Work.framework,com.talon-sec.Work.framework.AlertNotificationService,com.talon-sec.Work.helper,com.talon-sec.Work.helper.plugin,com.talon-sec.Work.helper.renderer,org.sparkle-project.Downloader,org.sparkle-project.InstallerLauncher,org.sparkle-project.Sparkle,org.sparkle-project.Sparkle.Updater'
- Version: 144.6.10.59
```
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#37806
This PR creates an activity bounded context and moves the following HTTP
endpoint (including the full vertical slice) there:
`/api/latest/fleet/activities`
NONE of the other activity functionality is moved! This is an
incremental approach starting with just 1 API/service endpoint.
A significant part of this PR is tests. This feature is now receiving
significantly more unit/integration test coverage than before.
Also, this PR does not remove the `ListActivities` datastore method in
the legacy code. That will be done in the follow up PR (part 2 of 2).
This refactoring effort also uncovered an activity/user authorization
issue: https://fleetdm.slack.com/archives/C02A8BRABB5/p1768582236611479
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Activity listing API now available with query filtering, date-range
filtering, and type-based filtering
* Pagination support for activity results with cursor-based and
offset-based options
* Configurable sorting by creation date or activity ID in ascending or
descending order
* Automatic enrichment of activity records with actor user details
(name, email, avatar)
* Role-based access controls applied to activity visibility based on
user permissions
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#13698
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33509
All changes were approved in PRs to this feature branch.
---------
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com>
Co-authored-by: George Karr <georgekarrv@users.noreply.github.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves oversight on initial bug fix:
https://github.com/fleetdm/fleet/issues/32920#issuecomment-3716712957
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#32920
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Resolves#36909.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34376
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35308
Changes file covered by merge for #35307
Changes Windows MDM enrollment and unenrollment logic to no longer send
notifications to orbit and as such, require customers to enroll through
Settings or Autopilot. When a new enrollment is detected, processes the
enrollment type(auto=autopilot, manual=settings app) as specified and
maps it to the user who performed the enrollment
This is unfortunately all done in the query processing code which is
where we currently process these enrollments. I didn't see a better way
to do it without significant rework that was unclear in scope and the
new logic is simple enough it didn't feel like the right time to do
that.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
**Related issue:** Resolves#35460, #35462
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **New Features**
* Added certificate templates for managing Android device certificates
at global and team levels
* Introduced API endpoints to create, list, retrieve, and delete
certificate templates
* Enabled GitOps workflow support for certificate template
specifications
* Implemented automatic variable substitution in certificate subjects
for host identifiers
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Scott Gress <scottmgress@gmail.com>
Co-authored-by: Scott Gress <scott@fleetdm.com>
Added a new PowerShell uninstall script for 1Password with process
termination, registry checks, logging, and timeout handling. Updated the
template uninstall script by adding timeout protection to generic MSI
uninstall scripts to prevent hangs. These changes improve reliability
and diagnostics for uninstall operations.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves #
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [ ] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
## Database migrations
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
## fleetd/orbit/Fleet Desktop
- [ ] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [ ] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35307
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
**Related issue:** Resolves#35357
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
For #35357.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
No changes file as this is a zero-functionality-change refactor.
Performance improvements are in their own PR, which includes a changes
file.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34827
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [ ] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
Follow up findings from running through the test plan
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34847
# Details
In testing the new end-user auth flow for enrolling hosts on Windows, I
noticed that the `&host_uuid=<uuid>` part of the query string was
missing when opening the browser window, causing the SSO login process
to ultimately fail. I discovered that this is because the command we're
using to open the browser interprets `&` as a command separator, so it
needs to be escaped. This PR updated the `open` package for Windows, by
adding escaping to all `&` characters in a URL that are not already
escaped.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
## Testing
- [X] QA'd all new/changed functionality manually
Added a new tool for testing:
```
go run ./tools/open -url "http://google.com?x=y&owl=hoot^&foo=bar"
```
to test that it escapes ampersands correctly (and doesn't
double-escape).
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34528
# Details
This PR implements the agent changes for allowing Fleet admins to
require that users authenticate with an IdP prior to having their
devices set up. I'll comment on changes inline but the high-level is:
1. Orbit calls the enroll endpoint as usual. This is triggered lazily by
any one of a number of subsystems like device token rotation or
requesting Fleet config
2. If the enroll endpoint returns the new `ErrEndUserAuthRequired`
response, then it opens a window to the `/mdm/sso` Fleet page and
retries the enroll endpoint every 30 seconds indefinitely.
3. Any other non-200 response to the enroll request is treated as before
(limited # of retries, with backoff)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-
changes.md#changes-files) for more information.
Will add changelog when story is one.
## Testing
- [X] Added/updated automated tests
Added test for new retry logic
- [X] QA'd all new/changed functionality manually
This is kinda hard to test without the associated backend PR:
https://github.com/fleetdm/fleet/pull/34835
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
This is compatible with all Fleet versions, since older ones won't send
the new error.
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
This is compatible with all platforms, although it currently should only
ever run on Windows and Linux since macOS devices will have end-user
auth taken care of before they even download Orbit.
- [ ] Verified that fleetd runs on macOS, Linux and Windows
Testing this now.
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added SSO (Single Sign-On) enrollment support for end-user
authentication
* Enhanced error messaging for authentication-required scenarios
* **Bug Fixes**
* Improved error handling and retry logic for enrollment failures
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#32084
This PR modifies `isValidAppFilePath` to allow subdirectors in
`Applications/`, like in this case `Applications/Cisco/Cisco Secure
Client.app`.
This also changes the metadata extraction from packageinfo to trim
`.app` from the name in all cases.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
### Test plan:
---
I ran this on my local environment and it seemed fine
- Have environment with the bug recreated, it has two software titles
for "Cisco Secure Client", and the one with the bundle id
`com.cisco.pkg.anyconnect.vpn` is used by the installer.
- URL to pkg:
https://fndtnfleetmsp.blob.core.windows.net/fndtnpkgs/cisco-secure-client-macos-5.1.3.62-core-vpn-webdeploy-k9.pkg
- Cisco Secure Client doesn't show as installed in UI even after
installing.
- Run the new migration.
- Cisco Secure Client shows as installed now in ui, software title with
bundle id `com.cisco.pkg.anyconnect.vpn` is gone from the database, and
the software installer references the correct title
(`com.cisco.secureclient.gui`).
- Check that deleting and reuploading the installer doesn't recreate the
bad software title.
### QA Note:
---
There are some problems with the install script, but that is probably a
different scope than this ticket.
`Reinstall` wont work, it says Cisco Secure Client is already installed.
Uninstalling through Fleet then Installing again works fine though.
Fixes#34066. Included a changes file because this catches fields other
than software icon.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34209
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34203
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes#31897.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [ ] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- GitOps now supports software icons: generate and include icon
files/paths in specs for packages and App Store apps.
- CLI adds flags to control concurrent icon uploads/updates.
- Icons are uploaded, updated, or deleted automatically during GitOps
runs.
- UI YAML modal now includes icon_url and offers icon download.
- Improvements
- Robust path resolution for icon assets across specs.
- Non-YAML outputs handle both string and byte file contents.
- Bug Fixes
- Removes stale icons after App Store app re-association.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Scott Gress <scottmgress@gmail.com>
Co-authored-by: Scott Gress <scott@fleetdm.com>
Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
**Related issue:** Resolves#32083
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
for #32014
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- GitOps manual labels can now reference hosts by Fleet host ID in
addition to hostname, hardware serial, or UUID.
- GitOps YAML/JSON accepts integers for host IDs; numeric IDs are
handled seamlessly alongside strings.
- Validation
- Stronger input validation for label hosts: only strings or integers
are allowed.
- Clear error returned for invalid types (e.g., floats) in hosts lists.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33250
Waived most new failures. Planning to come back and fix some of them in
subsequent PRs.
for #31321
# Details
Small updates from [community
PR](https://github.com/fleetdm/fleet/pull/31134):
* Updated config vars to match
[docs](https://github.com/fleetdm/fleet/blob/docs-v4.75.0/docs/Configuration/fleet-server-configuration.md#server_private_key_region)
* Added support for specifying region in config (already documented)
* Removed parsing of ARN for region
* Made retry backoff intervals a bit longer
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
(already added in the community PR
[here](https://github.com/fleetdm/fleet/blob/sgress454/updates-for-private-key-in-aws-sm/changes/private-key-secrets-manager#L0-L1)
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added support for specifying the AWS region for server private key
retrieval from AWS Secrets Manager via server.private_key_region.
- Chores
- Renamed configuration keys:
- server.private_key_secret_arn → server.private_key_arn
- server.private_key_secret_sts_assume_role_arn →
server.private_key_sts_assume_role_arn
- server.private_key_secret_sts_external_id →
server.private_key_sts_external_id
- Update your configuration to use the new keys.
- Adjusted retry backoff for Secrets Manager retrieval to improve
resilience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#31974
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
for #31087
for #32863
# Details
The [community PR](https://github.com/fleetdm/fleet/pull/30608) for
#31087 sets a number of new env vars to try and ensure that `xdg-open`
can correctly open URLs when called from our `open` package on linux.
However, it was discovered that the addition of the
`XDG_CURRENT_DESKTOP` env var actually broke `xdg-open` use on
environments using KDE Plasma (i.e. Kubuntu). After doing some testing I
determined that the best path forward was to just clear this env var
when KDE was detected, setting it back to the state that previous
versions of `fleet-desktop` have been in: that is, allowing `xdg-open`
to determine how to open links by other means.
# Checklist for submitter
## Testing
- [X] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
Fleet Desktop may fail to launch a browser, or launch the wrong browser,
in Ubuntu 24.04 LTS under some scenarios, including Wayland sessions
and/or when the browser is installed as a snap or flatpak (by default,
Ubuntu 24.04 LTS installs Firefox as a snap).
The previous approach of searching for an Xwayland process from which to
extract environment variables is incomplete. When the browser is
installed as a snap, `.desktop` files are installed by snapd into a
location which is only discoverable through the `XDG_DATA_DIRS`
environment variable.
To workaround this we need to change the approach in several ways:
- Look for `xdg-desktop-portal` processes, not just `Xwayland`. Both
Firefox and Chrome support native operation under Wayland now, so it's
not correct to rely upon the existence of an Xwayland process alone.
- Copy several more environment varibles: `XDG_CURRENT_DESKTOP`,
`XDG_RUNTIME_DIR`, `XDG_DATA_DIRS`, and `PATH`, in addition to
`XAUTHORITY`.
These variables allow fleet-desktop to honor the user's browser
preference according to the currently running DE.
The list of variables to grab was found through trial-and-error and
seems to be the minimum required for `xdg-open` to function.
Refactored and optimized the code to fetch environment variables from a
running process to match only processes running as the current user and
robustly handle multiple process results.
Fixes#19043, #27209.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in all relevant combinations of Linux
desktop environments.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved compatibility for launching browsers installed as Flatpak or
Snap under Wayland sessions on Linux, resolving issues where Fleet
Desktop failed to open these browsers.
* Enhanced environment variable handling to better support browser
launches in containerized and Wayland environments.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #29478, sans GitOps.
---------
Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
For #30849.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [n/a] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
There are still some TODOs particularly within Gitops test code which
will be worked on in a followup PR
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [x] Alerted the release DRI if additional load testing is needed
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
---------
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
For #30095.
#32482 is additional cleanup. Merging this to unblock orchestration
Linux setup experience work. Code has already been reviewed prior to
merging into the feature branch.
---------
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
Co-authored-by: Anthony Maxwell <133805840+Illbjorn@users.noreply.github.com>
- Added Jira and Zendesk integrations for "No team". (These are not
supported by GitOps for teams)
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [x] Setting(s) is/are explicitly excluded from GitOps
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Default (No Team) responses now include limited integrations (Jira,
Zendesk).
- You can configure or clear Jira/Zendesk integrations for the Default
(No Team) settings.
- Bug Fixes
- More consistent handling of the Default (No Team) when fetching team
details.
- Improved validation to prevent conflicting automation settings between
webhooks and integrations.
- Documentation
- Clarified that Jira/Zendesk integrations aren’t supported via GitOps
or at the team level (including No Team).
- Noted that certain options (e.g., Google Calendar, Conditional Access)
aren’t supported for the Default (No Team).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes#32060
This PR adds:
- new default_team_config_json table
- caching of config from that table, including deep copy methods -- all
of this is not absolutely needed for this change since we are only using
`webhook_settings.failing_policies_webhook` here but added for
completeness/future
- teams/0 API updates
- GitOps updates
- generate gitops updates
Future PRs will add:
- ticket automation
- primo mode migration
- frontend changes
- documentation
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Configure failing-policy webhooks for “No team” via GitOps
(no-team.yml) and API, including enable/disable, destination URL, policy
IDs, and batch size; settings clear when omitted.
- GitOps and CLI now read/apply the real “No team” settings with dry-run
support.
- Policy automation evaluates hosts without a team and triggers “No
team” webhooks when applicable.
- GET/PATCH team 0 returns/accepts a minimal, webhook-focused config.
- Chores
- Added persistence and caching for the default “No team” configuration.
- Introduced a database table to store the default configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Fixes#32393
httpsig-go library has encorporated the changes needed to support TPM,
so we are removing our local version of this library.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
Fixes#31477
Docs PR: https://github.com/fleetdm/fleet/pull/32116
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- GitOps now supports FLEET_SECRET_ placeholders in macOS
(.mobileconfig/.xml) profiles. Secrets are expanded only for validation,
while remaining unexpanded in uploaded content.
- Improved environment variable handling: non-secret vars expand as
before; server-side secrets are preserved.
- Validation enforces that profile display names cannot contain
FLEET_SECRET_ values.
- Bug Fixes
- Resolves validation issues when FLEET_SECRET_ appears in <data> tags
by performing safe client-side expansion for validation.
- More accurate error reporting during profile parsing and validation.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Fixes#29894 and probably #31980.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
fixes: #29222
This is a feature branch that was completed last week, but did not get
merged in time.
All pr's going in was approved, and reviewed.
I will after this is merged, do a cherry pick onto the RC 4.73 branch,
and initiate the FR merge process.
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
For #31055.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Also removed the automatic install flag on YAML FMAs as it's
undocumented/unspec'd
Fixes#25636.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
Fixes#31338, #31525.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [s] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
For #31063
# Details
This PR adds the `RequireBitLockerPIN` config to app-wide and team
configs. This maps to a new `windows_require_bitlocker_pin` JSON field
for gitops and `fleetctl apply`.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
* Will add changelog when feature is complete
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
* Will add to docs when feature is complete
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
* No UI yet
- [X] Manual QA for all new/changed functionality
* Tested No Team and team config via Postman API calls
* Tested Gitops for no-team and team YML files using `fleetctl`
* Tested `fleetctl generate-gitops`
Including WARP/Box Drive but not Chrome here because it's swapping to an
EXE in #27756, which is currently WIP.
For #31077.
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Allen Houchins <allenhouchins@mac.com>
Closes#31077.
- Added logic to wait for the uninstall command to finish running before
exiting the script.
- Also added the `/norestart` flag so users who click uninstall in
self-service aren't at risk of a sudden and unintentional reboot as the
result of software uninstalling.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Added/updated automated tests
- [x] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [x] Manual QA for all new/changed functionality
#30461
This PR contains the changes for the happy path.
On a separate PR we will be adding tests and further fixes for edge
cases.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for using a TPM-backed key and SCEP-issued certificate
to sign HTTP requests, enhancing security through hardware-based key
management.
* Introduced new CLI and environment flags to enable TPM-backed client
certificates for Linux packages and Orbit.
* Added a local HTTPS proxy that automatically signs requests using the
TPM-backed key.
* **Bug Fixes**
* Improved cleanup and restart behavior when authentication fails with a
host identity certificate.
* **Tests**
* Added comprehensive tests for SCEP client functionality and TPM
integration.
* **Chores**
* Updated scripts and documentation to support TPM-backed client
certificate packaging and configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Closes#27447
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
Fixes#27758.
<img width="807" height="303" alt="image"
src="https://github.com/user-attachments/assets/58e5b9bc-42d6-4195-868e-bf6206ec9cd5"
/>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes 30636
I am adding a handful of additional unit tests but this is ready for
review now. Integrates changes from Victor's PoC for Account Driven User
Enrollment including a nice end to end integration test including the
SAML portion
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#30435.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Fixes#30458
Contributor docs PR: https://github.com/fleetdm/fleet/pull/30651
# Checklist for submitter
- We will add changes file later.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- Did not do manual QA since the SCEP client I have doesn't support ECC.
Will rely on next subtasks for manual QA.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Introduced Host Identity SCEP (Simple Certificate Enrollment Protocol)
support, enabling secure host identity certificate enrollment and
management.
* Added new API endpoints for Host Identity SCEP, including certificate
issuance and retrieval.
* Implemented MySQL-backed storage and management for host identity SCEP
certificates and serials.
* Added new database tables for storing host identity SCEP certificates
and serial numbers.
* Provided utilities for encoding certificates and keys, and handling
ECDSA public keys.
* **Bug Fixes**
* None.
* **Tests**
* Added comprehensive integration and unit tests for Host Identity SCEP
functionality, including certificate issuance, validation, and error
scenarios.
* **Chores**
* Updated test utilities to support unique test names and new SCEP
storage options.
* Extended mock datastore and interfaces for new host identity
certificate methods.
* **Documentation**
* Added comments and documentation for new SCEP-related interfaces,
methods, and database schema changes.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#25587. SubEthaEdit packgeInfo file is a bit bigger, but the only
thing different is the list of package IDs included, and that's not what
was broken/fixed here, so went with an abbreviated version that better
demonstrates what got fixed here.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved extraction of application names from uploaded PKG packages by
using the install path as a fallback method.
* **Tests**
* Added a new test case to verify correct name extraction from PKG
packages using the install path.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
> Fixes#29851
> Fixes#29902
> Mainly followups from https://github.com/fleetdm/fleet/pull/30295,
plus improved integration testing
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
# Details
This PR adds support for a new label membership type, `host_vitals`.
Membership for these labels is based on a database query created from
user-supplied criteria. In this first iteration, the allowed criteria
are very simple: a label can specify either an IdP group or IdP
department, and hosts with linked users with a matching group or
department.
Groundwork is laid here for more complex host vitals queries, including
`and` and `or` logic, different data types and different kinds of vitals
(rather than just the "foreign" vitals of which IdP is an example).
Note that this PR does _not_ include the cron job that will trigger
membership updating, and it doesn't include ; for sake of simplicity in
review that will be done in a follow-on PR.
## Basic flow
### Creating a host vitals label
1. A new label is created via the API / GitOps with membership type
`host_vitals` and a `criteria` property that's a JSON blob. Currently
the JSON can only contain `vital` and `value` keys (and must contain
those keys)
2. The server validates that the specified `vital` exists in our [set of
known host
vitals](https://github.com/fleetdm/fleet/pull/30278/files#diff-b6d4c48f2624b82c2567b2b88db1de51c6b152eeb261d40acfd5b63a890839b7R418-R436).
3. The server validates that the [criteria can be parsed into a
query](https://github.com/fleetdm/fleet/pull/30278/files?diff=unified&w=1#diff-4ac4cfba8bed490e8ef125a0556f5417156f805017bfe93c6e2c61aa94ba8a8cR81-R86).
This also happens during GitOps dry run.
4. The label is saved (criteria is saved as JSON in the db)
### Updating membership for a host vitals label
1. The label's criteria is used to generate a query to run on the
_Fleet_ db.
1. For each vital criteria, check the vital type. Currently only foreign
vitals are supported.
2. For foreign vitals, add its group to a set we keep track of.
3. Add a `WHERE` clause section for the vital and value, e.g.
`end_user_idp_groups = ?`
4. Once we have all the `WHERE` clauses, create the query as `SELECT %s
FROM %s` + any joins contributed by foreign vitals groups + `WHERE ` +
all the `WHERE` clauses we just calculated. The `%s` provide some
flexibility if we want to use these queries in other contexts.
2. Delete all existing label members
3. Do an `INSERT...SELECT` using the query we calculated from the label
criteria. The query will be `SELECT <label id> as label_id, hosts.id
FROM hosts JOIN ...`
## Future work
### Domestic vitals
These can be anything that we already store in the `hosts` table.
Domestic vitals won't add any `JOIN`s to the calculated label query, and
will simply be e.g. `hosts.hostname = ?`
### Custom vitals
We currently support an `additional_queries` config that will cause
other queries to run on hosts. The data returned from these queries is
stored in a `hosts_additional` table as a JSON blob. We can use MySQL
JSON functions to match values in this data, e.g.
`JSON_EXTRACT(host_additional, `$.some_custom_vital`) = ?`
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
> I'll add the changelog item when I add the cron job PR
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [X] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [X] Verified that the setting is exported via `fleetctl
generate-gitops`
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Adds support for the Apple MDM user channel however we are waiting on
stories around verification among other things for this and we are not
shipping as part of 4.70 so this can be reviewed but should not be
merged yet
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
For #26996 and #28452
Demo video: https://www.youtube.com/shorts/WGS3JmKiZTs
The device/machine info is extracted from the PKCS7 signed body of the
POST request.
I did manual QA on iPhone since I don't have an ADE macOS device with
me.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #27522.
See GDrive installer testdata for the Google Workspace Sync MSI used for
testing this.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Manual QA for all new/changed functionality
For #24083, #26597.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #28215
Allows users to use fleet secret variables for macos setup script for
gitops.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #28855
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
https://github.com/fleetdm/fleet/issues/24469
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #28140
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26692 (fixes permission issue when extracting dirs).
Reverts changes to `update.go` to remove TUF test surface.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
For #27476
# Checklist for submitter
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
# Details
This PR adds a new command `generate-gitops` to the `fleetctl` tool. The
purpose of this command is to output GitOps-ready files that can then be
used with `fleetctl-gitops`.
The general usage of the command is:
```
fleectl generate-gitops --dir /path/to/dir/to/add/files/to
```
By default, the outputted files will not contain sensitive data, but
will instead add comments where the data needs to be replaced by a user.
In cases where sensitive data is redacted, the tool outputs warnings to
the user indicating which keys need to be updated.
The tool uses existing APIs to gather data for use in generating
configuration files. In some cases new API client methods needed to be
added to support the tool:
* ListConfigurationProfiles
* GetProfileContents
* GetScriptContents
* GetSoftwareTitleByID
Additionally, the response for the /api/latest/fleet/software/batch
endpoint was updated slightly to return `HashSHA256` for the software
installers. This allows policies that automatically install software to
refer to that software by hash.
Other options that we may or may not choose to document at this time:
* `--insecure`: outputs sensitive data in plaintext instead of leaving
comments
* `--print`: prints the output to stdout instead of writing files
* `--key`: outputs the value at a keypath to stdout, e.g. `--key
agent_options.config`
* `--team`: only generates config for the specified team name
* `--force`: overwrites files in the given directory (defaults to false,
which errors if the dir is not empty)
# Technical notes
The command is implemented using a `GenerateGitopsCommand` type which
holds some state (like a list of software and scripts encountered) as
well as a Fleet client instance (which may be a mock instance for tests)
and the CLI context (containing things like flags and output writers).
The actual "action" of the CLI command calls the `Run()` method of the
`GenerateGitopsCommand` var, which delegates most of the work to other
methods like `generateOrgSettings()`, `generateControls()`, etc.
Wherever possible, the subroutines use reflection to translate Go struct
fields into JSON property names. This guarantees that the correct keys
are written to config files, and protects against the unlikely event of
keys changing.
When sensitive data is encountered, the subroutines call `AddComment()`
to get a new token to add to the config files. These tokens are replaced
with comments like `# TODO - Add your enrollment secrets here` in the
final output.
# Known issues / TODOs:
* The `macos_setup` configuration is not output by this tool yet. More
planning is required for this. In the meantime, if the tool detects that
`macos_setup` is configured on the server, it outputs a key with an
invalid value and prints a warning to the user that they'll need to
configure it themselves.
* `yara_rules` are not output yet. The tool adds a warning that if you
have Yara rules (which you can only upload via GitOps right now) that
you'll have to migrate them manually. Supporting this will require a new
API that we'll have to discuss the authz for, so punting on it for now.
* Fleet maintained apps are not supported by GitOps yet (coming in
https://github.com/fleetdm/fleet/issues/24469). In the meantime, this
tool will output a `fleet_maintained_apps` key and trigger a warning,
and GitOps will fail if that key is present.
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
For #27700
When uploading bootstrap package for macOS setup experience, validate
that it is a Distribution package since that is required by Apple's
InstallEnterpriseApplication MDM command.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26692.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
Changes file included in FE PR.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
