fleet/server
Scott Gress 26e4395926
Allow GitOps to clear global settings more easily using overwrite option (#29215)
for #28118 

# Checklist for submitter

- [X] Manual QA for all new/changed functionality

## Details

This PR adds an `overwrite` option to the "modify app config" API which,
if set, causes the code to replace certain keys in the existing config
with keys from the incoming config, without attempting any merge. This
is then used by GitOps to allow it to easily clear settings that were
otherwise being merged together or ignored entirely due to the PATCH
semantics expected for the `fleetctl apply` use case.

The new setting is utilized in this first pass for the following
settings:

* `sso_settings`
* `smtp_settings`
* `features`
* `mdm.end_user_authentication`

It could be expanded to several more keys that we currently handle
piecemeal in the GitOps code by attempting to send empty values to the
server (with varying success).

Targeting `mdm.end_user_authentication` vs. all of `mdm` is based on
[this bug](https://github.com/fleetdm/fleet/issues/26175) being opened.
The concern with doing all of `mdm` would be that anyone who had e.g.
VPP set up in their app and hadn't set it up in GitOps would have it
wiped out. If we're comfortable with that risk I can update that here
and update the warning accordingly.

### More detail 

**The way this code works _without_ Overwrite mode on**

1. We unmarshall the incoming JSON from GitOps into a fresh AppConfig
struct `newAppConfig`. Anything keys not present in the incoming JSON
will result in default values being set in `newAppConfig`
2. We unmarshall the incoming JSON from GitOps into the current
`appConfig`. This uses an internal merge algorithm where keys not
present in the JSON will generally leave the matching keys in
`appConfig` untouched. We've been dealing with this by having GitOps
find missing keys and explicitly set them to non-nil empty states. When
arrays are encountered, they are _merged_, not replaced, which is
problematic for the `features.additional_queries` use case and probably
others.
3. We piecemeal replace certain data in `appConfig` with data from
`newAppConfig`, and save it to the db.

**The way this works _with_ Overwrite mode on**

Between steps 1 and 2 above, we _copy_ certain keys from `newAppConfig`
to `appConfig`. If the incoming JSON didn't have a key, the effect will
be that `appConfig` now has default values for that key. For nested
arrays like `features.additionalQueries`, the value in `appConfig` will
be precisely what the user put in GitOps.

## Testing

I tested adding/removing these settings with GitOps manually via
`fleetctl gitops`. On the main branch I could reproduce the issue where
omitting out these keys in my YAML did not lead to the settings being
reset on my instance. With the Features settings, the issue was more
granular, with inconsistent behavior when trying to remove individual
nested settings. On this branch, the settings are cleared as expected at
all levels of granularity.

I also added some new automated tests to verify the expected behavior
for these keys. All existing tests pass.

If accepted this PR would supercede
https://github.com/fleetdm/fleet/pull/29180 which approaches the issue
from the GitOps side for sso, smtp and mdm. Adapting that approach for
`features` would require custom logic to declare nested properties as
"cleared".
2025-05-19 11:18:28 -05:00
..
archtest Basic Android MDM on/off functionality (#26309) 2025-02-18 09:43:11 -06:00
authz Added scim/details endpoint (#28007) 2025-04-10 14:08:45 -05:00
bindata
config API, Environment variable for bespoke functionality for customer-preston (#28893) 2025-05-12 13:04:18 -07:00
contexts Update to Go 1.24.1 (#27506) 2025-03-31 11:14:09 -05:00
cron Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
datastore Batch host_software inserts in macOS names migration to improve performance for large host counts (#29238) 2025-05-17 12:42:01 -05:00
errorstore Adding telemetry for specific Fleet Desktop errors (#23349) 2024-10-31 14:24:42 -05:00
fleet Allow GitOps to clear global settings more easily using overwrite option (#29215) 2025-05-19 11:18:28 -05:00
goose Add gosimple linter (#23250) 2024-10-29 14:17:51 -05:00
health 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
launcher Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
live_query Fix flakey livequery test (#21666) 2024-08-29 10:03:45 -06:00
logging Webhook logging destination (#28692) 2025-05-01 16:13:04 -04:00
mail Add To: header when constructing emails (#28507) 2025-04-24 09:00:35 -04:00
mdm Force enrollment profile sync when an Apple device was added to ABM. (#29147) 2025-05-15 12:29:38 -05:00
mock Fixing issues with Apple DDM profile status (#29059) 2025-05-15 13:05:25 -05:00
policies 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
ptr Added better handling of timeout and insufficient permissions errors in NDES SCEP proxy. (#23654) 2024-11-11 14:57:28 -06:00
pubsub Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
service Allow GitOps to clear global settings more easily using overwrite option (#29215) 2025-05-19 11:18:28 -05:00
shellquote Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
sso Merge commit from fork 2025-03-05 07:31:17 -06:00
test Bulk script execution backend (#28299) 2025-04-30 12:54:46 -04:00
version Move external dependency fleetdm/kolide-kit to monorepo (#15861) 2024-01-02 18:22:52 -03:00
vulnerabilities Skip vulnerability checks on Docker DX VSCode plugin to avoid false positives due to overly broad Docker CPEs (#29156) 2025-05-15 09:16:33 -05:00
webhooks Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
websocket Enable errcheck linter for golangci-lint (#8899) 2022-12-05 16:50:49 -06:00
worker Final fixes after testing with Entra ID. (#28987) 2025-05-09 09:27:23 -05:00
utils.go Fix CleanupExpiredHosts to prevent deletion of DEP-assigned hosts (#28313) 2025-04-18 12:49:03 -05:00
utils_test.go feature: target profiles by labels (#16202) 2024-01-26 11:00:58 -05:00