Addresses
https://github.com/fleetdm/fleet/issues/26443#issuecomment-2749360869
after https://github.com/fleetdm/fleet/pull/27176 was merged. Reading
XML as a string in this way feels wrong, but I don't want to avoid a
refactor, so I'm checking for a "comment" string in this PR.
I tested by building fleetctl locally and running:
```sh
$ make fleetctl; ./build/fleetctl gitops -f it-and-security/teams/test.yml --dry-run
...
Client Version: tf-mod-addon-monitoring-v1.5.1-1091-g8eb9111c6-dirty
Server Version: 0.0.0-SNAPSHOT-85f4f65
[+] applying MDM profiles for team TEST
Error: applying custom settings for team "TEST": POST /api/latest/fleet/mdm/profiles/batch received status 422 Validation Failed: disable-onedrive is not a valid macOS or Windows configuration profile. macOS profiles must be valid .mobileconfig or .json files. Windows configuration profiles can only have <Replace> or <Add> top level elements.
```
I'm not sure if the error above
([code](8eb9111c67/server/service/mdm.go (L2160)))
is caused by my test environment not yet having the updated server code.
The `--dry-run` passed in my test, as seen by the `[+] applying MDM
profiles for team TEST` line. I can't get any test code to be reflected
in the server response, so my hunch is that the issue should be fixed
after this PR.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- I did this in https://github.com/fleetdm/fleet/pull/27176, same change
message.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26713
# Details
This PR updates Fleet and its related tools and binaries to use Go
version 1.24.1.
Scanning through the changelog, I didn't see anything relevant to Fleet
that requires action. The only possible breaking change I spotted was:
> As [announced](https://tip.golang.org/doc/go1.23#linux) in the Go 1.23
release notes, Go 1.24 requires Linux kernel version 3.2 or later.
Linux kernel 3.2 was released in January of 2012, so I think we can
commit to dropping support for earlier kernel versions.
The new [tools directive](https://tip.golang.org/doc/go1.24#tools) is
interesting as it means we can move away from using `tools.go` files,
but it's not a required update.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet
- [x] Orbit runs on macOS ✅ , Linux ✅ and Windows.
- [x] Manual QA must be performed in the three main OSs, macOS ✅,
Windows and Linux ✅.
> No issue, we noticed this while testing FMA for Windows
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #27553
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
I manually verified this fix by running the new
`github.com/fleetdm/fleet/v4/server/mdm/apple/gdmf/integrationtest` test
with and without the fix on a cloud Ubuntu server.
Without fix:
```
=== RUN TestGetAssetMetadata
gdmf_test.go:14:
Error Trace: /root/fleet/server/mdm/apple/gdmf/integrationtest/gdmf_test.go:14
Error: Received unexpected error:
retrieving asset metadata: Get "https://gdmf.apple.com/v2/pmv": tls: failed to verify certificate: x509: certificate signed by unknown authority
Test: TestGetAssetMetadata
--- FAIL: TestGetAssetMetadata (3.53s)
FAIL
FAIL github.com/fleetdm/fleet/v4/server/mdm/apple/gdmf/integrationtest 3.542s
FAIL
```
With fix:
```
=== RUN TestGetAssetMetadata
--- PASS: TestGetAssetMetadata (0.39s)
PASS
ok github.com/fleetdm/fleet/v4/server/mdm/apple/gdmf/integrationtest 0.397s
```
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #24862
Unreleased bug.
Made disk encryption errors different between `configuration_profiles`
and `batch` endpoints.
# Checklist for submitter
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
- [x] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
For #24862
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26082.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26623
- Updated `github.com/groob/plist` to `github.com/micromdm/plist` -- it
was renamed
- Added validation that restricts DigiCert Fleet variables to
`com.apple.security.pkcs12` payloads plus additional restrictions
- Added validation that restricts Custom SCEP Fleet variables to
`com.apple.security.scep` payloads plus additional restrictions
- Enabled multiple CAs (Fleet variables) to be present in an Apple MDM
profile. But each CA can only be used once. For example, we can have
DigiCert CA and Custom SCEP CA in one Apple profile.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26623
This PR enables deploying an Apple configuration profile with Fleet
proxying a custom SCEP server.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26652. No changes file as that'll come in another PR. Will stack
additional PRs on top of this one (for ingestion changes etc.) to get
this merged more quickly.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #26345
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26219
Refactoring the interface between Android service and Android datastore
to use 1 common datastore interface: `fleet.AndroidDatastore`
These changes are based on feedback from the recent Backend Sync.
```mermaid
---
title: Partial class diagram
---
classDiagram
direction LR
class `android.Service`
<<interface>> `android.Service`
class `android/service.Service`
`android/service.Service` ..|> `android.Service`: implements
class `fleet.AndroidDatastore`
<<interface>> `fleet.AndroidDatastore`
class `fleet.Datastore`
<<interface>> `fleet.Datastore`
class `android.Datastore`
<<interface>> `android.Datastore`
`android/service.Service` *-- `fleet.AndroidDatastore`: USES (THIS IS THE KEY CHANGE)
`fleet.Datastore` *-- `fleet.AndroidDatastore`: contains
`mysql.Datastore` ..|> `fleet.Datastore`: implements
`fleet.AndroidDatastore` *-- `android.Datastore`: contains
`mysql.Datastore` *-- `android.Datastore`: contains
`android/mysql.Datastore` ..|> `android.Datastore`: implements
```
> For #26083
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
For #26609
This PR includes
- ability to get a DigiCert certificate to a macOS device
- integration test for the above
- some validation
This PR does not include the following. They will be included in
subsequent PRs:
- support for User Principal Name in certificate
- support for $FLEET_VAR_HOST_HARDWARE_SERIAL
- saving certificate expiration date
- not resending DigiCert profile after failure
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26736
adds a self closing page that is the final part of connecting with
Android Enterprise. This allows the user to see the fleet page
notification that android mdm is now enabled.
For #26219
Frontend wasn't displaying the right Android page due to missing
details.
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
For #26218
- Added signup_token authentication for Android enterprise callback and
fixed API path to match API doc
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
This is a dev helper feature (since it will be implemented in
fleetdm.com). It fixes the issue of stale PubSub topics. Having stale
PubSub topics around may cause extra traffic/errors on dev server.
- [x] Manual QA for all new/changed functionality
> For #26675
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26219
Need to get in this security change for the RC
- Added auth (token stored in `mdm_config_assets`) for PubSub endpoint
- unknown notification types, including `test` are not authenticated
- Status reports for deleted devices are not authenticated -- no DB
changes are done for these since they only happen when we disable MDM
right now
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
For #26218
- Added `users_deleted` table to track user actions if the user was
actually deleted.
- Added enable/disable Android MDM activities
Note: I could not auto-generate fleet.Service mock because it has issues
with methods that don't return anything. I ended up using testify mock
instead.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
- Added `GET /api/_version_/fleet/android_enterprise/signup_sse`
endpoint and tests
- Fixed up handling of Android status reports with a deleted device. We
don't actually expect this to happen in production since the proxy
should delete the pubSub connection when the enterprise is deleted.
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
- Added `GET /api/_version_/fleet/android_enterprise` andpoint and tests
- Set up some testing infrastructure for Android service tests -- see
new README.md
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26219
Video explaining the PR: https://www.youtube.com/watch?v=t2hmpE7N-50
This PR includes:
- applying a default/dummy policy to Android enterprise
- Setting up PubSub topic and subscription to point at Fleet Server
- Generating device enrollment token
- Enrolling a device
Although some tests were added, this PR is mostly manually tested.
Integration tests are an outstanding task.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
Refactoring android/mysql/testing_utils.go to remove duplication with
main mysql package.
No functional changes.
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
For #26218
Basic Android MDM on/off backend functionality. Manually tested.
The following env vars must be set:
```
FLEET_DEV_ANDROID_ENABLED=1
FLEET_DEV_ANDROID_SERVICE_CREDENTIALS=$(cat credentials.json)
FLEET_DEV_ANDROID_PUBSUB_TOPIC=projects/your-project/topics/your-topic
```
I picked https://github.com/go-json-experiment/json as the JSON library,
which seems like the safest option.
- will become json/v2 at some point
- currently used in production by other companies, like Tailscale
- well-maintained
- Some context here: https://github.com/fleetdm/fleet/issues/25512
Plan for next work:
- refactoring from 1st PR
- add pubsub with device enroll -> spec proxy for fleetdm.com
- come back to this sub-task to add tests and finish TODOs
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> For #24222
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
Android scaffold code and refactorings
- Android packages intended to be decoupled from other Fleet code
Video explaining the PR: https://www.youtube.com/watch?v=cza-35Z9Wxk
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #25901
Need to close `com.electron.dockerdesktop`
Check if destination for symlink exists prior to symlinking
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
