fixes: #29222
This is a feature branch that was completed last week, but did not get
merged in time.
All pr's going in was approved, and reviewed.
I will after this is merged, do a cherry pick onto the RC 4.73 branch,
and initiate the FR merge process.
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Added/updated automated tests
- [x] Where appropriate, automated tests simulate multiple hosts and
test for host isolation (updates to one hosts's records do not affect
another.)
- [x] Manual QA for all new/changed functionality
Fixes 30636
I am adding a handful of additional unit tests but this is ready for
review now. Integrates changes from Victor's PoC for Account Driven User
Enrollment including a nice end to end integration test including the
SAML portion
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
> Fixes#29851
> Fixes#29902
> Mainly followups from https://github.com/fleetdm/fleet/pull/30295,
plus improved integration testing
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Adds support for the Apple MDM user channel however we are waiting on
stories around verification among other things for this and we are not
shipping as part of 4.70 so this can be reviewed but should not be
merged yet
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
For #26996 and #28452
Demo video: https://www.youtube.com/shorts/WGS3JmKiZTs
The device/machine info is extracted from the PKCS7 signed body of the
POST request.
I did manual QA on iPhone since I don't have an ADE macOS device with
me.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26713
# Details
This PR updates Fleet and its related tools and binaries to use Go
version 1.24.1.
Scanning through the changelog, I didn't see anything relevant to Fleet
that requires action. The only possible breaking change I spotted was:
> As [announced](https://tip.golang.org/doc/go1.23#linux) in the Go 1.23
release notes, Go 1.24 requires Linux kernel version 3.2 or later.
Linux kernel 3.2 was released in January of 2012, so I think we can
commit to dropping support for earlier kernel versions.
The new [tools directive](https://tip.golang.org/doc/go1.24#tools) is
interesting as it means we can move away from using `tools.go` files,
but it's not a required update.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet
- [x] Orbit runs on macOS ✅ , Linux ✅ and Windows.
- [x] Manual QA must be performed in the three main OSs, macOS ✅,
Windows and Linux ✅.
For #26623
- Updated `github.com/groob/plist` to `github.com/micromdm/plist` -- it
was renamed
- Added validation that restricts DigiCert Fleet variables to
`com.apple.security.pkcs12` payloads plus additional restrictions
- Added validation that restricts Custom SCEP Fleet variables to
`com.apple.security.scep` payloads plus additional restrictions
- Enabled multiple CAs (Fleet variables) to be present in an Apple MDM
profile. But each CA can only be used once. For example, we can have
DigiCert CA and Custom SCEP CA in one Apple profile.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #25581
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
for #21019
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
for #21019
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Part 2 of https://github.com/fleetdm/fleet/issues/19447
- iOS and iPadOS user-installed apps are loaded into Fleet
- Added an additional identifier into software_titles table to
differentiate between iOS/iPadOS apps
- Updated nano queue timestamp precision
Note: TestIntegrationsMDM/TestVPPApps fails when run as part of the
suite, but passes standalone. I'd like to proceed with merging this PR,
and figure out the issue next week.
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
`go-kit/kit/log` was deprecated and generating warnings
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
#18119
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
Sample on how to simulate 50 iPads and 50 iPhones:
```sh
go run ./cmd/osquery-perf -host_count 100 -os_templates iphone_14.6.tmpl:50,ipad_13.18.tmpl:50 -mdm_scep_challenge <...>
```
it delegates any extra unmarshaling to the caller. We might consider
building our own types in the future instead of relying on micromdm, but
these are used only for tests right now.
Following the pattern set up by Martin and Lucas for similar PRs. Can be
reviewed by commits:
1. move scep directory into monorepo
2. update import paths
3. update go.mod, go.sum
4. fix golint errors in scep package
5. skip a failing test that's been broken for a while
6. fix that failing test
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
Also verified that our test suite runs the `scep` tests.
#15557
Following the precedent that Lucas used for other similar PRs, the best
way to review is probably by commits.
* The first one simply copies over the files from the fork to the
monorepo
* Second one adjusts all import paths
* Third one tidies up the `go.mod` files
* Last one fixes the linter issues in the nanomdm package
# Checklist for submitter
- ~~Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.~~ (not a user-visible change)
- [x] Manual QA for all new/changed functionality (ran test suite,
re-generated mocks)
I also verified that our Go test suite did run the newly moved `nanomdm`
package steps:
```
ok github.com/fleetdm/fleet/v4/server/mdm/nanomdm/cryptoutil 0.003s coverage: 0.0% of statements in github.com/fleetdm/fleet/v4/...
ok github.com/fleetdm/fleet/v4/server/mdm/nanomdm/mdm 0.005s coverage: 46.2% of statements in github.com/fleetdm/fleet/v4/...
ok github.com/fleetdm/fleet/v4/server/mdm/nanomdm/service/certauth 1.320s coverage: 20.7% of statements in github.com/fleetdm/fleet/v4/...
ok github.com/fleetdm/fleet/v4/server/mdm/nanomdm/storage/file 0.007s coverage: 24.1% of statements in github.com/fleetdm/fleet/v4/...
```
for #14426.
In order to prevent import cycles and be able to use some type
definitions/constants I followed the same pattern we did for macOS by
creating a new package named `syncml`. This makes the changelog look
bigger than it actually is, so I split it into two commits to make it
easier to review:
-
[d7c233d](d7c233d54c)
moves the relevant bits to this new package
-
[7531a07](7531a0742b)
implements profile verification
#11528
osquery-perf simulated hosts enroll and are identified as manually
enrolled. (Enrolling as DEP requires more work, e.g. a new mocked Apple
DEP endpoint).
Given that these are simulated MDM clients, they cannot be woken up with
push notifications. Instead, these check for new commands to execute
every 10 seconds (which is not realistic, but could serve as a good
loadtesting exercise).
I will now start setting up the loadtest environment with MDM enabled
and configured to test this.
- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.~
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
