Builds on #27080 / #32133. Shows disk space if we can calculate it, otherwise, shows 'Not supported'. Excludes unsupported hosts from low disk space filter.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33250
Waived most new failures. Planning to come back and fix some of them in
subsequent PRs.
fixes: #29222
This is a feature branch that was completed last week, but did not get
merged in time.
All pr's going in was approved, and reviewed.
I will after this is merged, do a cherry pick onto the RC 4.73 branch,
and initiate the FR merge process.
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Fixes#31353. Adds private key validation to Android MDM enterprise signup to prevent failed Android enablement when server private key is not configured.
Fixes#31444
The changes are primarily in tests. The only changes in production code
are a couple validations/checks for invalid values in:
- mysql/apple_mdm.go
- mysql/hosts.go
- mysql/queries.go
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved handling of timestamp and default values across various
features to prevent database errors and warnings.
* Enhanced validation and data consistency for Apple Business Manager
tokens and MDM profiles.
* Updated test data and logic to comply with stricter database
constraints and realistic scenarios, including date handling and field
lengths.
* **Chores**
* Updated test setups to reflect schema changes, improve data integrity,
and avoid future compatibility issues.
* Standardized SQL mode and timestamp usage in test environments.
* Refined test data for VPP apps, software installers, and device
enrollments for better reliability.
* **Tests**
* Expanded and updated tests to cover new fields, stricter validation,
and more accurate simulation of real-world conditions.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For https://github.com/fleetdm/confidential/issues/9931.
[Here](ec3e8edbdc/docs/Contributing/Testing-and-local-development.md (L339))'s
how to test SAML locally with SimpleSAML.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Improved SSO and SAML integration with enhanced session management
using secure cookies.
* Added support for IdP-initiated login flows.
* Introduced new tests covering SSO login flows, metadata handling, and
error scenarios.
* **Bug Fixes**
* Enhanced validation and error handling for invalid or tampered SAML
responses.
* Fixed session cookie handling during SSO and Apple MDM SSO flows.
* **Refactor**
* Replaced custom SAML implementation with the crewjam/saml library for
improved reliability.
* Simplified SAML metadata parsing and session store management.
* Streamlined SSO authorization request and response processing.
* Removed deprecated fields and redundant code related to SSO.
* **Documentation**
* Updated testing and local development docs with clearer instructions
for SSO and IdP-initiated login.
* **Chores**
* Upgraded dependencies including crewjam/saml and related packages.
* Cleaned up tests and configuration by removing deprecated fields and
unused imports.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #26519
This PR allows Fleet server to use Android with either fleetdm.com proxy
or locally. It also removes the Android feature flag from the backend.
The frontend changes and proxy API documentation will be in separate
PRs.
Updated contributor docs:
https://github.com/fleetdm/fleet/pull/29880/files
Integration tests are missing and tracked as a separate issue:
https://github.com/fleetdm/fleet/issues/27080
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26219
Refactoring the interface between Android service and Android datastore
to use 1 common datastore interface: `fleet.AndroidDatastore`
These changes are based on feedback from the recent Backend Sync.
```mermaid
---
title: Partial class diagram
---
classDiagram
direction LR
class `android.Service`
<<interface>> `android.Service`
class `android/service.Service`
`android/service.Service` ..|> `android.Service`: implements
class `fleet.AndroidDatastore`
<<interface>> `fleet.AndroidDatastore`
class `fleet.Datastore`
<<interface>> `fleet.Datastore`
class `android.Datastore`
<<interface>> `android.Datastore`
`android/service.Service` *-- `fleet.AndroidDatastore`: USES (THIS IS THE KEY CHANGE)
`fleet.Datastore` *-- `fleet.AndroidDatastore`: contains
`mysql.Datastore` ..|> `fleet.Datastore`: implements
`fleet.AndroidDatastore` *-- `android.Datastore`: contains
`mysql.Datastore` *-- `android.Datastore`: contains
`android/mysql.Datastore` ..|> `android.Datastore`: implements
```
For #26736
adds a self closing page that is the final part of connecting with
Android Enterprise. This allows the user to see the fleet page
notification that android mdm is now enabled.
For #26219
Frontend wasn't displaying the right Android page due to missing
details.
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
For #26218
- Added signup_token authentication for Android enterprise callback and
fixed API path to match API doc
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
This is a dev helper feature (since it will be implemented in
fleetdm.com). It fixes the issue of stale PubSub topics. Having stale
PubSub topics around may cause extra traffic/errors on dev server.
- [x] Manual QA for all new/changed functionality
For #26219
Need to get in this security change for the RC
- Added auth (token stored in `mdm_config_assets`) for PubSub endpoint
- unknown notification types, including `test` are not authenticated
- Status reports for deleted devices are not authenticated -- no DB
changes are done for these since they only happen when we disable MDM
right now
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
For #26218
- Added `users_deleted` table to track user actions if the user was
actually deleted.
- Added enable/disable Android MDM activities
Note: I could not auto-generate fleet.Service mock because it has issues
with methods that don't return anything. I ended up using testify mock
instead.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
- Added `GET /api/_version_/fleet/android_enterprise/signup_sse`
endpoint and tests
- Fixed up handling of Android status reports with a deleted device. We
don't actually expect this to happen in production since the proxy
should delete the pubSub connection when the enterprise is deleted.
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
- Added `GET /api/_version_/fleet/android_enterprise` andpoint and tests
- Set up some testing infrastructure for Android service tests -- see
new README.md
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26219
Video explaining the PR: https://www.youtube.com/watch?v=t2hmpE7N-50
This PR includes:
- applying a default/dummy policy to Android enterprise
- Setting up PubSub topic and subscription to point at Fleet Server
- Generating device enrollment token
- Enrolling a device
Although some tests were added, this PR is mostly manually tested.
Integration tests are an outstanding task.
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #26218
Refactoring android/mysql/testing_utils.go to remove duplication with
main mysql package.
No functional changes.
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
For #26218
Basic Android MDM on/off backend functionality. Manually tested.
The following env vars must be set:
```
FLEET_DEV_ANDROID_ENABLED=1
FLEET_DEV_ANDROID_SERVICE_CREDENTIALS=$(cat credentials.json)
FLEET_DEV_ANDROID_PUBSUB_TOPIC=projects/your-project/topics/your-topic
```
I picked https://github.com/go-json-experiment/json as the JSON library,
which seems like the safest option.
- will become json/v2 at some point
- currently used in production by other companies, like Tailscale
- well-maintained
- Some context here: https://github.com/fleetdm/fleet/issues/25512
Plan for next work:
- refactoring from 1st PR
- add pubsub with device enroll -> spec proxy for fleetdm.com
- come back to this sub-task to add tests and finish TODOs
# Checklist for submitter
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
Android scaffold code and refactorings
- Android packages intended to be decoupled from other Fleet code
Video explaining the PR: https://www.youtube.com/watch?v=cza-35Z9Wxk
# Checklist for submitter
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
