For #23579
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
QA'd locally successfully. It just took a bit longer for the vuln showed
up.
#24548
This PR covers Apple legacy commands, Windows commands, and scripts.
Apple DDM commands and Software install/uninstall scripts will be
covered in separate PRs.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
If the `team_id` parameter is included the query report will filter the
hosts by the team id specified. The `team_id` parameter is included by
default from the front end queries pages.
https://github.com/fleetdm/fleet/issues/24006
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
## Addresses #24456
- host detail response (for Host details page and My device page)
excludes `mdm.os_settings` field if disk encryption isn't enabled for
the host
- confirmed it is still included when setting is enabled
- confirmed expected banner is still shown when setting enabled
<img width="2555" alt="Screenshot 2024-12-05 at 10 10 48 PM"
src="https://github.com/user-attachments/assets/e3852b7f-51ae-4e87-bceb-476ccdba2459">
- [x] Changes file added for user-visible changes in `changes/`,
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
for #24041
This PR addresses an issue that can cause Windows vulnerability checks
to fail (possibly causing false negatives). We determine whether a
vulnerability in an MSRC bulletin applies to any hosts in a Fleet
instance by attempting to matching the data in [each row of the
`operating_systems`
table](65e374c85c/cmd/fleet/cron.go (L297-L303))
with [at least one "product" in a
bulletin](e2d9a9016c/server/vulnerabilities/msrc/analyzer.go (L39)),
including [matching architecture and "display
version"](76f5baced9/server/vulnerabilities/msrc/parsed/product.go (L26-L39)).
However a subset of products listed in these bulletins do not include
the display version, so for example a host whose OS was listed as
`Microsoft Windows Server 2022 Datacenter 21H2` (21H2 being the "display
version") would match nothing in the bulletins because no listed Server
2022 products include "21H2" in their names.
The fix made here is to add relevant version info to the products list
when we do our ETL of the MSRC bulletins. The version info was gleaned
from https://en.wikipedia.org/wiki/List_of_Microsoft_Windows_versions.
We see logs related to this issue a lot, so cleaning this up will
alleviate some noise and infra costs as well.
For #22723.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
When a policy is deleted clean up any pending software installs or
scripts generated from the policy
https://github.com/fleetdm/fleet/issues/23886
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
For #24166
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#24725
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
FYI this was diagnosed and fixed using the
[RandoKiller](https://github.com/fleetdm/fleet/pull/24696).
---
This PR fixes the TestEnqueueMDMCommand, which has been failing
intermittently
[here](https://github.com/fleetdm/fleet/blob/main/server/service/integration_mdm_test.go#L2922).
Most of the time the `/api/latest/fleet/mdm/apple/commands` API is
returning one result as expected, but occasionally it returns 2, for
example:
```
[
{
"device_id": "B11F1FC1-F176-48CF-88A4-CB7A3DFEF987",
"command_uuid": "63bb4313-ccbf-4647-ac07-7d15df5f92d7",
"updated_at": "2024-12-12T02:41:36Z",
"request_type": "ProfileList",
"status": "Acknowledged",
"hostname": "test-host"
},
{
"device_id": "B11F1FC1-F176-48CF-88A4-CB7A3DFEF987",
"command_uuid": "7de9d712-7524-4443-a20a-7127e6064f6e",
"updated_at": "2024-12-12T02:41:36.141498Z",
"request_type": "InstallEnterpriseApplication",
"status": "Pending",
"hostname": "test-host"
}
]
```
It seems that the second command is related to trying to install a
bootstrap package (uploaded by a previous test) to the newly-enrolled
host.
The fix in this PR is to filter the API response to only the command
we're verifying the presence of. It's a decent solve, but leaves open
the edge case of a bug that causes multiple commands to be sent
unexpectedly. The ideal solution would be to remove the interaction
between the two tests, perhaps by deleting any created bootstraps before
those tests complete, or by re-initializing the state in some other way.
I don't currently have enough context to easily implement a solution
like that (i.e. I know there's a "delete bootstrap" API, but not sure if
that's enough to solve this issue).
#23816
This catches the case when host re-enrolls in MDM with pending profiles.
Demo of the issue/fix: https://youtu.be/ol3xbJWw8HQ
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
For #24509
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
## Addresses #21855 and all of its subtasks
**Frontend:**
- Update list queries API call to include pagination and filter-related
query params, including new `platform` param for filtering queries by
platforms they've been set to target
- Convert all filtering, sorting, and pagination functionality of the
Manage queries page from client-side to server-side
- Remove unneeded variable declarations / logic
- Various typing and naming improvements
**Server:**
- Add new `platform` `ListQueryOption`
- Update service and datastore level list queries logic to handle
filtering queries by targeted platform
- Update service and datastore level list queries logic to include
`meta` and `count` fields in addition to filtered/paginated queries
- [x] Changes file added for user-visible changes in `changes/`, `
- [x] Added/updated tests
- [x] update DB, integration
- [x] add integration (pagination)
- [x] add integration (platform filter)
- [x] add DB (pagination)
- [x] add DB (platform filter)
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
> Related issue: #24120
## Changes
- Added the `software_title_id` field to the activity details for
`added_software` activities, which get generated when adding a customer
installer or a FMA
- Added a return value (`titleID`) to
`ds.MatchOrCreateSoftwareInstaller`
- Removed `ds.GetSoftwareTitleIDByMaintainedAppID`. Since we're
returning the new value above, this method was no longer needed.
## Testing steps
1. Add a custom installer
2. Add a FMA
3. Check the activity details in the response to `GET /activities`.
Verify that the `software_title_id` field exists and is correct.
4. Add a FMA with automatic install. Make sure the policy is correctly
created, has the correct software title ID associated with it, and that
it installs the app.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#24545
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
The software versions endpoint cve details can be truncated using the
`without_vulnerability_details` flag.
https://github.com/fleetdm/fleet/issues/23679
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
#23760
Manual QA done:
1. Get the current APNS key using tools/mdm/assets tool.
2. Convert it to a PKCS8 key like: `openssl pkcs8 -topk8 -inform PEM
-outform PEM -in mdm_assets/apns_key.key -out mdm_assets/apns_pkcs8.key
-nocrypt`
3. Delete all `apns` and `scep` rows from `mdm_config_assets` DB table.
4. Point to the PKCS8 key like:
```
export FLEET_MDM_APPLE_APNS_KEY=/Users/victor/work/fleet/mdm_assets/apns_pkcs8.key
export FLEET_MDM_APPLE_APNS_CERT=/Users/victor/work/fleet/mdm_assets/apns_cert.crt
export FLEET_MDM_APPLE_SCEP_CERT=/Users/victor/work/fleet/mdm_assets/ca_cert.crt
export FLEET_MDM_APPLE_SCEP_KEY=/Users/victor/work/fleet/mdm_assets/ca_key.key
export FLEET_MDM_APPLE_SCEP_CHALLENGE=$(cat /Users/victor/work/fleet/mdm_assets/scep_challenge)
```
This step may be tricky. Might be simpler to spin up a fresh Fleet
server with the SCEP/APNS values.
5. Start fleet server. Make sure APNS works by pushing/deleting a
profile.
6. Renew APNS cert and make sure the new cert still works.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
For #23488.
We see duplicates for queries that show up in both WHEREs since UNION
ALL doesn't deduplicate. Since we're grabbing all of the same columns,
GROUP BY'ing all columns on the final result gets us a deduplicated set
without having to do any cleanup server-side.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
For consistency with new MFA email in #22078.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
For #23305.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
> Related issue: #24204
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#23816
This fix may not completely fix the customer's issue. However, I'd like
to see if there are improvements from this fix combined with the
previous query optimization fix.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
#22790
Changes file is on the FE PR.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
> Related issue: #23982
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
> Related issue: #22077
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
for #24131
This PR fixes an issue that can occur when reconciling entries in the
`software` and `software_titles` tables. This can cause
vulnerability-scanning cron jobs to exit early. The error is:
```
upsert software titles: Error 1869 (HY000): Auto-increment value in UPDATE conflicts with internally generated values
```
I haven't had any luck reproducing this locally; the relevant query
seems to cover all the bases as far as ensuring there are no duplicates.
I suspect it may be due to data in these instances that existed prior to
the introduction of the `additional_identifier` column in
`software_titles`. In any case, replacing the `id = id` no-op with a
clause that updates non-auto-incrementing fields is the standard
practice here (in fact it's done in other ON DUPLICATE KEY UPDATE
queries in the same file).
There are automated tests for `ReconcileSoftwareTitles`. I'd gladly add
one to test this case if I could figure out how to reliably cause it,
but if that involves altering the database to put in bad data, it's so
far from the happy path that it's probably not worth it.
