Changes to add `team_identifier` signing information to macOS
applications on the `/api/latest/fleet/hosts/:id/software` API endpoint.
Docs: https://github.com/fleetdm/fleet/pull/23743
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ X Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
---------
Co-authored-by: Tim Lee <timlee@fleetdm.com>
Co-authored-by: Ian Littman <iansltx@gmail.com>
> Related issue: #23247
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
# Checklist for submitter
The intention here should be to initialize a slice with a capacity of
totalTitlesToProcess rather than initializing the length of this slice.
The only demo: https://go.dev/play/p/q1BcVCmvidW
#20757
API endpoint `/api/v1/fleet/mdm/profiles/batch` will now not log an
activity for profile types that did not change in the database (Apple
configuration profiles, Windows configuration profiles, or Apple
declarations).
Demo video: https://www.loom.com/share/8b75cbd8e7394c12ac6b56746b72c244
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
#21427
Also fixed#21488
- Demo video:
https://www.loom.com/share/0a97afa680c74fa884ac07e87857098a
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality
Unreleased bug for iOS/iPadOS VPP #19447
# Checklist for submitter
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
Missing part of #19447
Added `available_for_install` query parameter to `GET
/api/v1/fleet/hosta/:id/software`
# Checklist for submitter
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#19447
iOS and iPadOS apps can be installed using Apple's VPP (Volume Purchase
Program)
VPP apps are now using a composite primary key (Adam ID and platform)
because we want to keep iOS/iPadOS/macOS separate. It is possible for
one app to be installable on all Apple platforms.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
Part 2 of https://github.com/fleetdm/fleet/issues/19447
- iOS and iPadOS user-installed apps are loaded into Fleet
- Added an additional identifier into software_titles table to
differentiate between iOS/iPadOS apps
- Updated nano queue timestamp precision
Note: TestIntegrationsMDM/TestVPPApps fails when run as part of the
suite, but passes standalone. I'd like to proceed with merging this PR,
and figure out the issue next week.
# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
> Related issue: #20662
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
during load testing we found this query to be a bottleneck, locally
splitting in to two different statements makes a difference since we can
effectively use the indexes.
```
mysql> explain UPDATE software s
-> JOIN software_titles st
-> ON COALESCE(s.bundle_identifier, '') = '' AND s.name = st.name AND s.source = st.source AND s.browser = st.browser
-> SET s.title_id = st.id
-> WHERE (s.title_id IS NULL OR s.title_id != st.id)
-> AND COALESCE(s.bundle_identifier, '') = '';
+----+-------------+-------+------------+-------+-------------------------------------------------------------------------------------+----------------------------+---------+------------------------------------------------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+-------------------------------------------------------------------------------------+----------------------------+---------+------------------------------------------------+------+----------+-------------+
| 1 | SIMPLE | st | NULL | index | idx_sw_titles | idx_sw_titles | 2302 | NULL | 765 | 100.00 | Using index |
| 1 | UPDATE | s | NULL | ref | software_listing_idx,software_source_vendor_idx,title_id,idx_sw_name_source_browser | idx_sw_name_source_browser | 2302 | fleet.st.name,fleet.st.source,fleet.st.browser | 1 | 91.00 | Using where |
+----+-------------+-------+------------+-------+-------------------------------------------------------------------------------------+----------------------------+---------+------------------------------------------------+------+----------+-------------+
2 rows in set (0.00 sec)
mysql> explain UPDATE software s
-> JOIN software_titles st
-> ON s.bundle_identifier = st.bundle_identifier
-> SET s.title_id = st.id
-> WHERE s.title_id IS NULL
-> OR s.title_id != st.id;
+----+-------------+-------+------------+------+-----------------------------------------------------+---------------------------------------+---------+---------------------------+------+----------+--------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+-----------------------------------------------------+---------------------------------------+---------+---------------------------+------+----------+--------------------------+
| 1 | UPDATE | s | NULL | ALL | title_id,idx_software_bundle_id | NULL | NULL | NULL | 788 | 100.00 | Using where |
| 1 | SIMPLE | st | NULL | ref | idx_software_titles_bundle_identifier,idx_composite | idx_software_titles_bundle_identifier | 1023 | fleet.s.bundle_identifier | 1 | 100.00 | Using where; Using index |
+----+-------------+-------+------------+------+-----------------------------------------------------+---------------------------------------+---------+---------------------------+------+----------+--------------------------+
2 rows in set (0.00 sec)
```
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
for #19144
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
> Related issue: #20050
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
`go-kit/kit/log` was deprecated and generating warnings
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
#19348
Fixed host details page and device details page not showing the latest
software.
- During software ingestion, software titles are now added if needed and
software items have their title_id field populated.
- In addition, after refreshing via UI, the software will be re-fetched
if it has been modified.
Added `exclude_software` query parameter to the
`/api/latest/fleet/hosts/:id` endpoint to exclude software from the
response.
PR for API doc change: #19617
Related issue filed for the Device User Page:
https://github.com/fleetdm/fleet/issues/19618
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#11942
To check if you have duplicate software, install multiple python
versions, like:
```shell
brew install python@3.11
brew install python@3.12
```
Then query: `SELECT * FROM python_packages` to see the packages with the
same versions but different paths.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
#18838
1. During software ingestion, switched to updating `last_updated_at` as
a batch for 1 host.
2. Removed `DELETE FROM software` statement that ran for every host
update (where software was deleted). This is only done now during the
vulnerability job.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
