Commit graph

2637 commits

Author SHA1 Message Date
Jahziel Villasana-Espinoza
caa7fd74e4
fix: remove temporary functions in tests (#20615)
> Related issue: #20229

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-19 17:10:03 -04:00
Sarah Gillespie
80faef8327
Set source = 'apps' when creating software titles for vpp apps (#20622)
Issue #20229
2024-07-19 15:35:10 -04:00
Gabriel Hernandez
b5f64b7d95 Merge branch 'main' into feat-vpp-apps-18867 2024-07-19 14:33:56 +01:00
Roberto Dip
97cfaebe3a
add retries to iTunes API (#20602)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->


- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-18 15:54:34 -07:00
Sarah Gillespie
f5296ab400
Fix unreleased issues in VPP feature branch (#20590) 2024-07-18 17:39:19 -05:00
Roberto Dip
87f9a9a3e7
feat: VPP app installation flow (#20448)
> Related issue: #19868 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
2024-07-18 18:35:26 -04:00
Martin Angers
b8b03b1e5a
VPP: update list software titles/list host's software response payloads (#20553)
#20536 

# Checklist for submitter

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-07-18 14:33:07 -07:00
Tim Lee
a47db8bbaf
Disable calendar callbacks (#20547) 2024-07-17 15:27:03 -06:00
Martin Angers
e115516e8c
VPP: fix matching software title to existing one if one exists (#20534) 2024-07-17 15:41:51 -04:00
Martin Angers
74b1dc4f0d
VPP: Add PK to vpp_apps_teams to avoid migration issues (#20542) 2024-07-17 15:22:22 -04:00
Martin Angers
1c85c34216
VPP: fix download package path, implement delete VPP app support (#20530) 2024-07-17 14:19:13 -04:00
Martin Angers
75bdc05f5d
VPP: unify List host's software and List my device's software response payload (#20519) 2024-07-16 16:52:04 -04:00
jacobshandling
22a9eb7d60
Include timezone write when updating events; Write updated gcal timezone if only event change (#20435)
## Addresses #20431 

https://www.loom.com/share/0d88eceb8fb44ef3bec70d2b0dc7479c?sid=350bb4c2-2abe-4b80-b99f-ef6c8109efac

- Include timezone write when updating events
- Write updated gcal timezone to Fleet events, even if it's the only
change
- Have frontend handle `"UTC"` being set as timezone as if it were `nil`
- Small cleanups
 
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-07-16 13:27:33 -07:00
Martin Angers
0f9a549ed4
VPP: add VPP apps to list host's software endpoint (#20483) 2024-07-16 16:18:44 -04:00
Sarah Gillespie
7dda2e9f84
Revert changes to DEP enrollment flow (#20506)
Reverts #20162 and #20355 per [QA
findings](https://github.com/fleetdm/fleet/issues/19185#issuecomment-2229544188)
2024-07-16 11:44:48 -05:00
Roberto Dip
5ea213e875
improve VPP API error handling (#20446)
reading the [docs][1] I realized we're missing some recommendations for
error management.

the docs also note that certain operations like assignments happen
asynchronously and you must subscribe to events to get those errors.

this part wasn't estimated nor considered.

[1]:
https://developer.apple.com/documentation/devicemanagement/app_and_book_management/handling_error_responses

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
2024-07-16 12:16:00 -04:00
Jahziel Villasana-Espinoza
5d2e40bc8b
feat: backend for VPP related global activities (#20484)
> Related issue: #19870 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-16 10:51:08 -04:00
Martin Angers
464c248f30
VPP: List/Get software title endpoints to return VPP apps (#20445) 2024-07-15 15:06:30 -04:00
Victor Lyuboslavsky
5cc5e527e5
Fixing Homebrew intellij/pycharm CE false negative. (#20461)
#19853
Fixing Homebrew intellij/pycharm CE false negative.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-15 12:47:10 -05:00
Tim Lee
60c45c3d7d
HotFix err on script timeouts for legacy orbit versions (#20462) 2024-07-15 11:17:20 -06:00
Victor Lyuboslavsky
dc7a3cd827
Stop channel bug fix and reload frequency (#20425)
Unreleased bug fix for #19352
2024-07-15 09:25:44 -05:00
Roberto Dip
f20e4037b2
send an APNs push to hosts with pending commands every minute (#19941)
Initial naive implementation that sends push notifications to hosts with
pending enqueued commands every 1 minute.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-12 15:39:59 -03:00
Tim Lee
78ec0b5254
Hotfix - Correct Async Run Script Validation Test (#20405) 2024-07-12 09:55:12 -06:00
Victor Lyuboslavsky
df78d83e2b
Fix TestIntegrations/TestGetHostSoftwareUpdatedAt (#20400)
Fix failing test `TestIntegrations/TestGetHostSoftwareUpdatedAt`
2024-07-11 16:42:14 -05:00
Tim Lee
80b11d873d
Part 2 of 2: Script Timeout Agent Options (#20356) 2024-07-11 15:03:36 -06:00
James Kane
4e0c447daa
Add "exclude software" parameter to get host by identifier (#20133)
For #19540

Just added the same "exclude_software" functionality that exists in "get
hosts" to the "get host by identifier" function.

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Manual QA for all new/changed functionality
2024-07-11 15:48:15 -05:00
Jahziel Villasana-Espinoza
aa1645628d
feat: get app store apps, add app store app to Fleet (#20362)
> Related issue: #19867 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
2024-07-11 16:09:30 -04:00
Roberto Dip
a410f330b9
additional fix for loadtesting issues for bundle_identifier (#20377)
previous work at https://github.com/fleetdm/fleet/pull/20354
2024-07-11 14:33:20 -03:00
Roberto Dip
bf6e506c50
bypass not found mdm commands (#20369)
for #20367

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-11 11:40:48 -03:00
Lucas Manuel Rodriguez
b8479fa9cd
Fix last_enrolled_at for macOS devices when re-enrolling to MDM (#20173)
#20059

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [X] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [X] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [X] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-07-10 21:40:40 -03:00
Sarah Gillespie
375c1bc319
Add index on host_uuid to mdm_idp_accounts table (#20355) 2024-07-10 16:39:46 -05:00
Roberto Dip
4d474a153c
separate queries for software matching to use indexes (#20354)
during load testing we found this query to be a bottleneck, locally
splitting in to two different statements makes a difference since we can
effectively use the indexes.

```
mysql> explain UPDATE software s
    -> JOIN software_titles st
    -> ON COALESCE(s.bundle_identifier, '') = '' AND s.name = st.name AND s.source = st.source AND s.browser = st.browser
    -> SET s.title_id = st.id
    -> WHERE (s.title_id IS NULL OR s.title_id != st.id)
    -> AND COALESCE(s.bundle_identifier, '') = '';
+----+-------------+-------+------------+-------+-------------------------------------------------------------------------------------+----------------------------+---------+------------------------------------------------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys                                                                       | key                        | key_len | ref                                            | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+-------------------------------------------------------------------------------------+----------------------------+---------+------------------------------------------------+------+----------+-------------+
|  1 | SIMPLE      | st    | NULL       | index | idx_sw_titles                                                                       | idx_sw_titles              | 2302    | NULL                                           |  765 |   100.00 | Using index |
|  1 | UPDATE      | s     | NULL       | ref   | software_listing_idx,software_source_vendor_idx,title_id,idx_sw_name_source_browser | idx_sw_name_source_browser | 2302    | fleet.st.name,fleet.st.source,fleet.st.browser |    1 |    91.00 | Using where |
+----+-------------+-------+------------+-------+-------------------------------------------------------------------------------------+----------------------------+---------+------------------------------------------------+------+----------+-------------+
2 rows in set (0.00 sec)

mysql> explain UPDATE software s
    -> JOIN software_titles st
    -> ON s.bundle_identifier = st.bundle_identifier
    -> SET s.title_id = st.id
    -> WHERE s.title_id IS NULL
    -> OR s.title_id != st.id;
+----+-------------+-------+------------+------+-----------------------------------------------------+---------------------------------------+---------+---------------------------+------+----------+--------------------------+
| id | select_type | table | partitions | type | possible_keys                                       | key                                   | key_len | ref                       | rows | filtered | Extra                    |
+----+-------------+-------+------------+------+-----------------------------------------------------+---------------------------------------+---------+---------------------------+------+----------+--------------------------+
|  1 | UPDATE      | s     | NULL       | ALL  | title_id,idx_software_bundle_id                     | NULL                                  | NULL    | NULL                      |  788 |   100.00 | Using where              |
|  1 | SIMPLE      | st    | NULL       | ref  | idx_software_titles_bundle_identifier,idx_composite | idx_software_titles_bundle_identifier | 1023    | fleet.s.bundle_identifier |    1 |   100.00 | Using where; Using index |
+----+-------------+-------+------------+------+-----------------------------------------------------+---------------------------------------+---------+---------------------------+------+----------+--------------------------+
2 rows in set (0.00 sec)
```

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-07-10 18:16:38 -03:00
Tim Lee
5ca22df90c
Script Timeout Agent Options Part 1 of 2 (#20266) 2024-07-10 14:33:39 -06:00
Sarah Gillespie
2425f98e92
Improve device user account creation during MDM IdP enrollment flow (#20162) 2024-07-10 14:48:23 -05:00
Dante Catalfamo
51e2979965
VPP GitOps Config (#20238)
Currently covers the ability to sync and verify config with fleet
server. Bulk API moved to its own ticket (#20278) while product decides
its capabilities
2024-07-10 14:53:03 -04:00
Roberto Dip
adc608ea6c
add bundle_identifier to software titles responses (#20298)
final bit for #19144, doc changes will be made separately

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-10 12:25:10 -03:00
Victor Lyuboslavsky
303c0523a0
Ignore -1 hardware_serial (#20279)
#19789
When osquery returns a serial number of -1 (default value), we keep the
existing serial number in the database.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-10 09:01:25 -05:00
Victor Lyuboslavsky
7bcd61a8bd
Distributed lock and store calendar_events UUID as binary in MySQL (#20277)
#19352

Fix for code review comment:
https://github.com/fleetdm/fleet/pull/20156#discussion_r1668421504

Also includes changes from https://github.com/fleetdm/fleet/pull/20252

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-07-10 08:49:05 -05:00
Roberto Dip
886ab9098d
Allow team_id=0 to signal "No Team" in os_versions (#20272)
for #20150

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-10 10:12:05 -03:00
Martin Angers
0945ef9889 Merge main and fix conflicts 2024-07-10 09:08:08 -04:00
Martin Angers
0913f77bdd Merge main and fix conflicts 2024-07-10 08:55:02 -04:00
Tim Lee
2ea369c093
Custom Vulnerability Matching (#20118) 2024-07-09 11:50:22 -06:00
jacobshandling
ec11e3d1d0
fleetctl, API, copy updates around host identifiers (#20220)
## Addresses #19127 
![Screenshot 2024-07-08 at 4 49
33 PM](https://github.com/fleetdm/fleet/assets/61553566/b4704eb9-9707-4cbf-8959-ec67dde57103)
- Also replace all ocurrences of "comma separated" with
"comma-separated"

- [x] Changes file added for user-visible changes in `changes/`
- [x] `SELECT *` is avoided, SQL injection is prevented (using
placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-07-09 10:25:01 -07:00
Tim Lee
812140a760
Add createdAt to Vulnerability responses (#20019) 2024-07-09 11:09:16 -06:00
Tim Lee
a3c44688c8
Get MDM Command Filters (#20154) 2024-07-09 11:06:06 -06:00
Lucas Manuel Rodriguez
2875a9dbb8
Fixes to fleetctl debug connection and TLS certs documentation (#20166)
#6085

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-09 14:04:23 -03:00
Roberto Dip
e90b90d905
Match macOS software <-> titles using bundle identifier (#19969)
for #19144

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2024-07-09 13:43:21 -03:00
Martin Angers
1cbd186a66 Merge main and fix conflicts 2024-07-09 10:54:19 -04:00
Jahziel Villasana-Espinoza
57fa67c8c0
fix: filter by vulnerable software (#20241)
> Related issue: #20050

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
2024-07-09 10:02:49 -04:00
Roberto Dip
cc13d3ce98
combine osquery + mdm information for connected_to_fleet (#20230)
for #20057

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-09 10:48:33 -03:00