Commit graph

2676 commits

Author SHA1 Message Date
Dante Catalfamo
0a15647e10
Host software deleted at remigration (#20996)
# Recreate out of order migration, replace `docker-compose` with `docker compose` in db test runner
2024-08-02 10:47:40 -04:00
Tim Lee
e3c3c870c9
Ignore Rejected CVEs Test (#19974)
#18913

This is a test to ensure Rejected CVEs do not match against software.
The related PR needs to be merged first, otherwise this test will fail:

https://github.com/fleetdm/fleet/pull/19972

- [X] Added/updated tests
2024-08-02 07:33:57 -03:00
Dante Catalfamo
bf3d849bd0
Fix Deleted Host Software Installs Details (#20820)
#20271

This fix requires an accompanying front end fix to pass the
`display_name` from the activity feed to the modal when displaying the
install results.
2024-08-01 15:41:37 -04:00
Dante Catalfamo
7abae84be5
Use gitops format for software installer query (#20891)
#20747
2024-08-01 14:36:40 -04:00
Victor Lyuboslavsky
580a81ded5
Fixed duplicate app install activity on host (#20944)
Unreleased bug for iOS/iPadOS VPP #19447

# Checklist for submitter
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-08-01 20:32:45 +02:00
Roberto Dip
30cbaf0a74
address PR feedback for FileVault improvements (#20935)
feedback left by @mna and @gillespi314 in
https://github.com/fleetdm/fleet/pull/20842

also fixes a bug found by @PezHub
https://github.com/fleetdm/fleet/issues/13157#issuecomment-2261615471

related to https://github.com/fleetdm/fleet/issues/13157

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-08-01 14:04:54 -03:00
Roberto Dip
7a4773cd0a
escape % in query (#20954)
for https://github.com/fleetdm/fleet/issues/20940. Ran `make
generate-doc` as well but docs don't change with this.
2024-08-01 13:56:09 -03:00
Georg Schölly
a41fb636bc
Detect software from deb_packages which is 'on hold' (#20751)
The list of installed software was missing packages put ['on
hold'](https://askubuntu.com/questions/18654/how-to-prevent-updating-of-a-specific-package)

The reason for this is that the old query looks for the status

    install ok installed

but there are other valid status which are also installed, like `hold ok
installed`. The syntax is `<desired> <error> <status>` so we only need
to look at the last or two last parts and ignore the first one.

See https://man7.org/linux/man-pages/man1/dpkg-query.1.html for a list
of status.

# Checklist for submitter

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
2024-08-01 11:39:01 -03:00
Roberto Dip
7a080a9b36
use Escrow Buddy to rotate FileVault keys on macOS (#20842)
back-end and agent part of #13157

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-07-31 16:59:30 -03:00
Sarah Gillespie
a33bd65272
Revert "Add minimum os version requirements to DEP enrollment flow" (#20878) 2024-07-31 11:34:01 -05:00
Victor Lyuboslavsky
438b0ff4d6
Added available_for_install query parameter to host details (#20876)
Missing part of #19447
Added `available_for_install` query parameter to `GET
/api/v1/fleet/hosta/:id/software`

# Checklist for submitter

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-31 16:27:43 +02:00
Tim Lee
bda812c329
Hotfix: Pass sql test dump in via stdin (#20871) 2024-07-31 05:09:52 -06:00
Victor Lyuboslavsky
31224ccd34
Added VPP support for iOS/iPadOS (#20837)
#19447 
iOS and iPadOS apps can be installed using Apple's VPP (Volume Purchase
Program)

VPP apps are now using a composite primary key (Adam ID and platform)
because we want to keep iOS/iPadOS/macOS separate. It is possible for
one app to be installable on all Apple platforms.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-07-30 22:43:51 +02:00
Tim Lee
0e9705fdf7
Add No Team to Software Backend (#20822) 2024-07-30 11:19:05 -06:00
Sarah Gillespie
53ba476516
Fix unreleased bug: API error message for duplicate VPP apps (#20808) 2024-07-29 15:46:27 -05:00
Sarah Gillespie
0f98e84bc8
Add minimum os version requirements to DEP enrollment flow (#20722) 2024-07-29 11:02:27 -05:00
Martin Angers
cc4c34ce8a
Unreleased bug fix: use of db.writer inside a transaction (#20803) 2024-07-29 11:56:01 -04:00
Victor Lyuboslavsky
66f4af0ada
Calendar event body is kept up to date (#20316)
#19280

# Video

https://www.loom.com/share/0f19e9be9a5d4022900d93a9779f850d?sid=f8b33c1f-ea64-4f4b-9f64-296dbc01614e

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-29 12:04:40 +02:00
Victor Lyuboslavsky
671fc628e7
Fetch installed apps from iPhone/iPad devices. (#20733)
Part 2 of https://github.com/fleetdm/fleet/issues/19447
- iOS and iPadOS user-installed apps are loaded into Fleet
- Added an additional identifier into software_titles table to
differentiate between iOS/iPadOS apps
- Updated nano queue timestamp precision

Note: TestIntegrationsMDM/TestVPPApps fails when run as part of the
suite, but passes standalone. I'd like to proceed with merging this PR,
and figure out the issue next week.

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-07-28 16:17:27 +02:00
Jahziel Villasana-Espinoza
dd3967c4ec
feat: enforce FileVault during Setup Assistant (#20759)
> Related issue: #16866

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-26 15:50:31 -04:00
Jahziel Villasana-Espinoza
5b97d01982 chore: merge main 2024-07-24 16:50:43 -04:00
Victor Lyuboslavsky
90a1ac9faa
iOS and iPadOS device details refetch (#20678)
Part 1 of #19447
- iOS and iPadOS device details refetch can now be triggered with the
existing `POST /api/latest/fleet/hosts/:id/refetch` endpoint

# Checklist for submitter

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-24 19:46:24 +02:00
Roberto Dip
544d5b20c4
increase Apple SCEP renewal period to 180 days (#20697)
related to #19684

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-24 14:42:53 -03:00
Jahziel Villasana-Espinoza
fda37851c1
fix: move last install data into apps and packages (#20664)
> Related issue: #20662

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-24 13:39:23 -04:00
Lucas Manuel Rodriguez
70d4558448
Backend support for iOS/iPadOS OS updates (#20649)
#20469 and #20471

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-07-24 14:34:23 -03:00
Tim Lee
362a0e545e
18913 ignore rejected NVD vulnerabilities (#20193)
#18913 Recreating PR (ref: https://github.com/fleetdm/fleet/pull/19972)
2024-07-24 10:53:33 -06:00
Victor Lyuboslavsky
c1a5e3b7b6
Fix calendar duplicated events and other issues (#20443)
#19352
Includes the following changes:
- Re-enable calendar callback
- Introduced a new Redis key that indicates event was updated by
calendar callback. In that case, we ignore subsequent callbacks for 10
seconds.
- This reduces the amount of Google API calls, including handling of the
unneeded callback generated by our own event change.
- Read event from DB after acquiring lock. This is critical since we get
the updated ETag of the Google Calendar event from our DB. Using the
previous ETag when fetching event sometimes returns stale data,
resulting in duplicate events.
- Fixed bug in getCalendarLock where calendar cron would always think it
got the lock
- Do not refetch timezone during calendar callback to reduce Google API
load
- Watch for calendar event changes for 1 week after event end (to
account for user moving event into the future)
- #20442: Speculative improvement for Google callback latency by keeping
the same notification channel (callback URL).
- processCalendarAsync now takes at least 1 sec to process all events,
to reduce CPU/Redis load
- Increased lock expiration time from 1 minute to 20 minutes to account
for potential Google API retries, fixing occasional duplicate events.
- Added `get-events.go` helper script that gets maintenance events from
user calendars, and checks for duplicates

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-24 13:40:33 +02:00
Roberto Dip
e4d8fcc3a5
fix: adjust host filters for VPP software (#20663)
this allows to filter hosts with pending, failed and installed VPP apps.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-23 16:56:24 -03:00
Roberto Dip
845bfc7e38
rearrange migrations and generate dump (#20652)
this re-arranges migrations so the VPP schema changes happen after
already merged migrations and commits the result of running `make
dump-test-schema` using mysql 8.

note that there are a bunch of seemingly unrelated changes in the diff,
but that's just the mysql 8 bump, the diff against `main` should be
smaller.
2024-07-23 11:01:23 -03:00
Jahziel Villasana-Espinoza
e3e7b6a115 chore: merge main 2024-07-22 17:25:42 -04:00
Jahziel Villasana-Espinoza
b449ef35fc
fix: return specified errors for VPP operations (#20640)
> Related issue: #20229

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-22 17:14:54 -04:00
Dante Catalfamo
5e1a3d03ae
MySQL 8.0 Migration (#20225)
#17249
2024-07-22 16:27:36 -04:00
Jahziel Villasana-Espinoza
746c442d01
feat: list host software integration tests (#20632)
> Related issue: #20229

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-22 16:11:07 -04:00
Lucas Manuel Rodriguez
1a512b2792
Do not set last_enrolled_at when re-enrolling orbit (#20620)
Server fix for #20397. (I incorrectly added that `last_enrolled_at =
NOW()` on the first iOS/iPadOS support PR.)

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests.
- [ ] Manual QA for all new/changed functionality.
2024-07-22 14:26:03 -03:00
Dante Catalfamo
9ec52cea9c
VPP Batch API (#20351)
#20278
2024-07-22 13:19:19 -04:00
jacobshandling
2282c495c2
Correctly handle NULL timezones (#20605)
## Addresses #20618 
- Update `CalendarEvent.TimeZone` type to be `*string`
- Consider any Fleet calendar event with `NULL` timezone to need an
update, and write the latest google calendar timezone
- [x] Changes file added for user-visible changes in `changes/`, 
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-07-22 10:04:29 -07:00
Gabriel Hernandez
446abc2dbc Merge branch 'main' into feat-vpp-apps-18867 2024-07-22 11:46:14 +01:00
Jahziel Villasana-Espinoza
b35724bd30
fix: store the VPP token encoded (#20606)
> Related issue: part of #20229

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-07-19 18:13:01 -04:00
Jahziel Villasana-Espinoza
caa7fd74e4
fix: remove temporary functions in tests (#20615)
> Related issue: #20229

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-19 17:10:03 -04:00
Sarah Gillespie
80faef8327
Set source = 'apps' when creating software titles for vpp apps (#20622)
Issue #20229
2024-07-19 15:35:10 -04:00
Lucas Manuel Rodriguez
fee0744a66
Perform early restart before starting sub-systems and minor refactor of orbit sub-systems (#20610)
#20397

Am calling these things that orbit run "sub-systems".

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-07-19 12:44:43 -03:00
Gabriel Hernandez
b5f64b7d95 Merge branch 'main' into feat-vpp-apps-18867 2024-07-19 14:33:56 +01:00
Roberto Dip
97cfaebe3a
add retries to iTunes API (#20602)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->


- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-18 15:54:34 -07:00
Sarah Gillespie
f5296ab400
Fix unreleased issues in VPP feature branch (#20590) 2024-07-18 17:39:19 -05:00
Roberto Dip
87f9a9a3e7
feat: VPP app installation flow (#20448)
> Related issue: #19868 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
2024-07-18 18:35:26 -04:00
Martin Angers
b8b03b1e5a
VPP: update list software titles/list host's software response payloads (#20553)
#20536 

# Checklist for submitter

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-07-18 14:33:07 -07:00
Tim Lee
a47db8bbaf
Disable calendar callbacks (#20547) 2024-07-17 15:27:03 -06:00
Martin Angers
e115516e8c
VPP: fix matching software title to existing one if one exists (#20534) 2024-07-17 15:41:51 -04:00
Martin Angers
74b1dc4f0d
VPP: Add PK to vpp_apps_teams to avoid migration issues (#20542) 2024-07-17 15:22:22 -04:00
Martin Angers
1c85c34216
VPP: fix download package path, implement delete VPP app support (#20530) 2024-07-17 14:19:13 -04:00