Commit graph

262 commits

Author SHA1 Message Date
Victor Lyuboslavsky
a452e01bb2
Changed activities.created_at timestamp precision to microseconds (#19848)
#19845 
Changed `activities.created_at` timestamp precision to microseconds

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
2024-06-18 15:37:08 -05:00
Jahziel Villasana-Espinoza
2a0de89797
fix: allow profiles larger than 65KB (#19829)
> Related issue: #19771 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-06-18 13:02:00 -04:00
Victor Lyuboslavsky
2350ffdf49
Added host_issues (#19654)
* /api/latest/fleet/hosts now returns `critical_vulnerabilities_count`
for premium users. This data is held in the new `host_issues` table. The
failing policies total is updated in real-time, while the critical
vulnerabilities total is updated every hour after vulnerabilities job.
* /api/latest/fleet/hosts can be sorted by total_issues_count by
specifying `order_key=issues` query parameter.

#18115

Video commentary:
https://www.loom.com/share/ff4425e217db43d6b3ba4aa075374808

@getvictor TODO: Replacing `disable_failing_policies` with
`disable_issues` will be done in a separate PR.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-06-17 12:15:42 -05:00
Roberto Dip
b01389ad31
don't rely on MDM solution name to know if the host has Fleet MDM on (#19688)
for #18977

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-14 15:01:12 -03:00
Lucas Manuel Rodriguez
904e8a6825
Added server_settings.query_report_cap (#19692)
#19600

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-06-14 12:24:01 -03:00
Martin Angers
3044eb9e6b
Soft-delete entries for host script results so the details are still available in activities (#19457) 2024-06-12 10:26:03 -04:00
Victor Lyuboslavsky
8c4c739ef3
Display ingested software on host details page. (#19576)
#19348 

Fixed host details page and device details page not showing the latest
software.
- During software ingestion, software titles are now added if needed and
software items have their title_id field populated.
- In addition, after refreshing via UI, the software will be re-fetched
if it has been modified.

Added `exclude_software` query parameter to the
`/api/latest/fleet/hosts/:id` endpoint to exclude software from the
response.

PR for API doc change: #19617

Related issue filed for the Device User Page:
https://github.com/fleetdm/fleet/issues/19618

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-12 08:38:57 -05:00
gillespi314
9afbd0e02f Update migrations 2024-05-31 17:17:37 -05:00
gillespi314
949acfb2bd Merge branch 'main' into feat-software-self-service 2024-05-31 16:35:56 -05:00
Roberto Dip
178c0d565f Merge remote-tracking branch 'origin/main' into feat-save-certs 2024-05-30 18:34:40 -03:00
Roberto Dip
545e56d288
19016 ingest certs on start (#19360)
For #19016

This changes all the places where we previously assumed that certs were
hardcoded when the Fleet server started to query the database instead.

The plan is to loadtest afterwards, but as a first preemptive measure,
this adds a caching layer on top the mysql datastore.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-05-30 18:18:42 -03:00
Martin Angers
6b9fecf172 Fix conflicts 2024-05-29 16:08:51 -04:00
Victor Lyuboslavsky
addbceece0
Activity feed webhooks backend (#19261)
This is a redo of the already-approved PR
https://github.com/fleetdm/fleet/pull/19125

The difference is that this merge is into main as opposed to the feature
branch.
2024-05-24 11:25:27 -05:00
Roberto Dip
283a232af4
add base schema to store MDM assets (#19200)
for #19013

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-05-22 17:42:37 -03:00
Dante Catalfamo
b55e2980c5
Add self_service bool to software_installers table (#19122)
#19116
2024-05-21 10:13:12 -04:00
Sarah Gillespie
8cc3d7dd4f
Update list host software to omit software installers for other platforms (#19088) 2024-05-16 17:09:41 -05:00
Roberto Dip
8612e1aa59 rearrange migrations 2024-05-15 20:03:08 -03:00
Martin Angers
4f9363fd78
Add cron job to cleanup unused software installers (#18812) 2024-05-07 16:50:44 -04:00
Roberto Dip
e5283ef2a8 Merge remote-tracking branch 'origin/main' into feat-software-installers 2024-05-07 13:37:29 -03:00
Martin Angers
7bb726ba8e
Create and return upcoming/past host activities for software installs (#18772) 2024-05-07 11:28:16 -04:00
Lucas Manuel Rodriguez
4a739fba8a
Cleanup query results after host is transferred to another team (#18712)
#18079

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- [X] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2024-05-03 17:37:55 -03:00
Victor Lyuboslavsky
4946c96e77
AI-generated calendar backend changes (#18571)
#18464

Added `/fleet/autofill/policy` endpoint to get autogenerated policy
description and resolution for a given SQL query.

Added `server_settings.ai_features_disabled` setting to disable the
above endpoint.

For Google calendar integration,
- changed the event title to: "💻 🚫 Scheduled maintenance"
- updated event description to include policy description and resolution
if only one policy is failing

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Luke Heath <luke@fleetdm.com>
2024-05-02 11:56:54 -05:00
Martin Angers
ad11f075c1
Add API endpoint to list host/device software (#18676) 2024-05-01 14:37:52 -04:00
Gabriel Hernandez
7983e91933 Merge branch 'main' into feat-software-installers 2024-04-30 11:40:55 +01:00
Sarah Gillespie
ced8e560c9
Update software installers schema with reference to software titles (#18589) 2024-04-29 12:22:59 -05:00
Lucas Manuel Rodriguez
1b26129465
Delete expired activities in cleanup cron (#18441)
#16989

API draft PR: #17809.
UI draft PR: #18243.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- [X] Added/updated tests
- ~[ ] If database migrations are included, checked table schema to
confirm autoupdate~
- ~For database migrations:~
- ~[ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.~
- ~[ ] Confirmed that updating the timestamps is acceptable, and will
not cause unwanted side effects.~
- ~[ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).~
- [X] Manual QA for all new/changed functionality
  - ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
2024-04-26 07:22:31 -03:00
Roberto Dip
28b2593570
add migrations for software installers (#18516)
for #18323

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
2024-04-25 08:48:44 -03:00
Martin Angers
47279ca2a9 Fix conflicts 2024-04-15 16:10:10 -04:00
Victor Lyuboslavsky
3367b7e036
Added orbit_version, fleet_desktop_version, and scripts_enabled to host details. (#18123)
#17361
#17148

In GET fleet/hosts/:id response, added the following fields:
- orbit_version
  - `orbit_version == null` means this agent is not an orbit agent
- fleet_desktop_version
- `fleet_desktop_version == null` means this agent is not an orbit agent
or it is an older version which is not collecting the desktop version
- `fleet_desktop_version == ""` means this agent is an orbit agent but
does not have fleet desktop
- scripts_enabled
- `scripts_enabled == null` means this agent is not an orbit agent or it
is an older version which is not collecting scripts_enabled

In orbit_info table, added the following fields:
- desktop_version
- scripts_enabled

Updated docs for orbit_info PR:
https://github.com/fleetdm/fleet/pull/18135

Updated API docs: https://github.com/fleetdm/fleet/pull/17814

MDM lock/unlock/wipe error messages are not part of this PR. They will
be in a separate PR.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-04-09 16:33:44 -05:00
Martin Angers
1983fc4f76 Update DB schema with stable timestamps 2024-04-03 14:15:06 -04:00
Martin Angers
4d8818c439 Update DB schema 2024-04-03 14:13:10 -04:00
Roberto Dip
e34b320ed5 Merge remote-tracking branch 'origin/main' into feat-macos-ddm 2024-03-27 11:58:22 -03:00
Martin Angers
b449900602 Regenerate schema.sql 2024-03-27 08:19:21 -04:00
Dante Catalfamo
7b13d9ce17
Add mdm_apple_declarative_requests table to log DDM requests (#17844)
#17792
2024-03-27 00:09:09 -03:00
Lucas Manuel Rodriguez
e8f177dd43
Additional changes to happy path and cleanup cron job (#17757)
#17441 & #17442
2024-03-26 13:39:37 -05:00
Victor Lyuboslavsky
2db8eb3c80
Update migrations for main rebase. 2024-03-26 13:39:36 -05:00
Victor Lyuboslavsky
1c311b73be
Fleet in your calendar configs (#17462)
Sub-task for #17230 

# Configuration changes
App configuration:
```yaml
integrations:
  google_calendar:
    - email: name@service-account.com
      private_key: ***
      domain: fleetdm.com
```
Team configuration:
```yaml
integrations:
  google_calendar:
    email: name@service-account.com
    enable_calendar_events: true
    policies:
      - name: My policy
        id: 12
    webhook_url: https://example.com/policy-remediation
```

Note: Policy is looked up by name when configuration is set. The policy
id is set/updated by the server for internal use.

# Checklist for submitter
<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-03-26 13:39:36 -05:00
Roberto Dip
95df7e2b0b
implement DDM cron and protocol bits (#17791)
for #17399
2024-03-25 17:32:27 -03:00
Martin Angers
994040b1c9
Send DeviceConfigured MDM command after DEP enrollment (#17737) 2024-03-25 13:25:29 -04:00
Roberto Dip
495638b45a
bugfixes, integration and cleanup for DDM (#17756)
Improvements and fixes I found while integrating this

- Renamed db columns to match the profile tables for consistency
- Added columns to `host_mdm_apple_declarations`
- Removed `team_declaration_checksum_view`
- Remove the ad-hoc `MDMAppleRecordDeclarativeCheckIn`, I confused
myself by developing this using tests, the device actually sends an
`Acknowledged` response, which is recorded by nano
- Fixed bugs in the `declaration/../..` endpoints
    - The prefix for the endpoint is `declaration` without `s`
- The response needs to include a `ServerToken`, otherwise the
declaration fails
2024-03-21 13:12:32 -03:00
Martin Angers
b0ab7bbdc4
Add enable_release_device_manually setting to team and no-team (#17698) 2024-03-19 13:21:16 -04:00
Roberto Dip
dc87ac2271
add schema for DDM (#17636)
#17405
2024-03-14 18:08:19 -03:00
Jahziel Villasana-Espinoza
2b9487197c
feat: script content cleanup and column removal (#17374)
> Related issue: #17374

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality
2024-03-13 15:53:08 -04:00
Martin Angers
0f8192348b
Add missing index to MDM Windows enrollments table (#17559) 2024-03-13 11:27:37 -04:00
Martin Angers
88eeb9c1a4 Reorder database migrations 2024-03-05 08:22:11 -05:00
Martin Angers
98bc2dc29a Fix conflicts 2024-03-05 08:11:28 -05:00
Jahziel Villasana-Espinoza
33a0324ebb
feat: adding, reading, and deleting flows for scripts updated to use new table (#17305)
> Related issues: #16842, #16843

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality
2024-03-04 11:00:08 -05:00
Gabriel Hernandez
e495144023 update schema file 2024-03-01 17:38:08 +00:00
Gabriel Hernandez
4fa50778ac merge main into feat-mdm-wipe-host 2024-03-01 17:08:41 +00:00
Sarah Gillespie
e0cb59d4cb
Surface MDM devices where DEP assignment failed (#16973) 2024-03-01 10:52:19 -06:00