Commit graph

855 commits

Author SHA1 Message Date
Martin Angers
242b89436b
Merge branch 'main' into feat-config-profile-excluding-labels 2024-07-02 16:32:03 -04:00
Victor Lyuboslavsky
fceff75c7c
Fixing GitOps integration tests. (#20167)
Removing no_mdm integration test since fleet-gitops no longer has MDM
settings in its default configs. This means all these tests are no mdm
tests.
2024-07-02 14:08:32 -05:00
Martin Angers
2fd25cae48 Merge main, fix conflicts 2024-07-02 08:44:34 -04:00
jacobshandling
91b9c4a107
Add host's next maintenance window to the hosts/{id} and hosts/identifier/{identifier} endpoints, and render that data on the host details page (#19820)
## Addresses full stack for  #18554 
- Add new `timezone` column to `calendar_events` table
- When fetched from Google's API, save calendar user's timezone in this
new column along with rest of event data
- Implement datastore method to retrieve the start time and timezone for
a host's next calendar event as a `HostMaintenanceWindow`
- Localize and add UTC offset to the `HostMaintenanceWindow`'s start
time according to its `timezone`
- Include the processed `HostMaintenanceWindow`, if present, in the
response to the `GET` `hosts/{id}` and `hosts/identifier/{identifier}`
endpoints
- Implement UI on the host details page to display this data
- Add new and update existing UI, core integration, datastore, and
`fleetctl` tests
- Update `date-fns` package to the latest version

<img width="1062" alt="Screenshot 2024-06-26 at 1 02 34 PM"
src="https://github.com/fleetdm/fleet/assets/61553566/c3ddad97-23da-42c1-b4ed-b7615ec88aed">

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified tables for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-06-28 10:51:13 -07:00
Dante Catalfamo
1f0346ccc7
Host mdminfo remover (#19885)
#17278
2024-06-28 11:09:22 -04:00
Victor Lyuboslavsky
be7b0876c6
GitOps rename teams (#19987)
#18471 
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-06-27 16:10:49 -05:00
Martin Angers
81a3569300 Merge main and fix conflicts 2024-06-26 10:43:05 -04:00
Martin Angers
30864c5bdf
CP Exclude Labels: CLI changes (#19990) 2024-06-25 16:06:56 -04:00
Victor Lyuboslavsky
cdb0d78245
Updated fleetctl get queries/labels/hosts descriptions. (#19908)
#19577 
Updated fleetctl get queries/labels/hosts descriptions.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-25 09:32:05 -05:00
Sarah Gillespie
7b4b9cb858
Update CLI help to document usage of --end-user-email flag (#19959) 2024-06-24 15:13:04 -05:00
Sarah Gillespie
0cb2767f18
Add support for --end-user-email option when building fleetd packages for Linux (#19795) 2024-06-18 10:10:19 -05:00
Tim Lee
2d6458b3f8
Custom Ubuntu Kernel Vuln Scanning (#19588) 2024-06-17 15:44:01 -06:00
Martin Angers
4b2919e5da
Attempt fix of failure to remove non-empty test directory (#19799) 2024-06-17 14:57:45 -04:00
Victor Lyuboslavsky
2350ffdf49
Added host_issues (#19654)
* /api/latest/fleet/hosts now returns `critical_vulnerabilities_count`
for premium users. This data is held in the new `host_issues` table. The
failing policies total is updated in real-time, while the critical
vulnerabilities total is updated every hour after vulnerabilities job.
* /api/latest/fleet/hosts can be sorted by total_issues_count by
specifying `order_key=issues` query parameter.

#18115

Video commentary:
https://www.loom.com/share/ff4425e217db43d6b3ba4aa075374808

@getvictor TODO: Replacing `disable_failing_policies` with
`disable_issues` will be done in a separate PR.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-06-17 12:15:42 -05:00
Roberto Dip
1cc13a09fb
🧹 friday cleanup party: substitute deprecated import of go-kit (#19774)
`go-kit/kit/log` was deprecated and generating warnings

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-06-17 10:27:31 -03:00
Roberto Dip
48ea95385e
final adjustments to return timeout errors for software (#19772)
for #19453


![image](https://github.com/fleetdm/fleet/assets/4419992/c9849cf4-ea78-4393-8693-8ff8ea612221)

I don't know how to test this in the server without a significant
refactor.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-14 18:39:03 -03:00
Roberto Dip
b01389ad31
don't rely on MDM solution name to know if the host has Fleet MDM on (#19688)
for #18977

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-14 15:01:12 -03:00
Lucas Rodriguez
63a6bb8bf3
Merge branch 'main' into feature_19010-ipad-ios-lock-wipe 2024-06-14 12:45:47 -03:00
Lucas Manuel Rodriguez
904e8a6825
Added server_settings.query_report_cap (#19692)
#19600

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-06-14 12:24:01 -03:00
Victor Lyuboslavsky
8b84b06a86
/api/latest/fleet/hosts/:id/lock returns unlock_pin for Apple hosts (#19720)
/api/latest/fleet/hosts/:id/lock returns `unlock_pin` for Apple hosts
#19545 
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-14 06:58:17 -05:00
Lucas Manuel Rodriguez
60b233e5f7
Return token when creating API-only users (#19525)
#16961

API changes here: https://github.com/fleetdm/fleet/pull/17609/files

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-06-13 19:10:27 -03:00
George Karr
4e2f7e53c3
Merge branch 'main' into feature_19010-ipad-ios-lock-wipe 2024-06-13 14:01:02 -05:00
Jahziel Villasana-Espinoza
7a552b3acd
fix: check old config val for full backwards compat (#19717)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-06-13 12:08:44 -04:00
Lucas Manuel Rodriguez
af525223f2
Implement support for Wipe for iOS/iPadOS devices (#19704)
Backend changes for #19010.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Manual QA for all new/changed functionality
2024-06-13 12:26:02 -03:00
Martin Angers
a154f4ca04
Improve handling of timeouts and maximum size for the upload software installer endpoint (#19657) 2024-06-12 09:33:25 -04:00
Victor Lyuboslavsky
05eb338561
Enable gitops to create teams with no enroll secrets, or clear enroll secrets for an existing team (#19616)
Enable gitops to create teams with no enroll secrets, or clear enroll
secrets for an existing team
#19332 

`fleetctl apply` also gains this extra functionality. In `fleetctl
apply` secrets will not be change if one of the following:
- secrets is missing from yml
- They are blank in yml, like: `secrets:`
- They are null in yml, like: `secrets: null`

They will only be cleared with `fleetctl apply` if the user explicitly
sets them to an empty array, like:
- `secrets: []`

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-11 13:20:32 -05:00
Lucas Manuel Rodriguez
3dbdbc1bcf
Add support for iOS/iPadOS to osquery-perf (#19522)
#18119

- [X] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.

Sample on how to simulate 50 iPads and 50 iPhones:
```sh
go run ./cmd/osquery-perf -host_count 100 -os_templates iphone_14.6.tmpl:50,ipad_13.18.tmpl:50 -mdm_scep_challenge <...>
```
2024-06-10 17:02:35 -03:00
Lucas Manuel Rodriguez
a37d0692b1
Fix fleetctl preview bug caused by creating enroll secrets (#19497)
#19129

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Manual QA for all new/changed functionality
2024-06-10 16:49:59 -03:00
Victor Lyuboslavsky
08c54d235b
Improved gitops test. (#19544)
Some minor gitops test improvements.

I was debugging a gitops read-after-write consistency issue that I ended
up filing as #19543
2024-06-10 14:27:55 -05:00
Victor Lyuboslavsky
6a20231fc4
Added FLEET_CALENDAR_PERIODICITY for internal demo use. (#19559)
#19491 

Video demo:
https://www.loom.com/share/c8fca008a9674cc685a5c209d9689271?sid=1f67e6c5-5e0b-4f10-9837-dc5d4c27f858

Changes file not added since this is an undocumented feature for
internal use.

New tests not created since this feature is for internal use, and will
likely be removed in the near future.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-10 14:27:28 -05:00
Jahziel Villasana-Espinoza
27b8a1364f
feat: new software installer and carves fields, kept original fields for backwards compat (#19597)
> Related issue; #19526

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-10 13:35:27 -04:00
Roberto Dip
f7665297fc
fix fleetctl generate mdm-apple-bm (#19507)
for #19506

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-04 18:19:09 -03:00
Roberto Dip
1f6534f933
generate/ingest SCEP challenges and improve error messages (#19468)
for #19454
2024-06-03 18:33:52 -03:00
Martin Angers
d878290027
Merge branch 'main' into feat-software-self-service 2024-06-03 13:25:03 -04:00
Roberto Dip
86a638aaaa
allow the server to start without having a SCEP challlenge (#19444)
in an unreleased bug we introduced as part of the feature to generate
MDM certificates in the app, the server is not able to start unless you
provide some env vars, even if you're not using MDM.

this is a band-aid to allow people to start their servers, I will follow
up with a PR to generate the SCEP challenge on the fly.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-06-03 11:49:36 -03:00
gillespi314
949acfb2bd Merge branch 'main' into feat-software-self-service 2024-05-31 16:35:56 -05:00
Gabriel Hernandez
af88da49bc Merge branch 'main' into feat-software-self-service 2024-05-31 18:15:19 +01:00
Roberto Dip
1c2ee0c949 Merge remote-tracking branch 'origin/main' into feat-save-certs 2024-05-31 09:34:48 -03:00
Victor Lyuboslavsky
f761827850
Move CalculateAggregatedPerfStatsPercentiles reads to the replica (#19206)
Move CalculateAggregatedPerfStatsPercentiles reads to the replica
#18838

I manually tested the aggregated query stats change by using a read
replica.
https://github.com/fleetdm/fleet/blob/main/tools/mysql-replica-testing/README.md
- But set `master_delay=0` due to issue
https://github.com/fleetdm/fleet/issues/19272
- Run a saved query as a live query, and see that its stats in
`aggregated_stats` table are updated.


# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-31 07:08:31 -05:00
Victor Lyuboslavsky
cfe9657f75
fleetctl gitops --dry-run now errors on duplicate (or conflicting) global/team enroll secrets. (#19344)
#19152
`fleetctl gitops --dry-run` now errors on duplicate (or conflicting)
global/team enroll secrets.

- One check is done on the backend to check against existing secrets
- Another check is done in fleetctl to detect duplicate secrets coming
in

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-31 07:01:13 -05:00
Roberto Dip
178c0d565f Merge remote-tracking branch 'origin/main' into feat-save-certs 2024-05-30 18:34:40 -03:00
Roberto Dip
545e56d288
19016 ingest certs on start (#19360)
For #19016

This changes all the places where we previously assumed that certs were
hardcoded when the Fleet server started to query the database instead.

The plan is to loadtest afterwards, but as a first preemptive measure,
this adds a caching layer on top the mysql datastore.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-05-30 18:18:42 -03:00
Martin Angers
6b9fecf172 Fix conflicts 2024-05-29 16:08:51 -04:00
Jahziel Villasana-Espinoza
3da2f095c7 fix: move setting of field so that it's correctly passed to DS 2024-05-29 11:02:40 -04:00
Jahziel Villasana-Espinoza
52a1d3f480 fix: truncate key, docs 2024-05-29 10:36:38 -04:00
Lucas Manuel Rodriguez
cdf2a0c47c
iPhone/iPad support (#19221)
#18119

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Jacob Shandling <61553566+jacobshandling@users.noreply.github.com>
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-05-28 19:17:14 -03:00
Jahziel Villasana-Espinoza
55a4c783e9 feat: validate key length 2024-05-28 16:22:42 -04:00
Jahziel Villasana-Espinoza
452952bf71 chore: merge feature branch 2024-05-28 13:46:26 -04:00
Lucas Manuel Rodriguez
f8cee672e0
Support environment variables in config profiles (#18891)
#17309

I added some missing env var replacement tests for policies, queries,
etc.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-05-28 13:44:43 -03:00
Roberto Dip
741a83ddf2
add logic to manage ABM assets (#19293)
for #19179

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-28 12:10:32 -03:00