Commit graph

1311 commits

Author SHA1 Message Date
Martin Angers
6dd365f266
Fix scripts that block execution of subsequent scripts when timing out on Windows (#19485) 2024-06-11 09:21:57 -04:00
Martin Angers
dec9bc53e3
Fix code linting issue where a slice was created non-empty and appended-to (#19490) 2024-06-11 08:55:07 -04:00
Lucas Manuel Rodriguez
5f65ea831c
Disable AI features on non-new installations upgrading to 4.51.X (#19482)
#19365

Assuming we release this fix in 4.51.0:
- Migration from a version without the feature (< 4.50.0) to 4.51.0:
Should disable (set `ai_features_disabled=true`).
- Migration from a version with the feature (>= 4.50.X < 4.51.0) to
4.51.0: Should keep `ai_features_disabled` as-is.
- New installation of Fleet: Should come with AI features enabled
(`ai_features_disabled=false`).

From
https://github.com/fleetdm/fleet/issues/19365#issuecomment-2145825363.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Manual QA for all new/changed functionality
2024-06-10 16:49:27 -03:00
Lucas Manuel Rodriguez
3e19cd90a9
Log warning when hosts enroll with duplicate hardware UUIDs (#19475)
#16393

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
2024-06-10 16:48:05 -03:00
Roberto Dip
a9a11e293a
Fixed a bug that prevented unused script contents to be cleaned up. (#19615)
for #19500

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-10 16:03:34 -03:00
Jahziel Villasana-Espinoza
27b8a1364f
feat: new software installer and carves fields, kept original fields for backwards compat (#19597)
> Related issue; #19526

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-06-10 13:35:27 -04:00
Martin Angers
e6de0dcebd Add integration test of batch-setting with self-service 2024-06-05 15:44:02 -04:00
gillespi314
9afbd0e02f Update migrations 2024-05-31 17:17:37 -05:00
gillespi314
949acfb2bd Merge branch 'main' into feat-software-self-service 2024-05-31 16:35:56 -05:00
Gabriel Hernandez
af88da49bc Merge branch 'main' into feat-software-self-service 2024-05-31 18:15:19 +01:00
Roberto Dip
83ecb0cd2b
Merge branch 'main' into feat-save-certs 2024-05-31 12:29:57 -03:00
Benjamin Edwards
d4a639414e
use the correct configuration to test for external id (#19404) 2024-05-31 10:47:49 -04:00
Victor Lyuboslavsky
d43ad0c824
Fixes from PR review #19206 (live query stats optimization) (#19403)
Fixes from PR review #19206
2024-05-31 07:35:26 -05:00
Roberto Dip
1c2ee0c949 Merge remote-tracking branch 'origin/main' into feat-save-certs 2024-05-31 09:34:48 -03:00
Victor Lyuboslavsky
f761827850
Move CalculateAggregatedPerfStatsPercentiles reads to the replica (#19206)
Move CalculateAggregatedPerfStatsPercentiles reads to the replica
#18838

I manually tested the aggregated query stats change by using a read
replica.
https://github.com/fleetdm/fleet/blob/main/tools/mysql-replica-testing/README.md
- But set `master_delay=0` due to issue
https://github.com/fleetdm/fleet/issues/19272
- Run a saved query as a live query, and see that its stats in
`aggregated_stats` table are updated.


# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-31 07:08:31 -05:00
Victor Lyuboslavsky
cfe9657f75
fleetctl gitops --dry-run now errors on duplicate (or conflicting) global/team enroll secrets. (#19344)
#19152
`fleetctl gitops --dry-run` now errors on duplicate (or conflicting)
global/team enroll secrets.

- One check is done on the backend to check against existing secrets
- Another check is done in fleetctl to detect duplicate secrets coming
in

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-31 07:01:13 -05:00
Roberto Dip
178c0d565f Merge remote-tracking branch 'origin/main' into feat-save-certs 2024-05-30 18:34:40 -03:00
Roberto Dip
545e56d288
19016 ingest certs on start (#19360)
For #19016

This changes all the places where we previously assumed that certs were
hardcoded when the Fleet server started to query the database instead.

The plan is to loadtest afterwards, but as a first preemptive measure,
this adds a caching layer on top the mysql datastore.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
2024-05-30 18:18:42 -03:00
Victor Lyuboslavsky
b3f0d07e51
Improved software ingestion performance by deduplicating incoming software. (#19325)
#11942 

To check if you have duplicate software, install multiple python
versions, like:
```shell
brew install python@3.11
brew install python@3.12
```

Then query: `SELECT * FROM python_packages` to see the packages with the
same versions but different paths.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-30 13:14:49 -05:00
Martin Angers
23f06e38fd Fix all software titles test 2024-05-29 16:47:23 -04:00
Martin Angers
6b9fecf172 Fix conflicts 2024-05-29 16:08:51 -04:00
Dante Catalfamo
6f7a0f98e7
Self service install endpoint (#19294) 2024-05-29 11:01:48 -04:00
Martin Angers
0b9b8c6279
Software SS: Add missing data to device software endpoints (#19304) 2024-05-29 08:54:48 -04:00
Roberto Dip
9ec92a601f
prevent a bug causing SCEP renewals to fail (#19313)
for #19311

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-28 20:31:53 -03:00
Lucas Manuel Rodriguez
cdf2a0c47c
iPhone/iPad support (#19221)
#18119

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: RachelElysia <71795832+RachelElysia@users.noreply.github.com>
Co-authored-by: Jacob Shandling <61553566+jacobshandling@users.noreply.github.com>
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-05-28 19:17:14 -03:00
Jahziel Villasana-Espinoza
7b2c25270a fix: broken test 2024-05-28 16:03:45 -04:00
Jahziel Villasana-Espinoza
71bb25ff76 fix: pr feedback 2024-05-28 14:51:41 -04:00
Lucas Manuel Rodriguez
984711060f
Some docs added while learning DDM for iOS/iPadOS support research (#18561)
Some docs added while working on #18423.
2024-05-28 15:03:39 -03:00
Jahziel Villasana-Espinoza
452952bf71 chore: merge feature branch 2024-05-28 13:46:26 -04:00
Jahziel Villasana-Espinoza
188d773c56 fix: update integration tests to use private key 2024-05-28 13:28:49 -04:00
Jahziel Villasana-Espinoza
d085f340b4 feat: update ds test 2024-05-28 11:53:30 -04:00
Jahziel Villasana-Espinoza
797c0050f5 feat: update ds test 2024-05-28 11:47:42 -04:00
Jahziel Villasana-Espinoza
cd2b254c6f fix: move encryption to ds 2024-05-28 11:31:17 -04:00
Roberto Dip
741a83ddf2
add logic to manage ABM assets (#19293)
for #19179

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-28 12:10:32 -03:00
Martin Angers
1b9f5a79a5
Software SS: activities (#19292) 2024-05-28 10:44:06 -04:00
Dante Catalfamo
7193d0e52f
Add software self_service bool to software titles list (#19258)
Add an endpoint I forgot as part of #19212 😬
2024-05-27 15:44:31 -04:00
Martin Angers
16c4e0c411
Software SS: add self-service filter to list software titles and list host's/device's software (#19186) 2024-05-27 10:53:41 -04:00
Jahziel Villasana-Espinoza
e11f44a89b
feat: upload and delete APNS certs (#19275)
> Related issue: #19014 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-05-27 11:13:08 -03:00
Sharon Katz
f75f0776e6
additional stats (#19078)
Additional telemetry stats to report to Heroku.
FYI - @eashaw, we could later add to datadog as well.

---------

Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2024-05-24 15:06:10 -04:00
Victor Lyuboslavsky
addbceece0
Activity feed webhooks backend (#19261)
This is a redo of the already-approved PR
https://github.com/fleetdm/fleet/pull/19125

The difference is that this merge is into main as opposed to the feature
branch.
2024-05-24 11:25:27 -05:00
Jahziel Villasana-Espinoza
209c122a52 chore: uncomment test 2024-05-24 10:36:49 -04:00
Jahziel Villasana-Espinoza
3b611fb5ec feat: db test for soft delete 2024-05-24 10:35:28 -04:00
Jahziel Villasana-Espinoza
c3e8427b13 feat: soft delete for mdm assets 2024-05-24 10:07:50 -04:00
Lucas Manuel Rodriguez
b7aac2cfab
Fix bundle_identifier for common software in osquery-perf (#19023)
bundle_identifier should be unique for the common software (should not
have hostname name in it).
2024-05-24 09:26:42 -03:00
Jahziel Villasana-Espinoza
2d8038ddd0 fix: address feedback 2024-05-23 20:55:36 -04:00
Jahziel Villasana-Espinoza
09039a9291 chore: fix lint 2024-05-23 18:29:49 -04:00
Jahziel Villasana-Espinoza
be8489a2fd chore: cleanup 2024-05-23 18:27:18 -04:00
Jahziel Villasana-Espinoza
49dbba0517 feat: integration and db test updates 2024-05-23 18:16:37 -04:00
Jahziel Villasana-Espinoza
396630213e feat: implement reusing existing apns key 2024-05-23 17:22:36 -04:00
Victor Lyuboslavsky
67ab732867
Optimizing MySQL master DB based on customer feedback (#19075)
#18838

1. During software ingestion, switched to updating `last_updated_at` as
a batch for 1 host.
2. Removed `DELETE FROM software` statement that ran for every host
update (where software was deleted). This is only done now during the
vulnerability job.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-23 14:45:50 -05:00