Commit graph

2002 commits

Author SHA1 Message Date
Roberto Dip
ca06f0aed6
prevent baseClient from trying to decode 204 responses (#16060)
noticed while working on #15916, we do a request that, when successful,
returns a 204 response (with no content)

currently the client will fail to parse the contents of the response and
return an error "response: unexpected end of JSON input, body" even if
the request was succesful.
2024-01-11 17:55:35 -03:00
Jahziel Villasana-Espinoza
bff2d76b19
fix: false positives on ms teams on macos (#16048)
> 📜 Related issue: #15538

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-01-11 11:56:27 -05:00
Tim Lee
26c070eb00
fix vulnerabilities resolved in value (#15905) 2024-01-11 09:22:35 -07:00
Sarah Gillespie
2bbef8c56e
Bugfix: Only return host timeout error message when script exit code is nil (#15967) 2024-01-10 15:54:41 -06:00
Martin Angers
3e305e26d6
Fix pending script execution max age when notifying fleetd (#16001) 2024-01-10 14:53:12 -05:00
Lucas Manuel Rodriguez
eeb9931f40
Move external dependency mockimpl to monorepo (#15863)
#15560

Probably best to review commit by commit.
First commit adds the mockimpl files, second commit amends README.md and
third commit fixes golangci-lint issues.

- [X] Manual QA for all new/changed functionality

Tested by adding a dummy method to service.go and running `make
generate-mock`.

---------

Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
2024-01-10 11:46:24 -03:00
Victor Lyuboslavsky
abc56d988a
Improve UX for globally enabling/disabling SSO (#15887)
#15236 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-01-10 08:26:55 -06:00
Sarah Gillespie
4be9ca3f73
Fix database migration to preserve updated at timestamp for MDM profiles (#15993) 2024-01-09 14:10:20 -06:00
Jahziel Villasana-Espinoza
71b7b8b46c
feat: disable script endpoints and add error to cli (#15941)
> 📜 Relevant PR: #14500

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-01-08 16:28:45 -05:00
Sarah Gillespie
fa14eaf63a
Update categorization of Windows OS updates to exclude from user-defined Windows MDM profiles in API responses (#15924) 2024-01-05 09:28:54 -06:00
Tim Lee
ca948da440
Service layer to return nil query results if discard data is true (#15913) 2024-01-05 06:05:29 -07:00
Tim Lee
d690867a6a
Cleanup Query Results Cron (#15917) 2024-01-05 06:05:01 -07:00
Roberto Dip
18d830a126
allow to verify profiles that are pending (#15911)
for #15678
2024-01-04 18:26:04 -03:00
Roberto Dip
3b00b70786
fix unreleased bugs for the increased script timeout (#15897)
for https://github.com/fleetdm/fleet/issues/15196. The main problem was
that we have two timeouts:

1. The timeout used by the host to kill the script execution
2. The timeout used by the server to wait for the script results

Before the changes in https://github.com/fleetdm/fleet/pull/15779, the
server timeout was longer than the host timeout, but we inadvertently
set both values to 5 minutes, which breaks the logic we have to handle
both kinds of timeouts.
2024-01-04 10:26:13 -03:00
Jahziel Villasana-Espinoza
6dc9d2f6f0
fix: return better error when password is too long (#15875)
> Relevant issue: #15625

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-01-03 17:24:28 -05:00
Martin Angers
d943fbbf8e
Extend script execution timeout (#15779)
#15196 This is the work of @ghernandez345 except for adding the
`ResponseController` thing in Go to override the server timeout for that
specific sync endpoint so that the calls don't timeout waiting for a
script response (the default HTTP server timeout was 90s for our
server).

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Roberto Dip <me@roperzh.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-01-03 16:39:53 -03:00
Roberto Dip
a8a8080b3b
Fix a bug that didn't allow to enable team disk encryption if macOS MDM is not configured (#15886)
for #15817
2024-01-03 16:15:09 -03:00
Roberto Dip
f3d400d48e
automatically install fleetd for hosts that turn MDM manually (#15883)
for #15057
2024-01-03 15:16:59 -03:00
Jahziel Villasana-Espinoza
2833c80cfe
feat: add option to disable scripts (#15815)
> 📜 Related issue: https://github.com/fleetdm/fleet/issues/14500

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-01-03 10:42:08 -05:00
Victor Lyuboslavsky
912abd211d
Added a new synchronous live query endpoint: POST /api/v1/fleet/queries/:id/run (#15860)
Added a new synchronous live query endpoint: POST
/api/v1/fleet/queries/:id/run
#14800 

All relevant integration tests have been updated to work with the old
endpoint and new endpoint.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-01-03 09:39:16 -06:00
Lucas Manuel Rodriguez
417f45fc61
Move external dependency fleetdm/kolide-kit to monorepo (#15861)
#15556

We will need to pay attention when releasing fleet (the github actions
were modified to use the local file now).

Should be reviewed by commits (first commit is the actual adding of the
`version.go` file)

- [X] Manual QA for all new/changed functionality

Manually tested the following:
- `Settings -> My account` on the UI and checked the `/version` endpoint
response. (Or also visiting https://localhost:8080/version on a
browser).
- Ran `make fleetctl fleet`, `./build/fleetctl --version` and
`./build/fleet version`.
2024-01-02 18:22:52 -03:00
Lucas Manuel Rodriguez
d2015d1a36
Remotely configure fleetd update channels (#15848)
#13825

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [X] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
2024-01-02 17:59:40 -03:00
Lucas Manuel Rodriguez
38b8c9cc58
Move external dependency goose to monorepo (#15859)
#15555

Probably the best way to review this is commit by commit:
- First commit does the actual moving.
- Second commit fixes golangci-lint issues (in the least effort way to
avoid refactoring or rearrangement of some of the code).
- Third commit moves a printf to before the migration step is executed.
In the past some customers hitting migration issues (like migration
steps hanging or taking long to execute) and wanted to know which one
was it. The only way to know was to look at the repository and looking
for the next migration after the last one logged.

Checks:
- [X] Manual QA for all new/changed functionality

Manual tests:
- `make fleet && make db-reset`.
- Adding a new migration via `make migration name=Foobar` and then
running `./build/fleet prepare db`.
- Enrolling a new device to Fleet (smoke test).
2024-01-02 17:52:00 -03:00
Sarah Gillespie
583d31b721
Update fleetd for macOS hosts to look for custom end user email field in Fleet MDM enrollment profile (#15761)
Issue #15057 (macOS flow)

Manual QA: 
1. Download a manual enrollment profile for a macOS device from the "My
device" page (click on "Turn on MDM" banner).
2. Open the profile in a text editor and find the following plist entry:
```xml
<key>PayloadIdentifier</key>
<string>com.fleetdm.fleet.mdm.apple.mdm</string>
```
3. Below that entry add a new entry as follows:
```xml
<key>EndUserEmail</key>
<string>user@example.com</string>
```
4. Save the file, then double-click in Finder to activate the MDM
enrollment notification.
5. Run orbit from source:
```sh
go run github.com/fleetdm/fleet/v4/orbit/cmd/orbit \  
    --dev-mode \
    --disable-updates \
    --root-dir /tmp/orbit \
    --fleet-url https://localhost:8080 \
    --insecure \
    --enroll-secret <YOUR ENROLL SECRET GOES HERE> \
    -- --verbose
```
6. Look for the following log entries in the terminal where you are
running orbit:
<img width="679" alt="Screenshot 2023-12-21 at 3 03 03 PM"
src="https://github.com/fleetdm/fleet/assets/73313222/cefc77e3-e209-49b3-a03e-abff0f7f982b">


7. In the UI, navigate to the host details page and check "Used by" in
the "About" section:
<img width="679" alt="Screenshot 2023-12-21 at 3 02 09 PM"
src="https://github.com/fleetdm/fleet/assets/73313222/c58fff3e-cee7-4a94-a53b-f30f5b4bcfa0">


# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2024-01-02 17:45:11 -03:00
Jahziel Villasana-Espinoza
1b9b67aa9c
fix: query host search by email (#15833)
> 📜 Related issue: https://github.com/fleetdm/fleet/issues/15522

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-01-02 09:25:09 -05:00
Victor Lyuboslavsky
1704eb4014
Fixed missing 'Last restarted' values when filtering hosts by label. (#15852)
Fixed missing 'Last restarted' values when filtering hosts by label.
#15822

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-29 08:36:15 -06:00
Victor Lyuboslavsky
ebf1650671
device_token endpoint improvements (#15849)
Fixed badly formatted error messages in /api/fleet/orbit/device_token
endpoint and others.
In /api/fleet/orbit/device_token:
- Added token validation -- empty token not allowed
- Replaced 500 error with 409 when token conflicts with another host

#15832 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA
2023-12-28 14:20:36 -06:00
Victor Lyuboslavsky
6725da2ad0
Added host_count_updated_at to policy API responses. (#15767)
Added `host_count_updated_at` to policy API responses.
#15323 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-27 15:24:27 -06:00
Victor Lyuboslavsky
289eb271a6
Fixing TestQueries/Delete flaky tests. (#15838)
When deleting queries, associated query stats and aggregated stats are
deleted with separate SQL statements. When testing, we need to wait in a
for loop for both of these to be deleted before continuing with the
test.
2023-12-27 15:22:18 -06:00
Victor Lyuboslavsky
ce8eb74897
Added gigs_total_disk_space to host endpoint responses. (#15756)
Added `gigs_total_disk_space` to host endpoint responses.
- Now user will know host disk size even when disk is full
(percent_disk_space_available==0)
#15058 

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-22 12:46:33 -06:00
Lucas Manuel Rodriguez
f3aab18873
Change noisy directIngestWindowsUpdateHistory log from warn to debug (#15772)
#14939

- [X] Manual QA for all new/changed functionality.
2023-12-22 14:44:36 -03:00
Jahziel Villasana-Espinoza
b348399f7e
fix: email searching when filtering by labels (#15774)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-21 15:42:44 -05:00
Victor Lyuboslavsky
846b3537c0
Improved error message when Fleet cannot write osquery log results to logging destination. (#15726)
Improved error message when Fleet cannot write osquery log results to
logging destination.
#15455 



# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
2023-12-21 12:15:25 -06:00
Victor Lyuboslavsky
72621e31ce
Added gigs_total_disk_space to host_disks. (#15753)
Added gigs_total_disk_space to host_disks.

This is just the migration that is needed for the rest of the changes in
#15058
2023-12-21 12:13:04 -06:00
Jahziel Villasana-Espinoza
367459fe02
fix: use other field for checking vuln scanning status (#15720)
# Checklist for submitter

To test, go through the repro steps in the issue; you should see that
the field is true instead of false!

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-21 12:38:21 -05:00
Sarah Gillespie
682ab4aa7e
Add host display name to list hosts query (#15781) 2023-12-21 11:26:08 -06:00
Martin Angers
2e8da551d0
Custom email device-mapping: implement the CLI (fleetd + fleetctl) changes (#15763)
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2023-12-21 11:22:59 -06:00
Martin Angers
235d2cf2dc
Custom email device-mapping: implement the REST API changes (#15748) 2023-12-21 11:21:39 -06:00
Victor Lyuboslavsky
44c748fded
Removing time.Sleep from TestQueries. (#15764)
Fixing flaky TestQueries tests by removing time.Sleep
2023-12-20 10:01:33 -06:00
Victor Lyuboslavsky
0e51a26bf9
Fixing TestQueries/Save test. (#15729)
Fixing TestQueries/Save test.
2023-12-18 17:00:52 -06:00
Sarah Gillespie
108fadaa2d
Expand ORDER BY in list software titles (#15721) 2023-12-18 14:08:53 -06:00
Tim Lee
8f55f20250
15696 Add Policy stats data migration (#15697) 2023-12-18 10:49:53 -07:00
Victor Lyuboslavsky
bb30ffb21f
Added live query stats to activity. (#15682)
📺 Loom:
https://www.loom.com/share/474ac220dd054c389ebed6ba13f943b1?sid=70c2d94e-9fd9-40f5-af73-13e6d2e4c90a
Final piece for #467 -- Added live query stats to activity feed.
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-18 09:31:07 -06:00
Victor Lyuboslavsky
b930f66229
Live query stats are cleared when query SQL is modified. (#15710)
Live query stats are cleared when query SQL is modified.

Also, when deleting one or more queries, the associated aggregated stats
are now deleted as well.

#15709

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-18 08:45:19 -06:00
Marcos Oviedo
f3515686e6
Adding support for MS-MDE2 version 6.0 (#15701)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Manual QA for all new/changed functionality
2023-12-18 11:10:58 -03:00
Victor Lyuboslavsky
0e040cc7b0
fleetctl now runs saved queries (#15667)
📺 Looom:
https://www.loom.com/share/1aec4616fa4449e7abac579084aef0ba?sid=0884f742-feb3-48bb-82dc-b7834bc9a6e1

Fixed fleetctl issue where it was creating a new query when running a
query by name, as opposed to using the existing saved query.
#15630

API change will be in a separate PR:
https://github.com/fleetdm/fleet/pull/15673

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-15 12:55:39 -06:00
Lucas Manuel Rodriguez
024a20ac11
Allow enrolling fleetd using osquery's instance identifier (#15570)
#14879

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2023-12-15 15:26:32 -03:00
Victor Lyuboslavsky
bac932bd19
Documentation for max_prepared_stmt_count requirements and update-interval. (#15495)
Documentation changes for #15476
2023-12-14 17:38:47 -06:00
Jahziel Villasana-Espinoza
bcb66e8893
feat: add populate software query param (#15661)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-12-14 16:18:30 -05:00
Tim Lee
ced538c916
15597 Observer list query filter (#15653) 2023-12-14 13:25:42 -07:00