Commit graph

1909 commits

Author SHA1 Message Date
Martin Angers
f5f66ebe63
Unify profiles database identifier to profile_uuid for macOS and Windows profiles. (#15297) 2023-12-04 10:04:06 -05:00
Lucas Manuel Rodriguez
ab0700b4e0
Fix osquery result logging when queries are configured outside of Fleet (#15393)
#15168

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests.
- [X] Manual QA for all new/changed functionality.

The issue can be reproduced by running `osqueryd` with
`--config_plugin=filesystem --config_path=/path/to/config.json`
This means the osquery config is fetched from a file rather than from
Fleet's agent settings.
The `/path/to/config.json` has the agent settings, e.g.:
```
{
  "decorators": {
    "load": [
      "SELECT uuid AS host_uuid FROM system_info;",
      "SELECT hostname AS hostname FROM system_info;"
    ]
  },
  "options": {
    "disable_distributed": false,
    "distributed_interval": 10,
    "distributed_plugin": "tls",
    "distributed_tls_max_attempts": 3,
    "logger_tls_endpoint": "/api/osquery/log",
    "logger_tls_period": 10,
    "pack_delimiter": "/"
  },
  "schedule": {
    "USB devices": {
      "query": "SELECT * FROM usb_devices;",
      "interval": 15
    },
    "OS version": {
      "query": "SELECT * FROM os_version;",
      "interval": 10
    }
  },
  "packs": {
    "Elsewhere": {
      "queries": {
        "Osquery Info": {
          "query": "SELECT * FROM osquery_info;",
          "interval": 30,
          "platform": "",
          "version": "",
          "snapshot": true
        }
      }
    }
  }
}
```

The three queries should be logged to Fleet's configured result logging
destination (default is `filesystem`).
2023-12-04 11:18:49 -03:00
Roberto Dip
a7be0be9e9
improve mdm_windows query to account for multiple registry entries (#15391)
for #15362, this adjusts the query we use to get MDM details for windows
to account for hosts that might have more than one matching value in the
registry for any of the items we query.
2023-12-04 11:03:05 -03:00
Sarah Gillespie
b660715e56
Add database migrations to support software titles feature (#15401)
Issue #15222
2023-12-01 11:33:07 -03:00
Victor Lyuboslavsky
a7ae828581
Adding extension_id and browser columns to software table. (#15285)
https://github.com/fleetdm/confidential/issues/2118
API doc changes are at: https://github.com/fleetdm/fleet/pull/15092

This change affects Chrome-based and Firefox browsers. I tried Safari,
but it appears that latest Safari extensions simply show up as regular
apps on macOS.

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
  - Tested detail query changes on macOS, Windows, Ubuntu, and ChromeOS
  - Tested Chrome, Edge, Brave, and Firefox browsers

Testing notes:
1. Install some browser extensions.
2. Update fleet to this change.
3. Refresh hosts, and trigger vulnerabilities job (which updates host
software).
4. Do something like `GET
https://localhost:8080/api/v1/fleet/software?query=Password%20Manager`
to see `extension_id` and `browser` for the hosts that have updated.
2023-11-30 19:06:17 -06:00
Sarah Gillespie
976eeba61d
Add validations to disallow custom MDM profiles that contain names reserved by Fleet (#15373) 2023-11-30 17:19:18 -06:00
Roberto Dip
b07fbdc1d7
implement profile verification for windows (#15348)
for #14426.

In order to prevent import cycles and be able to use some type
definitions/constants I followed the same pattern we did for macOS by
creating a new package named `syncml`. This makes the changelog look
bigger than it actually is, so I split it into two commits to make it
easier to review:

-
[d7c233d](d7c233d54c)
moves the relevant bits to this new package
-
[7531a07](7531a0742b)
implements profile verification
2023-11-30 09:17:07 -03:00
Roberto Dip
2c337a4f17
improve profile validation to detect invalid XML (#15365)
for #15361
2023-11-29 18:13:39 -03:00
Martin Angers
2f927df4f0
Implement Windows OS Updates (feature branch). (#15359) 2023-11-29 11:07:24 -05:00
Martin Angers
0b5eedb801
Fix edge case of AppConfig changes getting lost in cached mysql. (#15352) 2023-11-29 10:09:37 -05:00
Sarah Gillespie
1535b72305
Add hardware serial to host by identifier search (#15353) 2023-11-29 08:39:35 -06:00
Roberto Dip
f439432fae
feature: windows profiles (#15349) 2023-11-29 09:32:42 -05:00
Tim Lee
198b522132
Purge policy stats in cleanup (#15318) 2023-11-28 10:03:03 -07:00
Roberto Dip
4d56d25f11
show FV banner if the disk is encrypted but we don't get a key (#15317)
for #15068

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-11-27 18:23:01 -03:00
Tim Lee
b2568ad475
HotFix - ambiguous policy search name (#15312) 2023-11-27 12:21:39 -07:00
Roberto Dip
9c7b740b03
disable windows profiles feature in the back-end (#15278)
per discussion in slack, reverting relevant user-facing changes for
#13281
2023-11-27 16:08:52 -03:00
Victor Lyuboslavsky
e8b1041f47
Fix for #15171 (#15292)
Fix for #15171
2023-11-22 15:14:26 -06:00
Victor Lyuboslavsky
8cfe272091
filtering hosts with invalid team_id now returns 400 error. (#15266)
#15037 
For endpoint fleet/hosts, filtering hosts with invalid team_id now
returns 400 error.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-11-22 15:04:48 -06:00
Roberto Dip
5f313c8972
fix various bugs after testing windows MDM profiles (#15264)
for #13281
2023-11-22 10:48:28 -03:00
Victor Lyuboslavsky
fbbd81ff77
UmbrellaMenu.app no longer matches Cisco Umbrella (#15262)
#15176 
This was already fixed earlier in the sprint by
https://github.com/fleetdm/fleet/pull/15187
Adding docs and a test.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
2023-11-21 16:29:02 -06:00
Tim Lee
c5ea7eac4f
Cache policy counts (#15244) 2023-11-21 12:52:06 -07:00
Tim Lee
0557f10ac5
14729 smtp settings validation for TLS (#15029) 2023-11-21 11:48:21 -07:00
Lucas Manuel Rodriguez
4194c44131
Use NVD API 2.0 to download CVE information (#15102)
#14888

@getvictor This is ready for review, but keeping as draft as there are
probably many tests that need amending.

I used the new version of the `./tools/nvd/nvdvuln/nvdvuln.go` to
compare the current vulnerabilities found in our dogfood environment
with the vulnerabilities found by the code in this PR and both results
match:
```
go run -race -tags fts5 ./tools/nvd/nvdvuln/nvdvuln.go --debug --db_dir ./local --software_from_url <dogfood URL> --software_from_api_token <API_TOKEN> --sync 2>&1 | tee out.txt
[...]
CVEs found and expected matched!
```

- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [X] Manual QA for all new/changed functionality

---------

Co-authored-by: Victor Lyuboslavsky <victor@fleetdm.com>
Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
2023-11-21 12:30:07 -06:00
Tim Lee
97e88ae32e
hotfix: update CPE tests (#15250) 2023-11-21 10:25:27 -07:00
Victor Lyuboslavsky
eada583ff1
Updating CPE generator to use new NVD API. (#15018)
Loom explaining changes (hit 5 min limit):
https://www.loom.com/share/e59b63bf638e4d9cad7984ef589b878d?sid=111fff75-115a-4a44-ae4f-6f25fede0d51

#14887

- [x] Need to merge fleetdm/nvd PR
https://github.com/fleetdm/nvd/pull/25 before this one.

# Checklist for submitter

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- Manually tested (with corresponding fleetdm/fleet changes) in my
personal fork: https://github.com/getvictor/nvd/releases

# QA Plan (must be done before merging this PR, and after merging the
nvd PR)
- [ ] Fork https://github.com/fleetdm/nvd and point `generate.yml` to
this branch.
[example](9d8e54930b/.github/workflows/generate.yml (L26))
- [ ] Add NVD_API_KEY to nvd secrets, and run the the nvd generate
GitHub action. Get key:
https://nvd.nist.gov/developers/request-an-api-key
- [ ] Compare the generated `cpe-###.sqlite.gz` to the previous one. One
way is to open it up with sqlite3 and `select * from cpe_2 order by
cpe23;` and dump results to a CSV file. Known differences are:
   - New file has ~2,500 more records
- Backslashes are handled differently for `Backpack\CRUD` and `Philips
In.Sight B120\37` products -- not a new issue since we do not support
those products right now
- `cpe:2.3🅰️moodle:moodle:4.2.0:*:*:*:*:*:*:*` -- this appears OK.
Also, it is a PHP plugin, and we don't support these currently.
- [ ] Record the existing vulnerabilities of current hosts.
- [ ] Stop any running fleet server. Delete `/tmp/vulndbs/cpe.sqlite`.
Can also delete other files there, or not delete this file -- it should
be overwritten by the new file. Also delete all rows in software_cpe and
software_cve DB tables. (Or can just spin up a fresh fleet server with
fresh DB, and re-enroll hosts (after setting the new env variable
below))
- [ ] Find the path to the generated `cpe-###.sqlite.gz` file
- [ ] Set `FLEET_VULNERABILITIES_CPE_DATABASE_URL` environment variable
to the above path, and start fleet server.
- [ ] After server's vulnerabilities cron job runs, the new
vulnerabilities should match the previous vulnerabilities
2023-11-20 16:10:00 -06:00
Sarah Gillespie
57df2f250c
Add Windows MDM profiles to host details API response (#15210) 2023-11-20 14:34:57 -06:00
Victor Lyuboslavsky
8ae88cfe1a
Tightening the CPE matching to reduce false positive rate. (#15187)
#15143 and #15162 

Previous fix for #13889 caused false positives on software with similar
names. Tightening the matching to reduce false positive rate.
- Google Chrome Helper.app no longer matches Google Chrome.app
- Acrobat Uninstaller.app no longer matches Acrobat.app

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
2023-11-20 11:59:31 -06:00
Martin Angers
c1fa8de992
Fix data race in mysql tests (#15204) 2023-11-20 10:20:08 -05:00
Roberto Dip
d9f0f86002
update host profile status when we get a Windows MDM response (#15172)
related to #14364, this adds logic to update the `status` and `detail`
columns of `host_mdm_windows_profiles` when we get a management
response.
2023-11-20 11:25:54 -03:00
Roberto Dip
420dfe1cd0
batch set profiles as pending on profile-related actions (#15179)
final step of #14364
2023-11-20 11:16:02 -03:00
Zach Wasserman
d0cc940533
Remove atom_packages table from software inventory query (#15195)
Part of #15135

This needs to be removed ASAP because software inventory queries will
start failing when the table is removed from osquery and the agent
updates to that new version.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
  - For Orbit and Fleet Desktop changes:
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux. (macOS only)
2023-11-17 15:26:17 -06:00
Jahziel Villasana-Espinoza
b2e739eba0
fix: address citrix workspace false positive (#15152)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-11-17 13:42:05 -05:00
Sarah Gillespie
3399b31bfc
Update API endpoints that support os_setttings filter to include Windows MDM profiles status (#15188) 2023-11-17 12:33:06 -06:00
Victor Lyuboslavsky
91ee14f573
Allow reverting a deleted Munki. (#15183)
When Munki is deleted and reinstalled on the host, Fleet will show Munki
info again.
#12409 

# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-11-17 12:21:42 -06:00
Sarah Gillespie
0850367734
Add GET /mdm/profiles/summary endpoint (#15077) 2023-11-17 10:49:30 -06:00
Victor Lyuboslavsky
36e12d02e3
Show host display name in query results. (#15173)
Query report now shows the host display name instead of hostname.
#14763 

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-11-16 14:26:57 -06:00
Sharon Katz
c67d9eb3a5
Hackathon add metrics from Fleet servers. (#14216)
Adding a few stats to the metrics Fleet Servers send to Heroku.
2023-11-16 10:25:19 -05:00
Jahziel Villasana-Espinoza
1fca8b1e38
fix: sort order for Last restarted (#14878)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/REST API/rest-api.md or
docs/Contributing/API-for-contributors.md)
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Rachael Shaw <r@rachael.wtf>
2023-11-15 16:42:57 -05:00
Roberto Dip
f82cfa307d
allow fleetctl to configure windows mdm profiles for teams and "no team" (#15161)
final step of #14361 , this integrates the work of the other PRs.

currently branched off `14361-windows-custom-settings-configs`

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
2023-11-15 18:04:24 -03:00
Martin Angers
cb9774fff8
Automate skipping of old database migration tests (#15166) 2023-11-15 15:53:42 -05:00
Martin Angers
4f45942f38
Add endpoint to list macOS and Windows profiles combined, paginated (#15165) 2023-11-15 15:36:20 -05:00
Roberto Dip
319b64f368
allow to set mdm.windows_settings.custom_settings in configs (#15145)
for #14361 this adds the bits related to saving a slice of strings with
paths to configuration profiles.

---------

Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Marko Lisica <83164494+marko-lisica@users.noreply.github.com>
2023-11-15 13:58:46 -03:00
Martin Angers
8c3b23a142
Add endpoint to upload an MDM custom profile for Windows and macOS (#15150) 2023-11-15 10:58:59 -05:00
Martin Angers
9c87b12acb
Fix detection of Redis cluster on RedisLabs (#15104) 2023-11-15 09:03:06 -05:00
Roberto Dip
98c104a3b7
implement endpoint to batch set windows profiles (#15127)
for #14361. Avoiding a changelog entry intentionally since this API is
for contributors only.
2023-11-15 09:37:19 -03:00
Roberto Dip
6215d2e7d8
skip old migration tests (#15130)
migrations are static and not meant to change, so these tests have
successfully accomplished their purpose.

the main motivation is to prevent CI from timming out, as each migration
test runs all prior migrations from scratch, making these very time
expensive.
2023-11-14 13:42:17 -03:00
Jahziel Villasana-Espinoza
70048f5f91
14753 windows ps1 api (#15113)
# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2023-11-14 09:23:51 -05:00
Martin Angers
965a78d2de
Add endpoint to get or download a profile (Windows and macOS) (#15105) 2023-11-14 08:19:29 -05:00
Roberto Dip
bdadd5c288
Add logic for Windows profiles validation (#15120)
related to #14361 and #14366 this adds validations to user-provided
Windows profiles
2023-11-13 17:35:26 -03:00
Roberto Dip
8478171256
add a cron job to deliver windows MDM profiles (#15065)
for #14364 , this implements the delivery mechanism for windows hosts.

I will follow up in another PR with logic to update the profile status
when we get responses from the device.
2023-11-10 11:05:10 -03:00