Commit graph

591 commits

Author SHA1 Message Date
Eng Zer Jun
1ab171faf3
test: use T.Setenv to set env vars in tests (#6714)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-07-18 14:22:28 -03:00
Roberto Dip
69f8f2a73b
add API endpoints to retrieve pre-built installers (#6672)
Rel: #6365, this adds a new endpoint to check and download pre-built installers.
2022-07-18 13:44:30 -03:00
Juan Fernandez
5b90d11c85
Bug 6508: Don't show non-CVE entries (#6686)
Only include vulnerabilities (CVEs) in Fleet UI and API
2022-07-18 09:23:32 -04:00
Roberto Dip
faa3e136d3
improve installerstore tool w/ better errors and bucket creation (#6685)
This improves the installerstore CLI tool with:

- The ability to create tests buckets for local development (otherwise you have to interact with another CLI or the MinIO UI)
- Improved error handling and messaging.
2022-07-15 12:20:24 -03:00
Roberto Dip
6ce70b416d
add a new tool to upload installers to blob storage (#6661)
Related to #6365 this adds a new tool to upload pre-built Orbit installers to a storage blob. It uses the same file conventions that the Fleet server expects, making it useful for local testing and infra envs alike.\

Usage and details in the README
2022-07-14 16:11:52 -03:00
Roberto Dip
86366b2ed8
add a new S3 datastore to retrieve pre-built packages (#6631)
Related to #6365, this extends the datastore/s3 package to retrieve installers from S3 according to the conventions listed in the parent issue. This also includes:

- A minor refactor to decouple Carves-related functionality from the core S3 functionality
- Set-up to run tests using minio (only enabled via the FILE_STORAGE_TEST env flag)
2022-07-14 14:14:24 -03:00
Roberto Dip
1ed8efacfa
increase the length of host_batteries.health (#6560)
Rel to #6559, this increases the size of host_batteries.health for cases like "Check Battery" that exceed 10 chars
2022-07-08 21:06:50 +00:00
Juan Fernandez
ab8cde6371
Feature 6454: Add software_id column to software_cve table (#6489)
Added software_id to software_cve in preparation for the removal of the cpe_id column.
2022-07-07 10:22:10 -04:00
Lucas Manuel Rodriguez
700bb9aa98
Fix Datastore.HostByIdentifier to set SeenTime (#6484)
* Fix Datastore.HostByIdentifier to set SeenTime

* Add changes file
2022-07-05 08:08:43 -03:00
Roberto Dip
b2d07e56b5
report error analytics (#6341)
Related to #5898, this reports an anonymized summary of errors stored in Redis into the analytics payload.

For each error stored, this includes:

- A `count` attribute with the number of occurrences of the error
- A `loc` attribute with the 3 topmost lines in the stack trace. Note that stack traces only contain package name + line number (example: github.com/fleetdm/fleet/server.go:12

This also includes a minor refactor around error types.
2022-06-28 16:31:14 -03:00
Martin Angers
539be8ee09
Add battery info in host details response (#6394) 2022-06-28 14:11:49 -04:00
Juan Fernandez
9d01ba33c6
Feature 6096: Scan RHEL/CentOS hosts using OVAL definitions (#6241)
Extended the OVAL parser/analyzer so that we can scan RHEL based systems.
2022-06-23 16:44:45 -04:00
Martin Angers
8f66a50f2b
Add hosts enrolled by operating system to anonymous usage stats (#6329) 2022-06-22 16:38:11 -04:00
Roberto Dip
2948e112f6
ensure software_host_counts is cleaned when software is deleted (#6270)
Related to #5982, this ensures we clean up software_host_counts rows referencing software that is not longer present in the software table.
2022-06-22 17:35:53 -03:00
Martin Angers
dc9e158153
Add weekly active users count in anonymous usage stats (#6317) 2022-06-22 15:20:57 -04:00
Martin Angers
81f0e0ccfa
Track active hosts count and enforce limit (#6099) 2022-06-13 16:29:32 -04:00
Martin Angers
7f9bb6431e
Update team integrations to reference global integrations (part of failing policies automation support) (#6156) 2022-06-13 10:04:47 -04:00
Lucas Manuel Rodriguez
4cfeaa1580
Do not use golangci action for better reproducibility (use make lint-go) (#6175)
* Do not use golangci action for better reproducibility

* Add fix to trigger build

* Fix all reported issues

* fix more lint errors

* Add missing import

* Remove unused method

* Remove change not necessary
2022-06-10 18:52:24 -03:00
gillespi314
44f902218c
Improve live query UX (#5749) 2022-06-10 13:29:45 -05:00
gillespi314
a3ab5646f5
Add new fleet_desktop property to config object (#6151) 2022-06-10 10:39:02 -05:00
Lucas Manuel Rodriguez
99833a3b92
Rearrange migrations (#6140)
Co-authored-by: Juan Fernandez <juan@fleetdm.com>
2022-06-08 16:03:23 -03:00
Juan Fernandez
ef73039559
Improve vulnerability detection for Ubuntu (#6102)
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts

To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
2022-06-07 21:09:47 -04:00
Martin Angers
0b5fcd8e27
Add support for custom headers in fleetctl (#6118) 2022-06-07 16:00:09 -04:00
Michal Nicpon
30a864c8d2
Sync CVE scores periodically (#5838) 2022-06-01 10:06:57 -06:00
Lucas Manuel Rodriguez
98be6cfc29
Improve Fleet Desktop "My Device" menu item UX at install time (#5915)
* Improve Fleet Desktop My Device link availability

* Use svc.clock and add test

* Revert change and add check for LastEnrolledAt on tests
2022-05-31 12:56:51 -03:00
Michal Nicpon
a652292d0f
Don't return orphaned software from the API (#5840) 2022-05-30 09:23:27 -06:00
Roberto Dip
fc7a045209
ensure previous host membership policies are deleted on enrollment (#5890)
Related to #5776, this ensures that when a host is re-enrolled on a different team we cleanup existing policy memberships in the same way we do when a host is assigned a team through the API.
2022-05-30 10:30:15 -03:00
Michal Nicpon
6083b180a6
fix duplicate hosts in vuln webhook (#5843) 2022-05-25 11:40:12 -06:00
gillespi314
b4ffec10db
Remove skipLoadingExtras and includeCVEScores from GetHost and HostByIdentifier methods (#5874) 2022-05-25 11:30:03 -05:00
gillespi314
85c6ca0e5f
Improve precision in reporting of unseen hosts via host status webhook automations (#5889) 2022-05-25 10:54:56 -05:00
gillespi314
bbc1891420
Add ability to see Google Chrome profiles on the Hosts page (#5839) 2022-05-23 14:27:30 -05:00
gillespi314
2db2c16511
Include MIA hosts under total count for Offline hosts (#5854) 2022-05-23 14:11:02 -05:00
Michal Nicpon
983f9e59b3
Include CVE scores when listing software (#5673) 2022-05-20 10:58:40 -06:00
gillespi314
d172128183
Add label information to host summary response (#5573) 2022-05-10 10:32:55 -05:00
Martin Angers
1fa7bb7a19
Support async saving of hosts' last seen time (#5640) 2022-05-10 11:29:17 -04:00
Michal Nicpon
a005b7e7b7
fix schema dump (#5562) 2022-05-10 09:03:04 -06:00
gillespi314
998471f716
Omit empty device mapping from host responses where optional device_mapping param is missing (#5541) 2022-05-05 13:13:53 -05:00
gillespi314
991c7ccadc
Add device_mapping to GET /hosts response (#5383) 2022-05-02 16:34:14 -05:00
gillespi314
1dabf52834
Add Zendesk external service integration for vulnerability automations (#5372) 2022-05-02 15:58:34 -05:00
Roberto Dip
a4725518ac
filter out Google Chrome profiles without an associated email before ingesting (#5440)
To add support for #400, we're using the macadmins/osquery-extension to gather Google Chrome profiles from hosts.

Under the hood, the extension looks and parses a json file in which Chrome stores a bunch of data. Given that emails are not required to create Chrome profiles, some of the profiles stored in this file and returned by the query we're using contain empty emails.

The idea after this change is to prevent empty emails from being ingested in the first place instead of filtering them after the fact. I have also included a migration to clean the rows with empty email columns.

Fixes #4780
2022-05-02 09:55:40 -03:00
Roberto Dip
2c23cb5f24
fix mysql tests by with a regexp assertion of possible error messages (#5444)
this fixes #5377 by asserting that the error message starts with "tsl" or "x509" as either of the strings represent the same error depending on the mysql version.

we should refactor the code to avoid checking on the error messages and probably use different error types instead.
2022-04-29 09:31:58 -03:00
Martin Angers
0d0c17e0c6
Add osquery.min_software_last_opened_at_diff configuration option (#5394) 2022-04-27 09:47:09 -04:00
Lucas Manuel Rodriguez
407d8b527e
Remove unused Datastore.SaveHost and unused Modified fields (#5245) 2022-04-26 18:33:29 -03:00
Martin Angers
253baee657
Collect last_opened_at for macOS software, and return it in host details payload (#5376) 2022-04-26 14:16:59 -04:00
Michal Nicpon
a64982f96c
Remove use of JSON_ARRAYAGG (#5350)
Not supported in MySQL < 5.7.22
2022-04-25 14:43:08 -06:00
Lucas Manuel Rodriguez
42f53b63b2
Remove recursion and return doRetryErr on getOrGenerateSoftwareIdDB (#5239)
* Remove recursion and return doRetryErr on getOrGenerateSoftwareIdDB

* Use GetContext and LIMIT 1
2022-04-20 11:02:35 -03:00
Lucas Manuel Rodriguez
29a902960e
Fix typos and amend PR template (#5227) 2022-04-19 18:09:55 -03:00
Juan Fernandez
2479d58262
Using '@' in target search causes 422 error response (#5148)
* Bug 4852:

When searching for Labels, breakdown query strings containing '@' in multiple search terms to
avoid issues with MySQL FTS.
2022-04-19 09:28:49 -04:00
gillespi314
70a61b86f1
Update os version aggregated stats for all teams (#5083) 2022-04-18 16:19:58 -05:00
Michal Nicpon
2dbf1d7975
fix update updated_at for aggregated_stats (#5112)
Update the updated_at column when using ON DUPLICATE UPDATE so that
the counts_updated_at is up to date

* basic sql formatting in code ie whitespace around operators
2022-04-15 14:09:47 -06:00