Commit graph

57 commits

Author SHA1 Message Date
Juan Fernandez
0fab2fe0eb
Feature 6538: Remove cpe_id from the software_cve table (#6587)
* Don't insert dummy values for non-match CPEs, exclude OVAL platforms when getting software iterator.
* Removed CPEID from SoftwareVulnerability type and software_cve.cpe_id column.
* Removed GeneratedCPEID property
2022-08-24 13:10:58 -04:00
Juan Fernandez
7b612aa030
Bug 7320: Fixed dulp. vulns. on software details. (#7342)
* Bug 7320: Fixed dulp. vulns. on software details.

The software details page was showing duplicated vulnerabilities if the software was being used by many hosts.
2022-08-24 09:45:20 -04:00
Juan Fernandez
2932ea0899
Bug 7332: Include software without CPE entries (#7334)
When listing software for performing vuln. detection include software without CPE entries.
2022-08-22 11:45:17 -04:00
Juan Fernandez
8dccc42027
Bug 5983: Performance issues when listing software (#6879)
Improve performance when listing software by using the software_host_counts aggregate table.
2022-08-10 17:43:22 -04:00
Juan Fernandez
966bfbf85e
Feature 6487: Deprecate cpe_id from software_cve table (#6562)
Part 2/3 of the removal of the cpe_id column from the software_cve table in favor of using the newly added software_id coumn.
2022-08-04 09:24:44 -04:00
Juan Fernandez
5b90d11c85
Bug 6508: Don't show non-CVE entries (#6686)
Only include vulnerabilities (CVEs) in Fleet UI and API
2022-07-18 09:23:32 -04:00
Juan Fernandez
9d01ba33c6
Feature 6096: Scan RHEL/CentOS hosts using OVAL definitions (#6241)
Extended the OVAL parser/analyzer so that we can scan RHEL based systems.
2022-06-23 16:44:45 -04:00
Roberto Dip
2948e112f6
ensure software_host_counts is cleaned when software is deleted (#6270)
Related to #5982, this ensures we clean up software_host_counts rows referencing software that is not longer present in the software table.
2022-06-22 17:35:53 -03:00
Lucas Manuel Rodriguez
4cfeaa1580
Do not use golangci action for better reproducibility (use make lint-go) (#6175)
* Do not use golangci action for better reproducibility

* Add fix to trigger build

* Fix all reported issues

* fix more lint errors

* Add missing import

* Remove unused method

* Remove change not necessary
2022-06-10 18:52:24 -03:00
Juan Fernandez
ef73039559
Improve vulnerability detection for Ubuntu (#6102)
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts

To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
2022-06-07 21:09:47 -04:00
Michal Nicpon
30a864c8d2
Sync CVE scores periodically (#5838) 2022-06-01 10:06:57 -06:00
Michal Nicpon
a652292d0f
Don't return orphaned software from the API (#5840) 2022-05-30 09:23:27 -06:00
Michal Nicpon
6083b180a6
fix duplicate hosts in vuln webhook (#5843) 2022-05-25 11:40:12 -06:00
Michal Nicpon
983f9e59b3
Include CVE scores when listing software (#5673) 2022-05-20 10:58:40 -06:00
Martin Angers
0d0c17e0c6
Add osquery.min_software_last_opened_at_diff configuration option (#5394) 2022-04-27 09:47:09 -04:00
Lucas Manuel Rodriguez
407d8b527e
Remove unused Datastore.SaveHost and unused Modified fields (#5245) 2022-04-26 18:33:29 -03:00
Martin Angers
253baee657
Collect last_opened_at for macOS software, and return it in host details payload (#5376) 2022-04-26 14:16:59 -04:00
Martin Angers
7187f1adac
Queue jobs for Jira integration when enabled and new vulnerabilities are found. (#4975) 2022-04-11 16:42:16 -04:00
Martin Angers
4930ca2d0e
Support listing software hosts count filtered by team (#4388) 2022-02-28 13:55:14 -05:00
Tomas Touceda
8e68ec3b96
Insert one CVE per CPE when there are multiple matches (#4297)
* Insert one CVE per CPE when there are multiple matches

* Remove comment

* No need to do sqlx.In
2022-02-18 15:25:26 -03:00
Lucas Manuel Rodriguez
be72dc356c
Add CentOS parsing+post-processing to reduce false positives in vulnerability processing (#4037)
* Add CentOS parsing and post-processing in fleet

* Add tests and amend SyncCPEDatabase

* Add test for centosPostProcessing

* Changes from PR comments

* Amend software test

* Fix sync test

* Add index to source and vendor

* Use os.MkdirTemp

* Rearrange migrations

* Regenerate test schema

* Add support for testing migrations (#4112)

* Add support for testing migrations

* Rename migration in tests

* Changes suggested in PR

* Go mod tidy
2022-02-14 15:13:44 -03:00
Martin Angers
cc1cf69a0f
Use a dedicated table to store hosts_count and fix pagination with vulns (#4104) 2022-02-09 10:16:50 -05:00
Martin Angers
6e2ba62744
Trigger webhooks for recently published vulnerabilities (#3941) 2022-02-02 16:34:37 -05:00
Tomas Touceda
e956b0ba04
Add filter software by CVE and make osquery-perf also push vulnerable software (#3902)
* Add filter software by CVE and make osquery-perf also push vulnerable software

* Update based on review comments
2022-01-28 10:05:11 -03:00
Martin Angers
a6f3f02a85
Cleanup unused software after calculating the count of hosts (#3887) 2022-01-26 11:32:42 -05:00
Martin Angers
9a0f749641
Add hosts_count field to "list software" endpoint (#3873) 2022-01-26 09:47:56 -05:00
Lucas Manuel Rodriguez
371c533bfc
Improved Datastore usage of osquery hosts requests (#3601)
* WIP

* Amend tests

* Do not load aggregated stats for packs

* Add option to host lite

* Fix remaining TODOs

* Fix osquery_utils tests

* Fix SQL

* Fix SQL (bis)

* Restore AuthenticateHost to load once

* Code improvements and re-add deferred host save

* More fixes to the PR

* Wrap users table update on tx

* Add caching to ListPacksForHost and ListScheduledQueriesInPack

* Remove SaveHostSoftware (replaced by UpdateHostSoftware)

* Add unit tests for new functionality

* Add changes file

* Fix scheduled queries test
2022-01-17 22:52:09 -03:00
Lucas Manuel Rodriguez
49ceee59aa
Add fixes for running tests with mysql:8 and add mysql to test-go job matrix (#3627)
* Add fixes for running tests with mysql:8

* Add getServer function

* Test github matrix

* Add changes file for the user facing fix

* Remove unused mysql8 docker-compose
2022-01-11 22:44:37 -03:00
Tomas Touceda
a0404b6736
Add software count API (#3105)
* Add software count API

* Fix makefile

* Fine no mock generating at this point

* Actually, one last try

* Use go install instead

* Fix go sum/mod

* Improve documentation

* Try setting node to 14
2021-12-03 10:54:17 -03:00
Tomas Touceda
c65f8507fe
Remove host counts from software (#3082)
* Remove host counts from software

* Actually remove the host count from the struct

* Fix get test
2021-11-23 15:50:51 -03:00
Tomas Touceda
ddbfb7f621
Allow sort by more than one key (#2987)
* Allow sort by more than one key

* Add test for multi sort

* Expand documentation
2021-11-17 16:37:26 -03:00
Tomas Touceda
b802af6f44
Add host count to software API (#2879)
* Add host count to software API

* Update docs

* Update fleetctl tests to account for host counts

* Update docs to mention host_count special case

* Update func comment
2021-11-11 08:49:17 -03:00
Lucas Manuel Rodriguez
bb187a7885
Fix flaky TestSoftware/filters_by_team_and_paginates (#2835) 2021-11-08 13:07:42 -03:00
Lucas Manuel Rodriguez
837bfed167
Try goqu as sql builder on listSoftwareDB (#2773)
* Try goqu as sql builder on listSoftwareDB

* Fix case insensitive search and add test
2021-11-04 15:21:39 -03:00
Lucas Manuel Rodriguez
3136cc105e
Replace into software to update empty bundle_identifier entries (#2676) 2021-10-26 09:48:25 -03:00
Tomas Touceda
923d094471
Allow to skip loading CVEs in some cases (#2611) 2021-10-20 19:26:25 -03:00
Tomas Touceda
cbe931c0e9
Add vulnerable filter for software and also wire up the query search (#2604)
* Add vulnerable filter for software and also wire up the query search

* Add documentation

* Update to use software list options
2021-10-20 18:01:20 -03:00
Tomas Touceda
bb1dc401a3
Load vulnerabilities alongisde the software (#2518)
* Load vulnerabilities alongisde the software

* Update docs
2021-10-14 13:51:41 -03:00
Tomas Touceda
28be73c410
Add missing software in list hosts response (#2492) 2021-10-12 15:59:01 -03:00
Tomas Touceda
02b2c64b89
Make software list not flaky, hopefully (#2387) 2021-10-05 18:26:02 -03:00
Tomas Touceda
2df4389aa0
Ignore empty host users or software inventory (#2317)
* Ignore empty host users or software inventory

* Only store additional if it's not nil

* Update label/policy updated at when we record the executions and skip saving host

* Update changes file
2021-10-01 18:27:57 -03:00
Tomas Touceda
435178e93c
Add bundle identifier to software when available (#2220)
* Add bundle identifier to software when available

* Update docs

* Delete unneeded test
2021-09-28 18:13:34 -03:00
Martin Angers
86dce785ae
Improve performance of the Go test suite (#2060)
Closes #1805
2021-09-20 14:09:38 -04:00
Martin Angers
d956ca1bec
Use the context in the Datastore layer. (#2030) 2021-09-14 10:44:02 -04:00
Tomas Touceda
176037cad2
Implement fleetctl get software and the underlying API (#1999)
* Implement fleetctl get software and the underlying API

* Add documentation

* Simplify list software implementation

* Lint fixes

* Make team name unique

* Address review comments

* Fix lint

* Fix tests
2021-09-14 10:58:48 -03:00
Martin Angers
4f4185372d
Add support for context in datastore/mysql layer (#1962)
This is just to pass down the context to the datastore layer, it doesn't
use it just yet - this will be in a follow-up PR.
2021-09-14 08:11:07 -04:00
Martin Angers
a099854045
Add missing db transactions, read from primary when after a write (#1957) 2021-09-08 14:43:22 -04:00
Tomas Touceda
7c34956d31
Add coverage to uncovered mysql code (#1855)
* Add coverage to uncovered mysql code

* Add deleted method and update mock

* Fix test
2021-09-07 13:48:04 -03:00
Tomas Touceda
db2e04a7c3
Update new sched query stats if there's already some (#1918)
* Update new sched query stats if there's already some

* IGNORE if the sched query is not present

* Make tests less flaky
2021-09-02 17:39:08 -03:00
Martin Angers
3755a58070
Add support for MySQL read replicas (#1786)
Adds configuration options to use a read-only MySQL replica,
and uses it instead of the primary for reads.
2021-09-01 15:50:52 -04:00