Commit graph

67 commits

Author SHA1 Message Date
Juan Fernandez
7b612aa030
Bug 7320: Fixed dulp. vulns. on software details. (#7342)
* Bug 7320: Fixed dulp. vulns. on software details.

The software details page was showing duplicated vulnerabilities if the software was being used by many hosts.
2022-08-24 09:45:20 -04:00
Juan Fernandez
2932ea0899
Bug 7332: Include software without CPE entries (#7334)
When listing software for performing vuln. detection include software without CPE entries.
2022-08-22 11:45:17 -04:00
Juan Fernandez
8dccc42027
Bug 5983: Performance issues when listing software (#6879)
Improve performance when listing software by using the software_host_counts aggregate table.
2022-08-10 17:43:22 -04:00
Juan Fernandez
966bfbf85e
Feature 6487: Deprecate cpe_id from software_cve table (#6562)
Part 2/3 of the removal of the cpe_id column from the software_cve table in favor of using the newly added software_id coumn.
2022-08-04 09:24:44 -04:00
Juan Fernandez
ab8cde6371
Feature 6454: Add software_id column to software_cve table (#6489)
Added software_id to software_cve in preparation for the removal of the cpe_id column.
2022-07-07 10:22:10 -04:00
Juan Fernandez
9d01ba33c6
Feature 6096: Scan RHEL/CentOS hosts using OVAL definitions (#6241)
Extended the OVAL parser/analyzer so that we can scan RHEL based systems.
2022-06-23 16:44:45 -04:00
Roberto Dip
2948e112f6
ensure software_host_counts is cleaned when software is deleted (#6270)
Related to #5982, this ensures we clean up software_host_counts rows referencing software that is not longer present in the software table.
2022-06-22 17:35:53 -03:00
Juan Fernandez
ef73039559
Improve vulnerability detection for Ubuntu (#6102)
Feature: Improve our capability to detect vulnerable software on Ubuntu hosts

To improve the capability of detecting vulnerable software on Ubuntu, we are now using OVAL definitions to detect vulnerable software on Ubuntu hosts. If data sync is enabled (disable_data_sync=false) OVAL definitions are automatically kept up to date (they are 'refreshed' once per day) - there's also the option to manually download the OVAL definitions using the 'fleetctl vulnerability-data-stream' command. Downloaded definitions are then parsed into an intermediary format and then used to identify vulnerable software on Ubuntu hosts. Finally, any 'recent' detected vulnerabilities are sent to any third-party integrations.
2022-06-07 21:09:47 -04:00
Michal Nicpon
30a864c8d2
Sync CVE scores periodically (#5838) 2022-06-01 10:06:57 -06:00
Michal Nicpon
a652292d0f
Don't return orphaned software from the API (#5840) 2022-05-30 09:23:27 -06:00
Michal Nicpon
6083b180a6
fix duplicate hosts in vuln webhook (#5843) 2022-05-25 11:40:12 -06:00
Michal Nicpon
983f9e59b3
Include CVE scores when listing software (#5673) 2022-05-20 10:58:40 -06:00
Martin Angers
0d0c17e0c6
Add osquery.min_software_last_opened_at_diff configuration option (#5394) 2022-04-27 09:47:09 -04:00
Lucas Manuel Rodriguez
407d8b527e
Remove unused Datastore.SaveHost and unused Modified fields (#5245) 2022-04-26 18:33:29 -03:00
Martin Angers
253baee657
Collect last_opened_at for macOS software, and return it in host details payload (#5376) 2022-04-26 14:16:59 -04:00
Lucas Manuel Rodriguez
42f53b63b2
Remove recursion and return doRetryErr on getOrGenerateSoftwareIdDB (#5239)
* Remove recursion and return doRetryErr on getOrGenerateSoftwareIdDB

* Use GetContext and LIMIT 1
2022-04-20 11:02:35 -03:00
Martin Angers
7187f1adac
Queue jobs for Jira integration when enabled and new vulnerabilities are found. (#4975) 2022-04-11 16:42:16 -04:00
Martin Angers
4930ca2d0e
Support listing software hosts count filtered by team (#4388) 2022-02-28 13:55:14 -05:00
Tomas Touceda
6d582cffa0
Don't depend on last inserted id to get the id for the new software (#4298)
* Don't depend on last inserted id to get the id for the new software

* Correct typo
2022-02-18 16:30:24 -03:00
Tomas Touceda
8e68ec3b96
Insert one CVE per CPE when there are multiple matches (#4297)
* Insert one CVE per CPE when there are multiple matches

* Remove comment

* No need to do sqlx.In
2022-02-18 15:25:26 -03:00
Lucas Manuel Rodriguez
be72dc356c
Add CentOS parsing+post-processing to reduce false positives in vulnerability processing (#4037)
* Add CentOS parsing and post-processing in fleet

* Add tests and amend SyncCPEDatabase

* Add test for centosPostProcessing

* Changes from PR comments

* Amend software test

* Fix sync test

* Add index to source and vendor

* Use os.MkdirTemp

* Rearrange migrations

* Regenerate test schema

* Add support for testing migrations (#4112)

* Add support for testing migrations

* Rename migration in tests

* Changes suggested in PR

* Go mod tidy
2022-02-14 15:13:44 -03:00
Martin Angers
cc1cf69a0f
Use a dedicated table to store hosts_count and fix pagination with vulns (#4104) 2022-02-09 10:16:50 -05:00
Martin Angers
73d4794c55
Ignore software_id = 0 when calculating hosts count (#4080) 2022-02-08 10:59:17 -05:00
Martin Angers
1686bcafb8
Fix requesting subsequent pages in list software (#4061) 2022-02-07 15:57:55 -05:00
Tomas Touceda
cf529e70cf
Issue 3173 debug status processlist (#4009)
* Add innodb status and process list

* Make json output a bit prettier

* Add changes file

* fix lint issues
2022-02-03 14:56:22 -03:00
Martin Angers
6e2ba62744
Trigger webhooks for recently published vulnerabilities (#3941) 2022-02-02 16:34:37 -05:00
Martin Angers
6319812984
Fix column name when detecting order by (#3978) 2022-02-02 10:08:51 -05:00
Tomas Touceda
a63c549f07
Make software filter by counts faster (#3975)
* Make software filter by counts faster

* Sort only when it's for the agg table
2022-02-02 09:51:56 -03:00
Tomas Touceda
e956b0ba04
Add filter software by CVE and make osquery-perf also push vulnerable software (#3902)
* Add filter software by CVE and make osquery-perf also push vulnerable software

* Update based on review comments
2022-01-28 10:05:11 -03:00
Martin Angers
a6f3f02a85
Cleanup unused software after calculating the count of hosts (#3887) 2022-01-26 11:32:42 -05:00
Martin Angers
9a0f749641
Add hosts_count field to "list software" endpoint (#3873) 2022-01-26 09:47:56 -05:00
Lucas Manuel Rodriguez
371c533bfc
Improved Datastore usage of osquery hosts requests (#3601)
* WIP

* Amend tests

* Do not load aggregated stats for packs

* Add option to host lite

* Fix remaining TODOs

* Fix osquery_utils tests

* Fix SQL

* Fix SQL (bis)

* Restore AuthenticateHost to load once

* Code improvements and re-add deferred host save

* More fixes to the PR

* Wrap users table update on tx

* Add caching to ListPacksForHost and ListScheduledQueriesInPack

* Remove SaveHostSoftware (replaced by UpdateHostSoftware)

* Add unit tests for new functionality

* Add changes file

* Fix scheduled queries test
2022-01-17 22:52:09 -03:00
Tomas Touceda
66f4fc46c0
Only join big tables when absolutely needed (#3308)
* Only join big tables when absolutely needed

* Fix test

* Revert unintended change in test
2021-12-09 17:36:12 -03:00
Tomas Touceda
dc9686024b
Remove tx from SaveHost (#3232)
* Remove tx from SaveHost

Also change REPLACE for software for an insert.

* Remove bundle identifier from the get since it's not part of the key

* Revert unneeded change
2021-12-07 12:39:07 -03:00
Tomas Touceda
a0404b6736
Add software count API (#3105)
* Add software count API

* Fix makefile

* Fine no mock generating at this point

* Actually, one last try

* Use go install instead

* Fix go sum/mod

* Improve documentation

* Try setting node to 14
2021-12-03 10:54:17 -03:00
Tomas Touceda
c65f8507fe
Remove host counts from software (#3082)
* Remove host counts from software

* Actually remove the host count from the struct

* Fix get test
2021-11-23 15:50:51 -03:00
Martin Angers
3ae57a0242
Create errors with ctxerr, add the call to store them in redis (#2786) 2021-11-15 09:11:38 -05:00
Tomas Touceda
b802af6f44
Add host count to software API (#2879)
* Add host count to software API

* Update docs

* Update fleetctl tests to account for host counts

* Update docs to mention host_count special case

* Update func comment
2021-11-11 08:49:17 -03:00
Lucas Manuel Rodriguez
bb187a7885
Fix flaky TestSoftware/filters_by_team_and_paginates (#2835) 2021-11-08 13:07:42 -03:00
Lucas Manuel Rodriguez
837bfed167
Try goqu as sql builder on listSoftwareDB (#2773)
* Try goqu as sql builder on listSoftwareDB

* Fix case insensitive search and add test
2021-11-04 15:21:39 -03:00
Lucas Manuel Rodriguez
3136cc105e
Replace into software to update empty bundle_identifier entries (#2676) 2021-10-26 09:48:25 -03:00
Tomas Touceda
923d094471
Allow to skip loading CVEs in some cases (#2611) 2021-10-20 19:26:25 -03:00
Tomas Touceda
cbe931c0e9
Add vulnerable filter for software and also wire up the query search (#2604)
* Add vulnerable filter for software and also wire up the query search

* Add documentation

* Update to use software list options
2021-10-20 18:01:20 -03:00
Tomas Touceda
86b1926d7c
Add missing join condition (#2586) 2021-10-20 13:24:51 -03:00
Tomas Touceda
bb1dc401a3
Load vulnerabilities alongisde the software (#2518)
* Load vulnerabilities alongisde the software

* Update docs
2021-10-14 13:51:41 -03:00
Tomas Touceda
28be73c410
Add missing software in list hosts response (#2492) 2021-10-12 15:59:01 -03:00
Tomas Touceda
2df4389aa0
Ignore empty host users or software inventory (#2317)
* Ignore empty host users or software inventory

* Only store additional if it's not nil

* Update label/policy updated at when we record the executions and skip saving host

* Update changes file
2021-10-01 18:27:57 -03:00
Tomas Touceda
435178e93c
Add bundle identifier to software when available (#2220)
* Add bundle identifier to software when available

* Update docs

* Delete unneeded test
2021-09-28 18:13:34 -03:00
Martin Angers
86dce785ae
Improve performance of the Go test suite (#2060)
Closes #1805
2021-09-20 14:09:38 -04:00
Martin Angers
d956ca1bec
Use the context in the Datastore layer. (#2030) 2021-09-14 10:44:02 -04:00