fleet/server/vulnerabilities/nvd
Victor Lyuboslavsky eada583ff1
Updating CPE generator to use new NVD API. (#15018)
Loom explaining changes (hit 5 min limit):
https://www.loom.com/share/e59b63bf638e4d9cad7984ef589b878d?sid=111fff75-115a-4a44-ae4f-6f25fede0d51

#14887

- [x] Need to merge fleetdm/nvd PR
https://github.com/fleetdm/nvd/pull/25 before this one.

# Checklist for submitter

- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- Manually tested (with corresponding fleetdm/fleet changes) in my
personal fork: https://github.com/getvictor/nvd/releases

# QA Plan (must be done before merging this PR, and after merging the
nvd PR)
- [ ] Fork https://github.com/fleetdm/nvd and point `generate.yml` to
this branch.
[example](9d8e54930b/.github/workflows/generate.yml (L26))
- [ ] Add NVD_API_KEY to nvd secrets, and run the the nvd generate
GitHub action. Get key:
https://nvd.nist.gov/developers/request-an-api-key
- [ ] Compare the generated `cpe-###.sqlite.gz` to the previous one. One
way is to open it up with sqlite3 and `select * from cpe_2 order by
cpe23;` and dump results to a CSV file. Known differences are:
   - New file has ~2,500 more records
- Backslashes are handled differently for `Backpack\CRUD` and `Philips
In.Sight B120\37` products -- not a new issue since we do not support
those products right now
- `cpe:2.3🅰️moodle:moodle:4.2.0:*:*:*:*:*:*:*` -- this appears OK.
Also, it is a PHP plugin, and we don't support these currently.
- [ ] Record the existing vulnerabilities of current hosts.
- [ ] Stop any running fleet server. Delete `/tmp/vulndbs/cpe.sqlite`.
Can also delete other files there, or not delete this file -- it should
be overwritten by the new file. Also delete all rows in software_cpe and
software_cve DB tables. (Or can just spin up a fresh fleet server with
fresh DB, and re-enroll hosts (after setting the new env variable
below))
- [ ] Find the path to the generated `cpe-###.sqlite.gz` file
- [ ] Set `FLEET_VULNERABILITIES_CPE_DATABASE_URL` environment variable
to the above path, and start fleet server.
- [ ] After server's vulnerabilities cron job runs, the new
vulnerabilities should match the previous vulnerabilities
2023-11-20 16:10:00 -06:00
..
cpe.go Tightening the CPE matching to reduce false positive rate. (#15187) 2023-11-20 11:59:31 -06:00
cpe_matching_rule.go Fix CVE-2020-10146 false positive being detected on all Microsoft Teams versions (#13839) 2023-09-11 16:51:53 -03:00
cpe_matching_rule_test.go add CVE-2013-0340 to ignore list (#13942) 2023-09-15 16:38:33 -06:00
cpe_matching_rules.go Fix false positive vulnerability for python on macOS (#14531) 2023-10-13 14:24:35 -03:00
cpe_test.go Updating CPE generator to use new NVD API. (#15018) 2023-11-20 16:10:00 -06:00
cpe_translations.go Flock vulnerability false positive (#13827) 2023-09-14 10:38:36 -06:00
cpe_translations.json Tightening the CPE matching to reduce false positive rate. (#15187) 2023-11-20 11:59:31 -06:00
cve.go Fix false positive vulnerabilities on Chrome and Firefox extensions (#14791) 2023-11-01 09:26:26 -03:00
cve_test.go Fixing TestTranslateCPEToCVE (#14931) 2023-11-03 12:58:59 -05:00
db.go Updating CPE generator to use new NVD API. (#15018) 2023-11-20 16:10:00 -06:00
indexed_cpe_item.go Fixes various bugs with NVD vulnerability detection (#7963) 2022-10-04 07:04:48 -04:00
README.md Flock vulnerability false positive (#13827) 2023-09-14 10:38:36 -06:00
sanitize.go Fixes various bugs with NVD vulnerability detection (#7963) 2022-10-04 07:04:48 -04:00
sanitize_test.go upgrade Go version to 1.21.1 (#13877) 2023-09-13 15:59:35 -03:00
sync.go Add Description text to CVE Metadata (#13856) 2023-09-15 11:24:10 -06:00
sync_test.go Add Description text to CVE Metadata (#13856) 2023-09-15 11:24:10 -06:00
testing_utils.go Fixes various bugs with NVD vulnerability detection (#7963) 2022-10-04 07:04:48 -04:00

Testing CPE Translations

To improve accuracy when mapping software to CVEs, we can add data to cpe_translations.json which will get picked up by the NVD repo.

To test these changes locally, you can:

  1. make the appropriate changes to cpe_translations

  2. host this file on a local web server

    ./tools/file-server 8082 ./server/vulnerabilities/nvd/cpe_translations.json
    
  3. (re)launch your local fleet server with the following --config

    vulnerabilities:
    cpe_translations_url: "http://localhost:8082/cpe_translations.json"
    
  4. trigger the vulnerabilities scan

    fleetctl trigger --name vulnerabilities