We may need more effort to alert on this in a place where we can
see/action it, but for that alerting we can just catch warn (or
warn-and-above) logs, as I just ran the artifact build command and
didn't get any warnings.
Confirmed that this works by starting without the "es" case (since I
wasn't sure which language tags other than "en" and "en-US" we were
seeing) and seeing a bunch of CVEs from 2004 with "es" language tags
reported as warnings. Also confirmed (`gzcat
cvefeed/nvdcve-1.1-2024.json.gz | grep -A 5 -B 5 "description_data"`)
that language tags were correctly set in the resulting feed (caught a
bug locally due to Golang having an implied break at the end of each
switch case).
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
#21239
This PR fixes the generated vuln JSON files, which would get the current
customers up and running.
QA Done:
- New vuln JSON files generated at:
https://github.com/getvictor/vulnerabilities/releases
- Ran local server with new vuln JSON files by setting `export
TEST_VULN_GITHUB_OWNER=getvictor`
- Diff of JSON files from https://github.com/fleetdm/vulnerabilities and
https://github.com/getvictor/vulnerabilities
Steps for diff:
```
mkdir new
cd new
gh release download cve-202408111650 -D . -R getvictor/vulnerabilities
gunzip *.gz
cd ../
mkdir old
cd old
gh release download cve-202408111637 -D . -R fleetdm/vulnerabilities
gunzip *.gz
cd ..
diff old new
```
Diff results also remove a few Rejected CVEs from JSON files. The 2024
results also have a few minor diffs that don't seem significant.
#14888
@getvictor This is ready for review, but keeping as draft as there are
probably many tests that need amending.
I used the new version of the `./tools/nvd/nvdvuln/nvdvuln.go` to
compare the current vulnerabilities found in our dogfood environment
with the vulnerabilities found by the code in this PR and both results
match:
```
go run -race -tags fts5 ./tools/nvd/nvdvuln/nvdvuln.go --debug --db_dir ./local --software_from_url <dogfood URL> --software_from_api_token <API_TOKEN> --sync 2>&1 | tee out.txt
[...]
CVEs found and expected matched!
```
- [X] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Added/updated tests
- [X] Manual QA for all new/changed functionality
---------
Co-authored-by: Victor Lyuboslavsky <victor@fleetdm.com>
Co-authored-by: Victor Lyuboslavsky <victor.lyuboslavsky@gmail.com>
