Fixes#28261.
~~Of note, this logic will prefer a non-primary CVSSv3.1 score over a
primary CVSSv3.0 score if 3.1 doesn't have primary but 3.0 does. I
haven't seen any evidence of this in our dataset (looked at 2024
output).~~
Updated with logic that will prefer a primary CVSSv3.0 score over a
secondary CVSSv3.1 score for a given vulnerability. In the test dataset
(2023 vuln snapshot, ~20k vulns) there were no cases where this
situation presented itself, so output was identical to the prior
implementation.
Validated by comparing a vulns run from GitHub Actions to a local run
with the new code, and confirmed that existing v3 scores weren't
replaced when they already existed (just got adds of v2 when only v3
existed, and v2/v3 adds when no scoring existed).
Confirmed that all three CVEs mentioned in #28261 show up in feed data.
Added spot-checks for secondary CVSS scores to the feed validator tool.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Manual QA for all new/changed functionality
Temporarily fixes#29570 test failures while we figure out what desired
behavior is and determine how to get that behavior.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
For #29345.
Tested with Ubuntu 24.10. Can test again with 25.04 once
https://github.com/fleetdm/nvd/pull/42 is merged.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
for #21304
# Checklist for submitter
- [X] Manual QA for all new/changed functionality
## Details
This PR adds a new validator for NVD feed files to be run as part of the
nvd repo workflow. The intention is for that workflow to fail if any of
the files it creates are not valid (i.e. they would not be parseable by
the Fleet server) so that we don't publish and tag a release with bad
files in it.
This follows the pattern from
https://github.com/fleetdm/fleet/issues/21300 as suggested by @iansltx.
## Testing
I downloaded all of the latest release files to my local system using
```bash
gh release download 202505190037 -D ~/Downloads/nvd
```
and then ran the validator on them with
```bash
go run cmd/cpe/validate/main.go --db_dir ~/Downloads/nvd
```
To simulate file issues, I modified one section of each file to change a
value into the wrong type, and validated that this caused the validator
to panic. Examples:
```
panic: failed to load CPE translations: decode json: json: cannot unmarshal string into Go struct field CPETranslation.filter.vendor of type []string
goroutine 1 [running]:
main.checkCPETranslations({0x16dc975f9?, 0x14000192190?})
/Users/scott/Development/fleet/cmd/cpe/validate/main.go:34 +0xa8
main.main()
/Users/scott/Development/fleet/cmd/cpe/validate/main.go:24 +0xb0
exit status 2
```
---
```
panic: failed to parse MacOffice release notes fleet_macoffice_release_notes_macoffice-2025_05_19.json: parsing time "xyz" as "2006-01-02T15:04:05Z07:00": cannot parse "xyz" as "2006"
goroutine 1 [running]:
main.checkMacOfficeNotes({0x16f7af5f9, 0x1a})
/Users/scott/Development/fleet/cmd/cpe/validate/main.go:56 +0x1f0
main.main()
/Users/scott/Development/fleet/cmd/cpe/validate/main.go:25 +0xbc
exit status 2
```
---
```
panic: failed to parse MSRC feed fleet_msrc_Windows_Server_2012_R2-2025_05_19.json: json: cannot unmarshal array into Go struct field Vulnerability.Vulnerabities.RemediatedBy of type bool
goroutine 1 [running]:
main.checkMSRCVulnerabilities({0x16f49b5f9, 0x1a})
/Users/scott/Development/fleet/cmd/cpe/validate/main.go:74 +0x1ac
main.main()
/Users/scott/Development/fleet/cmd/cpe/validate/main.go:26 +0xc8
exit status 2
```
Additionally I tried the validator in [a run of the NVD
workflow](https://github.com/fleetdm/nvd/actions/runs/15121687898/job/42505283781)
and it executed successfully.
For #28983. No changes file since this is being released directly to the
vulns feed.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
No issue, just unblocking CI tests. This will be fixed in
https://github.com/fleetdm/fleet/issues/28490
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
For #27193.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [ ] Manual QA for all new/changed functionality
https://github.com/fleetdm/fleet/issues/21300
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #25090.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #26561
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26977.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #24784
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
From what I can tell, continue-on-error has been false for the
integration suites since the suites were renamed to `integration-*`, so
this fixes that issue in addition to continuing to run test suites when
the vulns suite fails (which may be due to vulns feed updates).
This also makes the vulns test more resilient to new CVEs being reported
on Python 3.12.0, which is rather likely to collect new CVEs.
# Checklist for submitter
- [x] Added/updated automated tests
For #26870.
On my local, this gets us a differential vulns feed update (for this
part of it) in under 90 seconds, vs. taking on the order of...40
minutes, I think? RAM usage is a few GB, but we have headroom on GitHub
Actions and the performance increase is worth it.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [ ] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [ ] Manual QA for all new/changed functionality
> For #24784
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #26748
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
For #25882
For #26615
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #26073.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> for #25991
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
The host software version and VersionEndExcluding did not always get
parsed by semver properly. Switching to using `SmartVerCmp` from the
nvdtools code. This is much more relaxed when parsing versions.
https://github.com/fleetdm/fleet/issues/24810
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [ ] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
Won't solve the false positive issues due to version number mismatches,
but will fix the false negative where CE wasn't matching at all, and
this is a full fix for IJ CE installed via JetBrains Toolbox.
For #25662.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
Also includes a CPE translation fix to avoid missing Python prereleases
on Windows, though we have further work to do to generate a CPE that
matches the one NVD uses.
For #24611.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #25597. Needs to be QA'd pre-merge /cc @jmwatts
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
> For #25130
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #22723.
Bundle IDs pulled from installs of the software, with regexes being
starts-with matches to include EAPs in matches. Used the products list
from CVE-2024-37051 to match up NVD product names (which is why
Writerside isn't included here; it doesn't have any published vulns
yet).
This fixes vuln detection in e.g. GoLand EAPs when the app name is
something other than the product name, similar to what we've done with
IntelliJ and PyCharm (but omitting homebrew handling for now).
No changes file as this doesn't need to be cherry-picked, and it will go
out in the next NVD pull after merged to `main`.
# Checklist for submitter
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality
For #23579
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
QA'd locally successfully. It just took a bit longer for the vuln showed
up.
for #24041
This PR addresses an issue that can cause Windows vulnerability checks
to fail (possibly causing false negatives). We determine whether a
vulnerability in an MSRC bulletin applies to any hosts in a Fleet
instance by attempting to matching the data in [each row of the
`operating_systems`
table](65e374c85c/cmd/fleet/cron.go (L297-L303))
with [at least one "product" in a
bulletin](e2d9a9016c/server/vulnerabilities/msrc/analyzer.go (L39)),
including [matching architecture and "display
version"](76f5baced9/server/vulnerabilities/msrc/parsed/product.go (L26-L39)).
However a subset of products listed in these bulletins do not include
the display version, so for example a host whose OS was listed as
`Microsoft Windows Server 2022 Datacenter 21H2` (21H2 being the "display
version") would match nothing in the bulletins because no listed Server
2022 products include "21H2" in their names.
The fix made here is to add relevant version info to the products list
when we do our ETL of the MSRC bulletins. The version info was gleaned
from https://en.wikipedia.org/wiki/List_of_Microsoft_Windows_versions.
We see logs related to this issue a lot, so cleaning this up will
alleviate some noise and infra costs as well.
Added a cpe translation for the `gh` command. The software is identified
as `gh`, however, the cpe (`cpe:2.3🅰️github:cli:2.62.0:*:*:*:*:*:*:*`)
name is labeled as `cli`, thus the mismatch.
https://github.com/fleetdm/fleet/issues/24009
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
I noticed a spelling issue and a grammar issue and fixed similar ones
elsewhere in our docs
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
> No ticket, just fixing a broken test
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated tests
See #22492
As of today, MSRC's API will 401 if you pass it a JWT it doesn't expect,
and by reusing the GitHub API HTTP client for MSRC pulls we were passing
the API an unexpected JWT. Wasn't able to reproduce this locally because
I didn't need a GitHub token locally to pull release details, while the
token is populated in Actions. Was able to repro both the issue and this
fix inside Actions on my fork.
This also updates to call v3.0 of the API directly, which v2.0 has been
redirecting to for awhile.
Finally, adds slightly better logging so we know which part of the feed
generation process we're in when we're running this in Actions.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
#20934
This is tied to https://github.com/fleetdm/vulnerabilities/pull/14; for
supported OS versions (currently Amazon Linux 1/2/2022/2023) we'll pull
XZ'd sqlite files from the vulnerabilities repo and query them to
determine what's vulnerable. See the associated issue for how I
self-QA'd this.
This replaced OVAL parsing for Amazon Linux 2, as we were using the
wrong data source there (Amazon has backported a bunch of fixes to their
own-named releases, so any RHEL fixes don't match).
Some checklist items are missing here; getting this set up in draft to
get code feedback now, and I'll push updates with e.g. docs changes, as
well ass an addition to the changes file.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] Add tests to oval_platform
- [x] Add sync_test
- [x] Add database_test
- [x] Manual QA for all new/changed functionality
- [x] Update vulnerability management docs
We may need more effort to alert on this in a place where we can
see/action it, but for that alerting we can just catch warn (or
warn-and-above) logs, as I just ran the artifact build command and
didn't get any warnings.
Confirmed that this works by starting without the "es" case (since I
wasn't sure which language tags other than "en" and "en-US" we were
seeing) and seeing a bunch of CVEs from 2004 with "es" language tags
reported as warnings. Also confirmed (`gzcat
cvefeed/nvdcve-1.1-2024.json.gz | grep -A 5 -B 5 "description_data"`)
that language tags were correctly set in the resulting feed (caught a
bug locally due to Golang having an implied break at the end of each
switch case).
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
