Warn on CVE description language tags from NVD feed that we haven't seen before (#21310)

We may need more effort to alert on this in a place where we can
see/action it, but for that alerting we can just catch warn (or
warn-and-above) logs, as I just ran the artifact build command and
didn't get any warnings.

Confirmed that this works by starting without the "es" case (since I
wasn't sure which language tags other than "en" and "en-US" we were
seeing) and seeing a bunch of CVEs from 2004 with "es" language tags
reported as warnings. Also confirmed (`gzcat
cvefeed/nvdcve-1.1-2024.json.gz | grep -A 5 -B 5 "description_data"`)
that language tags were correctly set in the resulting feed (caught a
bug locally due to Golang having an implied break at the end of each
switch case).

# Checklist for submitter

- [x] Manual QA for all new/changed functionality
This commit is contained in:
Ian Littman 2024-08-19 10:42:24 -05:00 committed by GitHub
parent 3cfe583ea0
commit 605779eee3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -780,16 +780,19 @@ func convertAPI20CVEToLegacy(cve nvdapi.CVE, logger log.Logger) *schema.NVDCVEFe
descriptions := make([]*schema.CVEJSON40LangString, 0, len(cve.Descriptions))
for _, description := range cve.Descriptions {
// Keep only english descriptions to match the legacy.
// Keep only English descriptions to match the legacy format.
var lang string
switch {
case description.Lang == "en":
switch description.Lang {
case "en":
lang = description.Lang
case description.Lang == "en-US":
// This occurred starting with Microsoft CVE-2024-38200
case "en-US": // This occurred starting with Microsoft CVE-2024-38200.
lang = "en"
// non-English descriptions with known language tags are ignored.
case "es": // This occurred in a number of 2004 CVEs
continue
// non-English descriptions with unknown language tags are ignored and warned.
default:
// Non-english descriptions are ignored.
level.Warn(logger).Log("msg", "Unknown CVE description language tag", "lang", description.Lang)
continue
}
descriptions = append(descriptions, &schema.CVEJSON40LangString{