mirror of
https://github.com/fleetdm/fleet
synced 2026-05-23 17:08:53 +00:00
Feature 5676: Update vulnerability detection docs to include OVAL vulnerability detection. (#6451)
Update vulnerability detection docs to include OVAL vulnerability detection.
This commit is contained in:
Juan Fernandez
GitHub
parent
e92ea532b6
commit
f20530e708
1 changed files with 85 additions and 39 deletions
|
|
@ -9,7 +9,11 @@
|
|||
|
||||
Vulnerability processing is enabled by default for new installations.
|
||||
|
||||
Fleet checks for vulnerabilities using the National Vulnerability Database (NVD).
|
||||
Fleet stategy to vulnerability detection varies according to the host's platform. For Windows/macOS,
|
||||
vulnerabilities are checked using the National Vulnerability Database (NVD), for Linux
|
||||
we use the official OVAL definitions maintained by the different publishers (Canonical, Red Hat et al).
|
||||
|
||||
### Windows/MacOS hosts
|
||||
|
||||
First, Fleet retrieves the installed software for each host using osquery queries. Then, Fleet
|
||||
translates each installed software into [Common Platform Enumeration (CPE)](https://en.wikipedia.org/wiki/Common_Platform_Enumeration) names.
|
||||
|
|
@ -27,16 +31,29 @@ Vulnerability processing happens on the Fleet instance and not on the host machi
|
|||
detected vulnerabilities cannot be used in the same way you would use an osquery query (e.g. you wouldn't
|
||||
be able write a query to retrieve all CVEs).
|
||||
|
||||
### Linux hosts
|
||||
|
||||
First, we determine what Linux distributions are part of your fleet (keep in mind that there will
|
||||
be a small delay between the time a new Linux hosts is added and the time the host is 'detected'). We then
|
||||
use
|
||||
that information to determine what OVAL definitions need to be downloaded and parsed - you can find
|
||||
a list of all the OVAL definitions we use
|
||||
[here](https://github.com/fleetdm/nvd/blob/master/oval_sources.json). OVAL definitions will be
|
||||
refreshed on a daily basis.
|
||||
|
||||
Finally, we look at the software inventory of each host and execute the assertions contained in the
|
||||
corresponding OVAL file - any match is reported using the same channels as with Windows/Mac OS vulnerabilities
|
||||
|
||||
## Coverage
|
||||
|
||||
Fleet attempts to detect vulnerabilities for installed software that falls into the following categories (types):
|
||||
For Windows/Mac OS Fleet attempts to detect vulnerabilities for installed software that falls into the following categories (types):
|
||||
|
||||
### Apps
|
||||
- macOS
|
||||
- Windows
|
||||
|
||||
### Browser plugins
|
||||
- macOS, Windows, Linux
|
||||
- macOS, Windows
|
||||
- Chrome extensions
|
||||
- Firefox extensions
|
||||
|
||||
|
|
@ -50,13 +67,14 @@ Fleet attempts to detect vulnerabilities for installed software that falls into
|
|||
- Atom
|
||||
- Packages installed using Chocolatey
|
||||
|
||||
- Linux
|
||||
- deb
|
||||
- RPM
|
||||
- Atom
|
||||
- Python
|
||||
- Packages installed using Portage (Gentoo)
|
||||
- Packages installed using NPM
|
||||
For Linux, we adhere to whatever is defined in the OVAL definitions, except for:
|
||||
- Kernel vulnerabilities.
|
||||
- Vulnerabilities involving configuration files.
|
||||
|
||||
As of right now, the following distributions are supported:
|
||||
- Ubuntu
|
||||
- RHEL based distros (Red Hat, CentOS, Fedora, and Amazon Linux)
|
||||
|
||||
|
||||
The ingestion of software varies per platform. For each platform, we run an [osquery query](#ingesting-software-lists-from-hosts) to ingest software.
|
||||
|
||||
|
|
@ -102,13 +120,16 @@ You'll need to restart the Fleet instances after changing these settings.
|
|||
|
||||
## Performance
|
||||
|
||||
### Windows/Mac OS
|
||||
|
||||
Vulnerability processing is performed in one Fleet instance. If your Fleet deployment uses multiple
|
||||
instances, only one will be doing the work.
|
||||
|
||||
In order to conduct vulnerability processing, Fleet downloads the following files:
|
||||
|
||||
1. A preprocessed CPE database generated by FleetDM to speed up the translation process: https://github.com/fleetdm/nvd/releases
|
||||
2. The historical data for all CVEs and how to match to a CPE: from https://nvd.nist.gov/vuln/data-feeds
|
||||
2. The historical data for all CVEs and how to match to a CPE: from
|
||||
https://nvd.nist.gov/vuln/data-feeds
|
||||
|
||||
The database generated in step 1 is processed from the original official CPE dictionary
|
||||
https://nvd.nist.gov/products/cpe. This CPE dictionary is typically updated once a day.
|
||||
|
|
@ -117,8 +138,31 @@ The matching occurs server-side to make the processing as fast as possible, but
|
|||
|
||||
For example, when running a development instance of Fleet on an Apple Macbook Pro with 16 cores, matching 200,000 CPEs against the CVE database will take around 10 seconds and consume about 3GBs of RAM.
|
||||
|
||||
The CPU and memory usages are in burst once every hour (or the configured periodicity) on the instance that does the processing. RAM spikes are expected to not exceed the 2GBs.
|
||||
The CPU and memory usages are in burst once every hour (or the configured periodicity) on the
|
||||
instance that does the processing. RAM spikes are expected to not exceed the 2GBs.
|
||||
|
||||
### Linux
|
||||
|
||||
As with Windows/Mac OS, vulnerability detection for Linux is performed in a single Fleet instance. The
|
||||
files downloaded will vary depending on what distributions are on your fleet. The list of all the
|
||||
OVAL files we use can be found [here](https://github.com/fleetdm/nvd/blob/master/oval_sources.json).
|
||||
|
||||
When determining what specific file(s) to download we use the reported OS version and map that to an
|
||||
entry in the `oval_sources.json` dictionary. The mapping rules we use are fairly simple, depending on the
|
||||
distribution, we either use the major and minor versions and the platform name (for example `Ubuntu
|
||||
22.4.0` -> `ubuntu_2204`) or just the major version (for example `Red Hat Enterprise Linux
|
||||
9.0.0` -> `rhel_09`).
|
||||
|
||||
To reduce memory footprint during the evaluation phase and because of performance reasons, all downloaded OVAL files are
|
||||
parsed, and the result is stored in a file following the following naming convention: `fleet_oval_platform_date.json`.
|
||||
|
||||
The performance will be a function of three variables:
|
||||
- The size of the OVAL file
|
||||
- The amount of hosts to scan
|
||||
- The amount of installed software
|
||||
|
||||
That said, the performance characteristic should be linear (if scanning 200 hosts take
|
||||
~20 seconds, then scanning 2000 hosts should take ~200 seconds).
|
||||
|
||||
## Detection pipeline
|
||||
|
||||
|
|
@ -140,19 +184,41 @@ The whole pipeline exists to compensate for these differences, and it can be div
|
|||
|
||||
2. Processing
|
||||
|
||||
Processing happens in a loop. The default interval is 1hr.
|
||||
Processing happens in a loop and varies depending on the platform - first Windows/Mac OS hosts
|
||||
will be processed, then we look at Linux hosts. The default interval is 1hr.
|
||||
|
||||
### General process
|
||||
|
||||
```mermaid
|
||||
graph TD;
|
||||
interval{Once an hour}-->normalize[Normalized software list]
|
||||
normalize-->downloadCPE(Download CPE database from Fleet)
|
||||
downloadCPE-->cpeTranslate[CPE translation]
|
||||
cpeTranslate-->cveDownload(CVE datastreams downloaded)
|
||||
cveDownload-->cveMap[CVE detection]
|
||||
cveMap-->falsePositive[False positive removal]
|
||||
falsePositive-->interval
|
||||
normalize-->process1[Process Windows/Mac OS hosts]
|
||||
process1-->process2[Process Linux hosts]
|
||||
process2-->interval
|
||||
```
|
||||
|
||||
### Windows/Mac OS
|
||||
|
||||
```mermaid
|
||||
graph TD;
|
||||
process[Process Windows/Mac OS hosts] -->downloadCPE(Download CPE database from Fleet)
|
||||
downloadCPE-->cpeTranslate[CPE translation]
|
||||
cpeTranslate-->cveDownload(CVE datastreams downloaded)
|
||||
cveDownload-->cveMap[CVE detection]
|
||||
```
|
||||
### Linux
|
||||
|
||||
```mermaid
|
||||
graph TD;
|
||||
process[Process Linux hosts] --> fresh{OVAL defs older than one day?}
|
||||
fresh --no--> execute(Analyze hosts using OVAL definitions)
|
||||
fresh --yes--> remove(Remove old OVAL definitions)
|
||||
remove --> download(Download new OVAL definitions)
|
||||
download --> parse(Parse OVAL definitions)
|
||||
parse --> execute
|
||||
```
|
||||
|
||||
|
||||
|
||||
### Ingesting software lists from hosts
|
||||
|
||||
|
|
@ -214,24 +280,4 @@ These are two illustrative examples. The reality is that there is no map or list
|
|||
|
||||
Once we have a good CPE, we can match it against the CVE database. We download the data streams locally and match each CPE to the whole list. The matching is done using the [nvdtools implementation](https://github.com/facebookincubator/nvdtools).
|
||||
|
||||
### Discarding false positives
|
||||
|
||||
In some cases, the developers for a piece of software are also the packagers and distributors for that software. In most cases, the developers offer a set of options to install the software, but particularly in the case of Linux distributions, the package maintainers tend to be a set of different people altogether.
|
||||
|
||||
This is great for developers, because they don't have to be versed in all the intricacies of each package manager, but for a platform such as Fleet, trying to understand what's installed in a host, this adds an extra layer of complexity.
|
||||
|
||||
Linux distributions do security patches. These patches sometimes are old versions of software that is in the list of a long term maintenance distribution versions (sometimes abbreviated as LTS) with a vulnerability fix from a newer version.
|
||||
|
||||
In this case, the software developer might release version, say, 2.5, but in Ubuntu version 2.3-ubuntu2 is what's available. This version is not the vulnerable 2.3, but 2.3 with the patch that fixes that security vulnerability. That same fix might be version `2.4-1` in CentOS because that's what happened to be in the LTS version.
|
||||
|
||||
So Fleet with a pipeline up until the section above would see `some_app-2.4` and mark it as vulnerable. In reality, this is a false positive that needs to be discarded based on the data we parse for the changelogs for each distribution.
|
||||
|
||||
The list of potential false positives is collected in the same process as the CPE list: for each Linux distribution we support, we go through each package for that version, parse the changelogs for occurrences of CVEs mentioned. If there are any, we mark it as a potential version that Fleet might've flagged in previous steps.
|
||||
|
||||
If there's a match, we remove the false positive from the list.
|
||||
|
||||
#### Supported Linux distributions and versions
|
||||
|
||||
- CentOS 6, 7 and 8.
|
||||
|
||||
<meta name="pageOrderInSection" value="1300">
|
||||
|
|
|
|||
Loading…
Reference in a new issue