fleet/server/chart/api/chart.go
Scott Gress 4334017b38
Add Vulnerabilities exposure dataset (#44124)
<!-- Add the related story/sub-task/bug number, like Resolves #123, or
remove if NA -->
**Related issue:** For #43769

# Details

Adds methods to collect data for the `cve` dataset. As with all sets
this is collected at hourly granularity, but unlike the `uptime` set,
the `cve` set uses the "snapshot" strategy so that we record at most one
change (the most recent) per hour.

For this first iteration, we are _recording_ data for all CVEs (i.e.,
which hosts were exposed to which CVEs at a given time), but we are only
_reporting_ a subset of CVEs for the dashboard chart. See [this
comment](https://github.com/fleetdm/fleet/pull/44124#discussion_r3155554405)
for more info.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.

- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.

## Testing

- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)

- [X] QA'd all new/changed functionality manually
- [X] Spot-checked the CVEs chosen by the `trackedCVESoftwareMatchers`
and didn't find any outside of the expected
- [X] With [front-end PR](https://github.com/fleetdm/fleet/pull/44261),
generated chart:
<img width="706" height="421" alt="image"
src="https://github.com/user-attachments/assets/539d9877-6573-4406-a159-1d2a711a045f"
/>



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Host vulnerability (CVE) chart added to the dashboard; CVE chart data
collection is now active.
  * Critical CVE tracking surfaces high-severity vulnerabilities.

* **Improvements**
* CVE chart refreshes every 3 hours (was daily) for more timely
insights.
* Snapshot collection reconciles and closes prior data during empty runs
to keep charts accurate.
* CVE queries may produce zero datapoints when no tracked CVEs exist,
without affecting other metrics.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-29 09:30:31 -05:00

141 lines
5.8 KiB
Go

package api
import (
"context"
"time"
)
// SampleStrategy describes how a dataset's samples combine within a bucket and
// whether rows can collapse across buckets when the bitmap is unchanged.
type SampleStrategy string
const (
// SampleStrategyAccumulate means each sample is a partial observation.
// Writes: every row is born closed (valid_to set at insert time to bucketEnd).
// Within-bucket samples OR-merge into the existing row via ODKU; a sample in
// a new bucket just creates a new row with a new valid_from. No explicit
// close step, no cross-bucket collapse.
// Reads: bucket value = OR of every row whose interval overlaps the bucket
// ("hosts observed at any point during the bucket").
// Used for datasets like uptime and software usage.
// @todo: implement job to collapse identical consecutive rows
// to optimize storage and query performance.
SampleStrategyAccumulate SampleStrategy = "accumulate"
// SampleStrategySnapshot means each sample is the full state of a single moment.
// Writes: rows are always keyed to 1h boundaries (so row transitions align
// to hour marks regardless of tz). Within a 1h write-bucket, the latest
// sample's bitmap overwrites via ODKU — last sample wins. Across buckets,
// unchanged state keeps the row open (valid_to = sentinel); a changed sample
// closes the prior row at the new hour boundary and opens a new one.
// Reads: bucket value = OR across entities of each entity's row active at
// bucketEnd ("state as of the end of the bucket"). An entity whose row was
// closed mid-bucket with no replacement is absent at bucketEnd.
// Used for datasets like CVE and software inventory.
SampleStrategySnapshot SampleStrategy = "snapshot"
)
// Dataset defines the interface for a chartable dataset.
type Dataset interface {
// Name returns the dataset identifier used in the DB and API path.
Name() string
// DefaultResolutionHours returns the default display granularity in hours.
// Used when the caller doesn't specify RequestOpts.Resolution. Unrelated
// to write-side granularity — all collectors write at 1h regardless of
// display resolution; see SampleStrategy for details.
DefaultResolutionHours() int
// SampleStrategy returns how samples combine within and across buckets.
SampleStrategy() SampleStrategy
// Collect is called by the cron job to populate data in bulk.
Collect(ctx context.Context, store DatasetStore, now time.Time) error
// DefaultVisualization returns the default visualization type (e.g. "line", "heatmap").
DefaultVisualization() string
}
// DatasetStore is the narrow interface that datasets need for their Collect
// method. It is satisfied by the chart internal Datastore, keeping dataset
// implementations decoupled from internals.
type DatasetStore interface {
// FindRecentlySeenHostIDs returns host IDs that have reported since the
// given cutoff. Used by datasets like uptime that derive their sample from
// recent host activity.
FindRecentlySeenHostIDs(ctx context.Context, since time.Time) ([]uint, error)
// AffectedHostIDsByCVE returns, for every CVE currently affecting any host,
// the slice of host IDs impacted by it. Unresolved-only is implicit in the
// underlying joins: a host's software/OS row transitions when it upgrades
// past the vulnerable version, so the join naturally stops matching.
AffectedHostIDsByCVE(ctx context.Context) (map[string][]uint, error)
// RecordBucketData writes one or more entity bitmaps for the given bucket
// using the specified sample strategy. See SampleStrategy for semantics.
RecordBucketData(
ctx context.Context,
dataset string,
bucketStart time.Time,
bucketSize time.Duration,
strategy SampleStrategy,
entityBitmaps map[string][]byte,
) error
}
// Host is a minimal host type for authorization checks within the chart bounded context.
// The JSON tags matter: the OPA rego policy reads object.team_id via the JSON-encoded
// input, so renaming or dropping the tag silently breaks team-scoped authorization.
type Host struct {
ID uint `json:"id"`
TeamID *uint `json:"team_id"`
}
// AuthzType implements platform_authz.AuthzTyper.
func (h *Host) AuthzType() string { return "host" }
// DataPoint represents a single data point in the chart response.
type DataPoint struct {
Timestamp time.Time `json:"timestamp"`
Value int `json:"value"`
}
// Response is the API response for chart data.
type Response struct {
Metric string `json:"metric"`
Visualization string `json:"visualization"`
TotalHosts int `json:"total_hosts"`
Resolution string `json:"resolution"`
Days int `json:"days"`
Filters Filters `json:"filters"`
Data []DataPoint `json:"data"`
}
// RequestOpts captures the parsed query parameters for a chart request.
type RequestOpts struct {
Days int
// Resolution is the display granularity in hours. Must be 0 or a positive
// divisor of 24. 0 means "use the dataset's default resolution."
Resolution int
// TZOffsetMinutes is the client's UTC offset as reported by JavaScript's
// Date.getTimezoneOffset() (positive = west of UTC, e.g. CDT = 300).
// Used to align hourly bucket boundaries to local time.
TZOffsetMinutes int
// TeamID scopes the request to a single team. nil = global (authz + data
// both fall back to the user's accessible scope). *TeamID == 0 means
// hosts with no team assignment, matching Fleet's convention elsewhere.
TeamID *uint
LabelIDs []uint
Platforms []string
IncludeHostIDs []uint
ExcludeHostIDs []uint
}
// Filters captures the applied filters for a chart request.
type Filters struct {
TeamID *uint `json:"fleet_id,omitempty"`
LabelIDs []uint `json:"label_ids,omitempty"`
Platforms []string `json:"platforms,omitempty"`
IncludeHostIDs []uint `json:"include_host_ids,omitempty"`
ExcludeHostIDs []uint `json:"exclude_host_ids,omitempty"`
}