mirror of
https://github.com/fleetdm/fleet
synced 2026-05-24 09:28:54 +00:00
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** For #43769 # Details Adds methods to collect data for the `cve` dataset. As with all sets this is collected at hourly granularity, but unlike the `uptime` set, the `cve` set uses the "snapshot" strategy so that we record at most one change (the most recent) per hour. For this first iteration, we are _recording_ data for all CVEs (i.e., which hosts were exposed to which CVEs at a given time), but we are only _reporting_ a subset of CVEs for the dashboard chart. See [this comment](https://github.com/fleetdm/fleet/pull/44124#discussion_r3155554405) for more info. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements), JS inline code is prevented especially for url redirects, and untrusted data interpolated into shell scripts/commands is validated against shell metacharacters. ## Testing - [X] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [X] QA'd all new/changed functionality manually - [X] Spot-checked the CVEs chosen by the `trackedCVESoftwareMatchers` and didn't find any outside of the expected - [X] With [front-end PR](https://github.com/fleetdm/fleet/pull/44261), generated chart: <img width="706" height="421" alt="image" src="https://github.com/user-attachments/assets/539d9877-6573-4406-a159-1d2a711a045f" /> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Host vulnerability (CVE) chart added to the dashboard; CVE chart data collection is now active. * Critical CVE tracking surfaces high-severity vulnerabilities. * **Improvements** * CVE chart refreshes every 3 hours (was daily) for more timely insights. * Snapshot collection reconciles and closes prior data during empty runs to keep charts accurate. * CVE queries may produce zero datapoints when no tracked CVEs exist, without affecting other metrics. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
141 lines
5.8 KiB
Go
141 lines
5.8 KiB
Go
package api
|
|
|
|
import (
|
|
"context"
|
|
"time"
|
|
)
|
|
|
|
// SampleStrategy describes how a dataset's samples combine within a bucket and
|
|
// whether rows can collapse across buckets when the bitmap is unchanged.
|
|
type SampleStrategy string
|
|
|
|
const (
|
|
// SampleStrategyAccumulate means each sample is a partial observation.
|
|
// Writes: every row is born closed (valid_to set at insert time to bucketEnd).
|
|
// Within-bucket samples OR-merge into the existing row via ODKU; a sample in
|
|
// a new bucket just creates a new row with a new valid_from. No explicit
|
|
// close step, no cross-bucket collapse.
|
|
// Reads: bucket value = OR of every row whose interval overlaps the bucket
|
|
// ("hosts observed at any point during the bucket").
|
|
// Used for datasets like uptime and software usage.
|
|
// @todo: implement job to collapse identical consecutive rows
|
|
// to optimize storage and query performance.
|
|
SampleStrategyAccumulate SampleStrategy = "accumulate"
|
|
|
|
// SampleStrategySnapshot means each sample is the full state of a single moment.
|
|
// Writes: rows are always keyed to 1h boundaries (so row transitions align
|
|
// to hour marks regardless of tz). Within a 1h write-bucket, the latest
|
|
// sample's bitmap overwrites via ODKU — last sample wins. Across buckets,
|
|
// unchanged state keeps the row open (valid_to = sentinel); a changed sample
|
|
// closes the prior row at the new hour boundary and opens a new one.
|
|
// Reads: bucket value = OR across entities of each entity's row active at
|
|
// bucketEnd ("state as of the end of the bucket"). An entity whose row was
|
|
// closed mid-bucket with no replacement is absent at bucketEnd.
|
|
// Used for datasets like CVE and software inventory.
|
|
SampleStrategySnapshot SampleStrategy = "snapshot"
|
|
)
|
|
|
|
// Dataset defines the interface for a chartable dataset.
|
|
type Dataset interface {
|
|
// Name returns the dataset identifier used in the DB and API path.
|
|
Name() string
|
|
|
|
// DefaultResolutionHours returns the default display granularity in hours.
|
|
// Used when the caller doesn't specify RequestOpts.Resolution. Unrelated
|
|
// to write-side granularity — all collectors write at 1h regardless of
|
|
// display resolution; see SampleStrategy for details.
|
|
DefaultResolutionHours() int
|
|
|
|
// SampleStrategy returns how samples combine within and across buckets.
|
|
SampleStrategy() SampleStrategy
|
|
|
|
// Collect is called by the cron job to populate data in bulk.
|
|
Collect(ctx context.Context, store DatasetStore, now time.Time) error
|
|
|
|
// DefaultVisualization returns the default visualization type (e.g. "line", "heatmap").
|
|
DefaultVisualization() string
|
|
}
|
|
|
|
// DatasetStore is the narrow interface that datasets need for their Collect
|
|
// method. It is satisfied by the chart internal Datastore, keeping dataset
|
|
// implementations decoupled from internals.
|
|
type DatasetStore interface {
|
|
// FindRecentlySeenHostIDs returns host IDs that have reported since the
|
|
// given cutoff. Used by datasets like uptime that derive their sample from
|
|
// recent host activity.
|
|
FindRecentlySeenHostIDs(ctx context.Context, since time.Time) ([]uint, error)
|
|
|
|
// AffectedHostIDsByCVE returns, for every CVE currently affecting any host,
|
|
// the slice of host IDs impacted by it. Unresolved-only is implicit in the
|
|
// underlying joins: a host's software/OS row transitions when it upgrades
|
|
// past the vulnerable version, so the join naturally stops matching.
|
|
AffectedHostIDsByCVE(ctx context.Context) (map[string][]uint, error)
|
|
|
|
// RecordBucketData writes one or more entity bitmaps for the given bucket
|
|
// using the specified sample strategy. See SampleStrategy for semantics.
|
|
RecordBucketData(
|
|
ctx context.Context,
|
|
dataset string,
|
|
bucketStart time.Time,
|
|
bucketSize time.Duration,
|
|
strategy SampleStrategy,
|
|
entityBitmaps map[string][]byte,
|
|
) error
|
|
}
|
|
|
|
// Host is a minimal host type for authorization checks within the chart bounded context.
|
|
// The JSON tags matter: the OPA rego policy reads object.team_id via the JSON-encoded
|
|
// input, so renaming or dropping the tag silently breaks team-scoped authorization.
|
|
type Host struct {
|
|
ID uint `json:"id"`
|
|
TeamID *uint `json:"team_id"`
|
|
}
|
|
|
|
// AuthzType implements platform_authz.AuthzTyper.
|
|
func (h *Host) AuthzType() string { return "host" }
|
|
|
|
// DataPoint represents a single data point in the chart response.
|
|
type DataPoint struct {
|
|
Timestamp time.Time `json:"timestamp"`
|
|
Value int `json:"value"`
|
|
}
|
|
|
|
// Response is the API response for chart data.
|
|
type Response struct {
|
|
Metric string `json:"metric"`
|
|
Visualization string `json:"visualization"`
|
|
TotalHosts int `json:"total_hosts"`
|
|
Resolution string `json:"resolution"`
|
|
Days int `json:"days"`
|
|
Filters Filters `json:"filters"`
|
|
Data []DataPoint `json:"data"`
|
|
}
|
|
|
|
// RequestOpts captures the parsed query parameters for a chart request.
|
|
type RequestOpts struct {
|
|
Days int
|
|
// Resolution is the display granularity in hours. Must be 0 or a positive
|
|
// divisor of 24. 0 means "use the dataset's default resolution."
|
|
Resolution int
|
|
// TZOffsetMinutes is the client's UTC offset as reported by JavaScript's
|
|
// Date.getTimezoneOffset() (positive = west of UTC, e.g. CDT = 300).
|
|
// Used to align hourly bucket boundaries to local time.
|
|
TZOffsetMinutes int
|
|
// TeamID scopes the request to a single team. nil = global (authz + data
|
|
// both fall back to the user's accessible scope). *TeamID == 0 means
|
|
// hosts with no team assignment, matching Fleet's convention elsewhere.
|
|
TeamID *uint
|
|
LabelIDs []uint
|
|
Platforms []string
|
|
IncludeHostIDs []uint
|
|
ExcludeHostIDs []uint
|
|
}
|
|
|
|
// Filters captures the applied filters for a chart request.
|
|
type Filters struct {
|
|
TeamID *uint `json:"fleet_id,omitempty"`
|
|
LabelIDs []uint `json:"label_ids,omitempty"`
|
|
Platforms []string `json:"platforms,omitempty"`
|
|
IncludeHostIDs []uint `json:"include_host_ids,omitempty"`
|
|
ExcludeHostIDs []uint `json:"exclude_host_ids,omitempty"`
|
|
}
|