mirror of
https://github.com/fleetdm/fleet
synced 2026-04-21 13:37:30 +00:00
2ddc2ae90a
<!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #43034 ## Before (correlated subqueries): The old query scans the policies table and for each policy row, MySQL executes up to 3 separate subqueries against policy_labels + label_membership: ```sql -- For EACH policy row p: -- Subquery 1: Does this policy have any include labels? NOT EXISTS ( SELECT 1 FROM policy_labels pl WHERE pl.policy_id = p.id AND pl.exclude = 0 ) -- Subquery 2: Is the host in at least one include label? OR EXISTS ( SELECT 1 FROM policy_labels pl INNER JOIN label_membership lm ON (lm.host_id = ? AND lm.label_id = pl.label_id) WHERE pl.policy_id = p.id AND pl.exclude = 0 ) -- Subquery 3: Is the host in any exclude label? AND NOT EXISTS ( SELECT 1 FROM policy_labels pl INNER JOIN label_membership lm ON (lm.host_id = ? AND lm.label_id = pl.label_id) WHERE pl.policy_id = p.id AND pl.exclude = 1 ) ``` With 200 policies, MySQL executes up to 600 subquery probes into policy_labels and label_membership. ## After (single aggregated LEFT JOIN): The new query first builds one aggregated result set from policy_labels + label_membership for this host, grouped by policy_id, then joins it once: ```sql LEFT JOIN ( SELECT pl.policy_id, MAX(CASE WHEN pl.exclude = 0 THEN 1 ELSE 0 END) AS has_include_labels, MAX(CASE WHEN pl.exclude = 0 AND lm.host_id IS NOT NULL THEN 1 ELSE 0 END) AS host_in_include, MAX(CASE WHEN pl.exclude = 1 AND lm.host_id IS NOT NULL THEN 1 ELSE 0 END) AS host_in_exclude FROM policy_labels pl LEFT JOIN label_membership lm ON lm.label_id = pl.label_id AND lm.host_id = ? GROUP BY pl.policy_id ) pl_agg ON pl_agg.policy_id = p.id ``` The subquery scans policy_labels once, LEFT JOINs to label_membership for the specific host, and aggregates per policy. Each policy gets three booleans: - has_include_labels: 1 if any policy_labels row with exclude=0 exists - host_in_include: 1 if any include label row matched a label_membership row for this host - host_in_exclude: 1 if any exclude label row matched a label_membership row for this host Then the WHERE clause uses these: ```sql (COALESCE(pl_agg.has_include_labels, 0) = 0 OR pl_agg.host_in_include = 1) AND COALESCE(pl_agg.host_in_exclude, 0) = 0 ``` The COALESCE handles policies with no policy_labels rows at all (the LEFT JOIN produces NULL). # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. ## Testing - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Refactor** * Optimized database query efficiency for policy operations, delivering approximately 77% faster query execution at scale while improving support for label-based policy scoping. <!-- end of auto-generated comment: release notes by coderabbit.ai --> |
||
|---|---|---|
| .. | ||
| .keep | ||
| 14827-prevent-TOCTOU-last-admin | ||
| 29657-custom-settings-configuration-profiles | ||
| 31289-acme-for-mdm-protocol | ||
| 32126-macos-fleetd-reinstall | ||
| 32662-include-correct-cpe | ||
| 32773-preview-windows-mdm | ||
| 33106-fix-generate-gitops-vpp | ||
| 33418-windows-mdm-profile-deletion | ||
| 34433-speedup-macos-profile-delivery | ||
| 34667-scim-user-host-emails-association | ||
| 34950-nano-tables-cleanup | ||
| 35067-windows-pro-missing-vulnerabilities | ||
| 35467-detail-query-config-preload | ||
| 35484-improve-policy_membership-contention | ||
| 36312-trim-spaces-from-fleets-names | ||
| 36751-add-fmas-to-policy-automation | ||
| 36799-macos-disk-space-purgeable | ||
| 37323-jetbrains-cve | ||
| 37546-android-certificate-install-activity | ||
| 37556-resend-android-certs | ||
| 38002-throttle-ca-certificate-profiles | ||
| 38036-gitops-ca-delete-order | ||
| 38041-entra-windows-conditional-access | ||
| 38793-python-scripts | ||
| 38929-reports-tab | ||
| 39066-vpp-timeout-install-details | ||
| 39082-setup-logo-light-background | ||
| 39190-display-sw-version-filter | ||
| 39308-team-ca-read-access | ||
| 39316-winoffice-vulnerability-detection | ||
| 39842-generate-gitops-bug | ||
| 39899-deterministic-cpe-matching | ||
| 39968-sso-validity-increase-default | ||
| 40050-server-core-msrc-differentiation | ||
| 40057-osv-vulns | ||
| 40117-fix-sql-table-alias-platform-detection | ||
| 40137-update-default-fleet | ||
| 40581-os-versions-vuln-details | ||
| 40715-allow-whitespace-end-users-form | ||
| 40751-google-drive-brew-version | ||
| 40785-fix-gitops-vpp-token-assignment | ||
| 40841-gitops-sw-upload-error | ||
| 40910-correct-request-certificate-pem | ||
| 40972-policy-description | ||
| 41324-support-labels-include-all-for-installers | ||
| 41409-use-fleetctl-new-templates-as-starter-lib | ||
| 41484-fix-windows-mdm-profile-upload-panic | ||
| 41500-validate-scripts | ||
| 41534-host-details-reports-api-end-point | ||
| 41540-host-details-reports-db-optimizations | ||
| 41542-android-cert-resend-backend | ||
| 41586-admin-by-request-false-positive | ||
| 41601-use-multiplatform-names-in-front-end | ||
| 41603-fix-query-responses | ||
| 41631-not-installed | ||
| 41636-typo-in-msrc-json | ||
| 41644-improve-cpe-matching | ||
| 41670-auto-rotate-recovery-lock | ||
| 41672-allow-omitting-manual-hosts-label | ||
| 41710-overwrite-software-title | ||
| 41742-fix-my-device-500-fleet-free | ||
| 41778-fix-enqueue-setup-experience-items-for-arch-linux | ||
| 41815-override-patch-policy-query | ||
| 41888-otel-service-name | ||
| 42017-host-details-reports-tab | ||
| 42047-android-web-app-banner | ||
| 42185-add-flatcar-coreos-linux-platforms | ||
| 42327-apple-profile-retries | ||
| 42383-android-display-name | ||
| 42399-support-vpp-policy-automations-in-generate-gitops | ||
| 42443-fix-show-disk-encryption-key-modal | ||
| 42572-fix-duplicate-text | ||
| 42600-android-cert-templates-cleared-on-reenroll | ||
| 42751-r2-fma | ||
| 42799-option-to-unlock-not-available-afler-lock | ||
| 42808-rwmutex-jitter-shouldupdate | ||
| 42814-sso-learn-more-link | ||
| 43034-optimize-policy-queries-for-host | ||
| allow-clearing-windows-update-settings | ||
| bump-mysql-8.0.42 | ||
| refactor-named-functions-nil-checks | ||
| up-default-software-batch | ||
| update-go-1.26.1 | ||