mirror of
https://github.com/fleetdm/fleet
synced 2026-05-17 14:08:25 +00:00
9c714c544d
#16562 Optimized policy_stats updates to NOT lock the policy_membership table. This should improve deployment performance with many global policies and team hosts. The original implementation that used INSERT ... SELECT (SELECT COUNT(*)) ... caused performance issues. Given 50 global policies, 10 teams, and 10,000 hosts per team, the INSERT query took 30-60 seconds to complete. Since it was an INSERT query, it blocked other hosts from updating their policy results in policy_membership. Now, we separate the INSERT from the SELECT, since SELECT by itself does not block other hosts from updating their policy results. In addition, we process one global policy at a time, which reduces the time to complete the SELECT query to <2 seconds, and limits the memory usage. We are not using a transaction to reduce locks. This means that INSERT may fail if the policy was deleted by a parallel process. Also, the INSERT may overwrite a clearing of the stats. This is acceptable, since these are very rare cases. We log and proceed in that case. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality |
||
|---|---|---|
| .. | ||
| .keep | ||
| 16562-deadlock | ||
| 16562-policy_stats-lock | ||
| 16989-delete-activities | ||
| 16989-ui-to-delete-old-activities | ||
| 17110-win-os-reporting | ||
| 17321-zsh-support | ||
| 17360-better-url-email-validators | ||
| 17771-invalid-query-platforms | ||
| 18039-private-IPv6-address | ||
| 18079-query-results-bug-transfer-hosts | ||
| 18085-fix-repeated-install-commands-of-fleetd-on-windows-mdm | ||
| 18157-update-platform-policy-stats | ||
| 18173-linux-async-wipe | ||
| 18187-ai-generated-backend | ||
| 18394-print-team-id | ||
| 18424-fix-users-query-for-linux | ||
| 18558-windows-mdm-start | ||
| 18597-missing-tooltips | ||
| 18601-add-ubuntu-oval | ||
| 18605-host-expiry-window-setting | ||
| 18640-gitops-remove-teams | ||
| hosts-lifecycle | ||
| issue-18389-fix-uploading-signed-apple-mobileconfig-profiles | ||