mirror of
https://github.com/fleetdm/fleet
synced 2026-05-17 22:18:39 +00:00
9c714c544d
#16562 Optimized policy_stats updates to NOT lock the policy_membership table. This should improve deployment performance with many global policies and team hosts. The original implementation that used INSERT ... SELECT (SELECT COUNT(*)) ... caused performance issues. Given 50 global policies, 10 teams, and 10,000 hosts per team, the INSERT query took 30-60 seconds to complete. Since it was an INSERT query, it blocked other hosts from updating their policy results in policy_membership. Now, we separate the INSERT from the SELECT, since SELECT by itself does not block other hosts from updating their policy results. In addition, we process one global policy at a time, which reduces the time to complete the SELECT query to <2 seconds, and limits the memory usage. We are not using a transaction to reduce locks. This means that INSERT may fail if the policy was deleted by a parallel process. Also, the INSERT may overwrite a clearing of the stats. This is acceptable, since these are very rare cases. We log and proceed in that case. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality |
||
|---|---|---|
| .. | ||
| cached_mysql | ||
| mysql | ||
| mysqlredis | ||
| redis | ||
| s3 | ||