mirror of
https://github.com/fleetdm/fleet
synced 2026-05-17 22:18:39 +00:00
9c714c544d
#16562 Optimized policy_stats updates to NOT lock the policy_membership table. This should improve deployment performance with many global policies and team hosts. The original implementation that used INSERT ... SELECT (SELECT COUNT(*)) ... caused performance issues. Given 50 global policies, 10 teams, and 10,000 hosts per team, the INSERT query took 30-60 seconds to complete. Since it was an INSERT query, it blocked other hosts from updating their policy results in policy_membership. Now, we separate the INSERT from the SELECT, since SELECT by itself does not block other hosts from updating their policy results. In addition, we process one global policy at a time, which reduces the time to complete the SELECT query to <2 seconds, and limits the memory usage. We are not using a transaction to reduce locks. This means that INSERT may fail if the policy was deleted by a parallel process. Also, the INSERT may overwrite a clearing of the stats. This is acceptable, since these are very rare cases. We log and proceed in that case. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality |
||
|---|---|---|
| .. | ||
| authz | ||
| bindata | ||
| config | ||
| contexts | ||
| cron | ||
| datastore | ||
| errorstore | ||
| fleet | ||
| goose | ||
| health | ||
| launcher | ||
| live_query | ||
| logging | ||
| mdm | ||
| mock | ||
| policies | ||
| ptr | ||
| pubsub | ||
| service | ||
| sso | ||
| test | ||
| version | ||
| vulnerabilities | ||
| webhooks | ||
| websocket | ||
| worker | ||
| utils.go | ||
| utils_test.go | ||