fleet/server/service
Victor Lyuboslavsky c1a5e3b7b6
Fix calendar duplicated events and other issues (#20443)
#19352
Includes the following changes:
- Re-enable calendar callback
- Introduced a new Redis key that indicates event was updated by
calendar callback. In that case, we ignore subsequent callbacks for 10
seconds.
- This reduces the amount of Google API calls, including handling of the
unneeded callback generated by our own event change.
- Read event from DB after acquiring lock. This is critical since we get
the updated ETag of the Google Calendar event from our DB. Using the
previous ETag when fetching event sometimes returns stale data,
resulting in duplicate events.
- Fixed bug in getCalendarLock where calendar cron would always think it
got the lock
- Do not refetch timezone during calendar callback to reduce Google API
load
- Watch for calendar event changes for 1 week after event end (to
account for user moving event into the future)
- #20442: Speculative improvement for Google callback latency by keeping
the same notification channel (callback URL).
- processCalendarAsync now takes at least 1 sec to process all events,
to reduce CPU/Redis load
- Increased lock expiration time from 1 minute to 20 minutes to account
for potential Google API retries, fixing occasional duplicate events.
- Added `get-events.go` helper script that gets maintenance events from
user calendars, and checks for duplicates

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-24 13:40:33 +02:00
..
async 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
calendar Fix calendar duplicated events and other issues (#20443) 2024-07-24 13:40:33 +02:00
externalsvc remove unused logic specific to DEP+Okta+ROP (#11261) 2023-04-21 14:57:52 -03:00
middleware Remove ineffective rate limit to /api/fleet/device/ping and api/fleet/orbit/ping endpoints (#16334) 2024-01-25 15:05:52 -03:00
mock Move nanomdm dependency in monorepo (#16015) 2024-01-11 23:28:48 -03:00
osquery_utils Revert changes to DEP enrollment flow (#20506) 2024-07-16 11:44:48 -05:00
redis_lock Distributed lock and store calendar_events UUID as binary in MySQL (#20277) 2024-07-10 08:49:05 -05:00
redis_policy_set Migrate team-related endpoints to new pattern (#3740) 2022-01-19 10:52:14 -05:00
schedule 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
testdata Match macOS software <-> titles using bundle identifier (#19969) 2024-07-09 13:43:21 -03:00
activities.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
activities_test.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
appconfig.go CP Exclude Labels: API changes (#19846) 2024-06-25 15:26:28 -04:00
appconfig_test.go fleetctl gitops --dry-run now errors on duplicate (or conflicting) global/team enroll secrets. (#19344) 2024-05-31 07:01:13 -05:00
apple_mdm.go Revert changes to DEP enrollment flow (#20506) 2024-07-16 11:44:48 -05:00
apple_mdm_test.go Revert changes to DEP enrollment flow (#20506) 2024-07-16 11:44:48 -05:00
base_client.go Fixes to fleetctl debug connection and TLS certs documentation (#20166) 2024-07-09 14:04:23 -03:00
base_client_errors.go Finalize MDM commands part 2: implement fleetctl mdm run-command (#10866) 2023-04-03 14:25:49 -04:00
base_client_test.go prevent baseClient from trying to decode 204 responses (#16060) 2024-01-11 17:55:35 -03:00
build_tags_test.go Return 405 when receiving POST request on the route path (#16750) 2024-02-14 12:40:43 -03:00
calendar.go Fix calendar duplicated events and other issues (#20443) 2024-07-24 13:40:33 +02:00
campaigns.go fleetctl, API, copy updates around host identifiers (#20220) 2024-07-09 10:25:01 -07:00
campaigns_test.go fleetctl, API, copy updates around host identifiers (#20220) 2024-07-09 10:25:01 -07:00
carves.go Update carve request block id mismatch error code from 500 –> 400 (#17974) 2024-04-04 09:38:10 -07:00
carves_test.go Update carve request block id mismatch error code from 500 –> 400 (#17974) 2024-04-04 09:38:10 -07:00
client.go Merge main, fix conflicts 2024-07-02 08:44:34 -04:00
client_appconfig.go fleetctl gitops --dry-run now errors on duplicate (or conflicting) global/team enroll secrets. (#19344) 2024-05-31 07:01:13 -05:00
client_carves.go Introduce API version 2022-04, deprecate use of /global in paths (#4731) 2022-04-05 11:35:53 -04:00
client_debug.go Merge conflicts 2024-04-16 10:20:59 -05:00
client_hosts.go 17257 validate bulk ops request filters (#17514) 2024-03-11 10:02:51 -06:00
client_labels.go Introduce API version 2022-04, deprecate use of /global in paths (#4731) 2022-04-05 11:35:53 -04:00
client_live_query.go fleetctl, API, copy updates around host identifiers (#20220) 2024-07-09 10:25:01 -07:00
client_live_query_test.go fleetctl, API, copy updates around host identifiers (#20220) 2024-07-09 10:25:01 -07:00
client_mdm.go Get MDM Command Filters (#20154) 2024-07-09 11:06:06 -06:00
client_mdm_test.go fix fleetctl to set different CmdUID for Windows (#14895) 2023-11-02 15:06:37 -03:00
client_packs.go Add fleetctl upgrade-packs command to migrate 2017 packs to queries (#13078) 2023-08-08 08:21:57 -04:00
client_policies.go fleetctl gitops (#16535) 2024-02-09 13:34:57 -06:00
client_profiles.go Unify profiles database identifier to profile_uuid for macOS and Windows profiles. (#15297) 2023-12-04 10:04:06 -05:00
client_queries.go fleetctl gitops (#16535) 2024-02-09 13:34:57 -06:00
client_scripts.go Part 2 of 2: Script Timeout Agent Options (#20356) 2024-07-11 15:03:36 -06:00
client_sessions.go Introduce API version 2022-04, deprecate use of /global in paths (#4731) 2022-04-05 11:35:53 -04:00
client_setup.go fleetctl preview to use v1 setup path, to support previous versions of fleet (#5331) 2022-04-25 10:02:21 -03:00
client_software.go fix: add missing software_package field (#18998) 2024-05-15 13:41:35 -04:00
client_targets.go Add display_name to more results (#8204) 2022-10-14 11:14:18 -03:00
client_teams.go add CLI and endpoints to set software via fleetctl apply (#18876) 2024-05-14 15:06:33 -03:00
client_test.go Improve error messages for parsing MDM config profiles via fleetctl (#19495) 2024-06-14 12:48:00 -05:00
client_trigger.go Add fleetctl trigger command (#8877) 2022-12-06 09:56:54 -06:00
client_users.go Return token when creating API-only users (#19525) 2024-06-13 19:10:27 -03:00
cron_schedules.go Implement schedule triggers (#8747) 2022-11-28 13:28:06 -06:00
cron_schedules_test.go Add instanceID to schedule logging (#10413) 2023-03-13 15:37:03 -03:00
debug_handler.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
debug_handler_test.go Add fleetctl debug migrations command to get missing migrations (#2967) 2021-11-22 14:06:12 -03:00
device_client.go Dont send 500 for every call to /debug/errors endpoint (#19827) 2024-06-21 13:12:06 +01:00
device_client_test.go 7766 backmerge main (#14324) 2023-10-05 11:39:48 -06:00
devices.go Hide "Self-service" in Fleet Desktop and My device page (#20047) 2024-07-02 11:32:49 -05:00
devices_test.go Hide "Self-service" in Fleet Desktop and My device page (#20047) 2024-07-02 11:32:49 -05:00
endpoint_campaigns.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
endpoint_middleware.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
endpoint_middleware_test.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
endpoint_setup.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
endpoint_utils.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
endpoint_utils_test.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
frontend.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
frontend_test.go Return 405 when receiving POST request on the route path (#16750) 2024-02-14 12:40:43 -03:00
full_test.go Return 405 when receiving POST request on the route path (#16750) 2024-02-14 12:40:43 -03:00
global_policies.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
global_policies_test.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
global_schedule.go 17745 queries backend (#18582) 2024-05-07 15:43:10 -04:00
global_schedule_test.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
handler.go Fix calendar duplicated events and other issues (#20443) 2024-07-24 13:40:33 +02:00
handler_test.go MySQL 8.0 Migration (#20225) 2024-07-22 16:27:36 -04:00
hosts.go MySQL 8.0 Migration (#20225) 2024-07-22 16:27:36 -04:00
hosts_test.go Add host's next maintenance window to the hosts/{id} and hosts/identifier/{identifier} endpoints, and render that data on the host details page (#19820) 2024-06-28 10:51:13 -07:00
http_auth.go Remove unused request-id in context (#3632) 2022-01-13 14:12:56 -08:00
http_auth_test.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
http_publicip.go Log all successful logins as activity and all attempts with ip in stderr. (#9095) 2022-12-21 18:29:51 +01:00
installer.go Add UUID to Fleet errors and clean up error msgs (#10411) 2023-03-13 13:44:06 -03:00
installer_test.go Add UUID to Fleet errors and clean up error msgs (#10411) 2023-03-13 13:44:06 -03:00
integration_core_test.go Correctly handle NULL timezones (#20605) 2024-07-22 10:04:29 -07:00
integration_desktop_test.go Dont send 500 for every call to /debug/errors endpoint (#19827) 2024-06-21 13:12:06 +01:00
integration_ds_only_test.go Support async saving of hosts' last seen time (#5640) 2022-05-10 11:29:17 -04:00
integration_enterprise_test.go Fix calendar duplicated events and other issues (#20443) 2024-07-24 13:40:33 +02:00
integration_live_queries_test.go fleetctl, API, copy updates around host identifiers (#20220) 2024-07-09 10:25:01 -07:00
integration_logger_test.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
integration_mdm_ddm_test.go CP Exclude Labels: API changes (#19846) 2024-06-25 15:26:28 -04:00
integration_mdm_dep_test.go use the latest fleetd-base version for MDM commands (#20078) 2024-06-28 16:35:07 -03:00
integration_mdm_lifecycle_test.go Merge main, fix conflicts and reorder migrations 2024-07-03 15:51:49 -04:00
integration_mdm_profiles_test.go CP Exclude Labels: apply exclusion logic when reconciling profiles/declarations (#20013) 2024-07-03 10:20:33 -04:00
integration_mdm_test.go Revert changes to DEP enrollment flow (#20506) 2024-07-16 11:44:48 -05:00
integration_sandbox_test.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
integration_smtp_test.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
integration_sso_test.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
invites.go Do not return empty SSO and SMTP settings for non-global-admins (#12180) 2023-06-07 16:06:36 -03:00
invites_test.go Add UUID to Fleet errors and clean up error msgs (#10411) 2023-03-13 13:44:06 -03:00
jitter.go Migrate special-case endpoints to new pattern (#4511) 2022-03-08 11:27:38 -05:00
jitter_test.go Migrate special-case endpoints to new pattern (#4511) 2022-03-08 11:27:38 -05:00
labels.go disable_issues and host issues support for other API endpoints (#19802) 2024-06-18 14:13:38 -05:00
labels_test.go Bugfix: only count hosts that user has permission to see in count field of label endpoints (#18859) 2024-05-21 12:02:08 -04:00
live_queries.go Moving context.WithoutCancel outside defer. (#17260) 2024-02-29 13:39:27 -03:00
mail_test.go 14729 smtp settings validation for TLS (#15029) 2023-11-21 11:48:21 -07:00
mdm.go Get MDM Command Filters (#20154) 2024-07-09 11:06:06 -06:00
mdm_scep.go 19016 ingest certs on start (#19360) 2024-05-30 18:18:42 -03:00
mdm_test.go Merge main, fix conflicts 2024-07-02 08:44:34 -04:00
mem_failing_policies_set_test.go Fix failing policy sets for redis cluster mode (#3725) 2022-01-17 15:16:54 -05:00
metrics.go
metrics_appconfig.go Fix SMTP e-mail send when SMTP server has credentials (#10758) 2023-03-28 15:23:15 -03:00
metrics_change_email.go
metrics_invites.go
metrics_labels.go add ability to create manual labels (#18303) 2024-04-16 17:22:08 +01:00
metrics_sessions.go add back-end implementation for SSO JIT provisioning (#7182) 2022-08-15 14:42:33 -03:00
metrics_users.go
microsoft_mdm.go use the latest fleetd-base version for MDM commands (#20078) 2024-06-28 16:35:07 -03:00
microsoft_mdm_test.go fix: process Add commands in Windows MDM profiles (#17570) 2024-03-12 15:21:17 -04:00
orbit.go Script Timeout Agent Options Part 1 of 2 (#20266) 2024-07-10 14:33:39 -06:00
orbit_client.go Perform early restart before starting sub-systems and minor refactor of orbit sub-systems (#20610) 2024-07-19 12:44:43 -03:00
orbit_client_test.go Perform early restart before starting sub-systems and minor refactor of orbit sub-systems (#20610) 2024-07-19 12:44:43 -03:00
orbit_test.go Host mdminfo remover (#19885) 2024-06-28 11:09:22 -04:00
osquery.go Added server_settings.query_report_cap (#19692) 2024-06-14 12:24:01 -03:00
osquery_test.go Ignore -1 hardware_serial (#20279) 2024-07-10 09:01:25 -05:00
packs.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
packs_test.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
queries.go Added server_settings.query_report_cap (#19692) 2024-06-14 12:24:01 -03:00
queries_test.go Added server_settings.query_report_cap (#19692) 2024-06-14 12:24:01 -03:00
scheduled_queries.go Combine Schedules and Queries: API changes (#12778) 2023-07-24 20:17:20 -04:00
scheduled_queries_test.go Combine Schedules and Queries: API changes (#12778) 2023-07-24 20:17:20 -04:00
scripts.go Part 2 of 2: Script Timeout Agent Options (#20356) 2024-07-11 15:03:36 -06:00
scripts_test.go Hotfix - Correct Async Run Script Validation Test (#20405) 2024-07-12 09:55:12 -06:00
service.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
service_appconfig.go Do not return empty SSO and SMTP settings for non-global-admins (#12180) 2023-06-07 16:06:36 -03:00
service_appconfig_test.go fleetctl gitops --dry-run now errors on duplicate (or conflicting) global/team enroll secrets. (#19344) 2024-05-31 07:01:13 -05:00
service_campaign_test.go Additional debug info and speculative fix for TestUpdateStatsOnReplica (#19901) 2024-06-21 14:50:12 -05:00
service_campaigns.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
service_errors.go Add UUID to Fleet errors and clean up error msgs (#10411) 2023-03-13 13:44:06 -03:00
service_users.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
sessions.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
sessions_test.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
software.go Fixing tests. (#17073) 2024-02-22 16:03:13 -06:00
software_installers.go Hide "Self-service" in Fleet Desktop and My device page (#20047) 2024-07-02 11:32:49 -05:00
software_installers_test.go fix: don't allow observer and observer+ to download software installers (#19938) 2024-06-25 16:22:51 -04:00
software_test.go Fixing tests. (#17073) 2024-02-22 16:03:13 -06:00
software_titles.go fix: add missing software_package field (#18998) 2024-05-15 13:41:35 -04:00
software_titles_test.go fix: add missing software_package field (#18998) 2024-05-15 13:41:35 -04:00
status.go feat: send back forbidden when live queries are disabled (#14804) 2023-10-31 08:58:36 -04:00
targets.go Set interface for response types (#9121) 2022-12-27 11:26:59 -03:00
targets_test.go Refactor license so it is stored in the context (#8544) 2022-11-15 09:08:05 -05:00
team_policies.go Delete team policies: 404 for nonexistent team (#19516) 2024-06-10 10:46:16 -07:00
team_policies_test.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
team_schedule.go 17745 queries backend (#18582) 2024-05-07 15:43:10 -04:00
team_schedule_test.go Activity feed webhooks backend (#19261) 2024-05-24 11:25:27 -05:00
teams.go GitOps remove teams (#18640) 2024-05-03 08:03:00 -05:00
teams_test.go Enable gitops to create teams with no enroll secrets, or clear enroll secrets for an existing team (#19616) 2024-06-11 13:20:32 -05:00
testing_client.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
testing_utils.go Fix calendar duplicated events and other issues (#20443) 2024-07-24 13:40:33 +02:00
translator.go BadRequest when no payloads present; BadRequest for invalid payload types (#18169) 2024-04-10 10:55:38 -07:00
transport.go disable_issues and host issues support for other API endpoints (#19802) 2024-06-18 14:13:38 -05:00
transport_error.go Merge branch 'main' into 15919-vulnerabilities-page 2024-02-22 16:27:15 -06:00
transport_error_test.go Add UUID to Fleet errors and clean up error msgs (#10411) 2023-03-13 13:44:06 -03:00
transport_setup.go
transport_test.go disable_issues and host issues support for other API endpoints (#19802) 2024-06-18 14:13:38 -05:00
trigger.go Set interface for response types (#9121) 2022-12-27 11:26:59 -03:00
user_roles.go ensure ds.TeamByName returns a 4xx response if no team is found (#12620) 2023-07-13 11:55:05 -03:00
users.go 🧹 friday cleanup party: substitute deprecated import of go-kit (#19774) 2024-06-17 10:27:31 -03:00
users_test.go Return token when creating API-only users (#19525) 2024-06-13 19:10:27 -03:00
validation_setup.go Use new error handling approach in other packages (#2954) 2021-11-22 09:13:26 -05:00
vulnerabilities.go Update MS vulnerability details links to point to NVD (#18991) 2024-05-14 16:00:33 -07:00
vulnerabilities_test.go Ignore leading/trailing whitespace when filtering entities; fix url param handling; test broad range of affected endpoints (#17455) 2024-04-10 11:50:08 -07:00