fleet/server/datastore/mysql/schema.sql

2895 lines
207 KiB
MySQL
Raw Normal View History

/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
CREATE TABLE `abm_tokens` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`organization_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`apple_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`terms_expired` tinyint(1) NOT NULL DEFAULT '0',
`renew_at` timestamp NOT NULL,
`token` blob NOT NULL,
`macos_default_team_id` int unsigned DEFAULT NULL,
`ios_default_team_id` int unsigned DEFAULT NULL,
`ipados_default_team_id` int unsigned DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_abm_tokens_organization_name` (`organization_name`),
KEY `fk_abm_tokens_macos_default_team_id` (`macos_default_team_id`),
KEY `fk_abm_tokens_ios_default_team_id` (`ios_default_team_id`),
KEY `fk_abm_tokens_ipados_default_team_id` (`ipados_default_team_id`),
CONSTRAINT `fk_abm_tokens_ios_default_team_id` FOREIGN KEY (`ios_default_team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_abm_tokens_ipados_default_team_id` FOREIGN KEY (`ipados_default_team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_abm_tokens_macos_default_team_id` FOREIGN KEY (`macos_default_team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `activities` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`user_id` int unsigned DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`user_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`activity_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`details` json DEFAULT NULL,
`streamed` tinyint(1) NOT NULL DEFAULT '0',
`user_email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`fleet_initiated` tinyint(1) NOT NULL DEFAULT '0',
`host_only` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
KEY `fk_activities_user_id` (`user_id`),
KEY `activities_streamed_idx` (`streamed`),
KEY `activities_created_at_idx` (`created_at`),
CONSTRAINT `activities_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `aggregated_stats` (
`id` bigint unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`json_value` json NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`global_stats` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`,`type`,`global_stats`),
KEY `idx_aggregated_stats_updated_at` (`updated_at`),
KEY `aggregated_stats_type_idx` (`type`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `android_devices` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`device_id` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`enterprise_specific_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`last_policy_sync_time` datetime(3) DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`applied_policy_id` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`applied_policy_version` int DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_android_devices_host_id` (`host_id`),
UNIQUE KEY `idx_android_devices_device_id` (`device_id`),
UNIQUE KEY `idx_android_devices_enterprise_specific_id` (`enterprise_specific_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `android_enterprises` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`signup_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`enterprise_id` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`signup_token` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`pubsub_topic_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_id` int unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `android_policy_requests` (
`request_uuid` varchar(36) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`request_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`policy_id` varchar(100) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`payload` json NOT NULL,
`status_code` int NOT NULL,
`error_details` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`applied_policy_version` int DEFAULT NULL,
`policy_version` int DEFAULT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`request_uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `app_config_json` (
`id` int unsigned NOT NULL DEFAULT '1',
`json_value` json NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
Stop setup experience on software install fail: admin (#33968) <!-- Add the related story/sub-task/bug number, like Resolves #123, or remove if NA --> **Related issue:** Resolves #33110 **Related issue:** Resolves #33109 # Details This PR implements the new "cancel setup if any software fails on macos" flag, including both backend and frontend logic. Half of the file changes are updating test expectations / auto-generated schema. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [X] QA'd all new/changed functionality manually ## New Fleet configuration settings - [ ] Verified that the setting is exported via `fleetctl generate-gitops` `macos_setup` is still excluded from generate-girtops - [X] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) Documented [here](https://github.com/fleetdm/fleet/pull/33016/files) - [X] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [X] Verified that any relevant UI is disabled when GitOps mode is enabled <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - New Features - Added a macOS setup option: “Cancel setup if software install fails.” - Configure at global or team level; team settings override global. - Toggle available in Setup Experience > Install software > Advanced options. - Saved state persists and can be updated without leaving the page. - Devices honor the resolved setting during provisioning. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-10-08 16:51:52 +00:00
INSERT INTO `app_config_json` VALUES (1,'{\"mdm\": {\"ios_updates\": {\"deadline\": null, \"minimum_version\": null}, \"macos_setup\": {\"script\": null, \"software\": null, \"bootstrap_package\": null, \"manual_agent_install\": null, \"macos_setup_assistant\": null, \"require_all_software_macos\": false, \"enable_end_user_authentication\": false, \"enable_release_device_manually\": false}, \"macos_updates\": {\"deadline\": null, \"minimum_version\": null}, \"ipados_updates\": {\"deadline\": null, \"minimum_version\": null}, \"macos_settings\": {\"custom_settings\": null}, \"macos_migration\": {\"mode\": \"\", \"enable\": false, \"webhook_url\": \"\"}, \"windows_updates\": {\"deadline_days\": null, \"grace_period_days\": null}, \"android_settings\": {\"custom_settings\": null}, \"apple_server_url\": \"\", \"windows_settings\": {\"custom_settings\": null}, \"apple_bm_terms_expired\": false, \"apple_business_manager\": null, \"enable_disk_encryption\": false, \"enabled_and_configured\": false, \"end_user_authentication\": {\"idp_name\": \"\", \"metadata\": \"\", \"entity_id\": \"\", \"issuer_uri\": \"\", \"metadata_url\": \"\"}, \"volume_purchasing_program\": null, \"windows_migration_enabled\": false, \"windows_require_bitlocker_pin\": null, \"android_enabled_and_configured\": false, \"windows_enabled_and_configured\": false, \"apple_bm_enabled_and_configured\": false}, \"gitops\": {\"repository_url\": \"\", \"gitops_mode_enabled\": false}, \"scripts\": null, \"features\": {\"enable_host_users\": true, \"enable_software_inventory\": false}, \"org_info\": {\"org_name\": \"\", \"contact_url\": \"\", \"org_logo_url\": \"\", \"org_logo_url_light_background\": \"\"}, \"integrations\": {\"jira\": null, \"zendesk\": null, \"google_calendar\": null, \"conditional_access_enabled\": null}, \"sso_settings\": {\"idp_name\": \"\", \"metadata\": \"\", \"entity_id\": \"\", \"enable_sso\": false, \"issuer_uri\": \"\", \"metadata_url\": \"\", \"idp_image_url\": \"\", \"sso_server_url\": \"\", \"enable_jit_role_sync\": false, \"enable_sso_idp_login\": false, \"enable_jit_provisioning\": false}, \"agent_options\": {\"config\": {\"options\": {\"logger_plugin\": \"tls\", \"pack_delimiter\": \"/\", \"logger_tls_period\": 10, \"distributed_plugin\": \"tls\", \"disable_distributed\": false, \"logger_tls_endpoint\": \"/api/osquery/log\", \"distributed_interval\": 10, \"distributed_tls_max_attempts\": 3}, \"decorators\": {\"load\": [\"SELECT uuid AS host_uuid FROM system_info;\", \"SELECT hostname AS hostname FROM system_info;\"]}}, \"overrides\": {}}, \"fleet_desktop\": {\"transparency_url\": \"\"}, \"smtp_settings\": {\"port\": 587, \"domain\": \"\", \"server\": \"\", \"password\": \"\", \"user_name\": \"\", \"configured\": false, \"enable_smtp\": false, \"enable_ssl_tls\": true, \"sender_address\": \"\", \"enable_start_tls\": true, \"verify_ssl_certs\": true, \"authentication_type\": \"0\", \"authentication_method\": \"0\"}, \"server_settings\": {\"server_url\": \"\", \"enable_analytics\": false, \"query_report_cap\": 0, \"scripts_disabled\": false, \"deferred_save_host\": false, \"live_query_disabled\": false, \"ai_features_disabled\": false, \"query_reports_disabled\": false}, \"webhook_settings\": {\"interval\": \"0s\", \"activities_webhook\": {\"destination_url\": \"\", \"enable_activities_webhook\": false}, \"host_status_webhook\": {\"days_count\": 0, \"destination_url\": \"\", \"host_percentage\": 0, \"enable_host_status_webhook\": false}, \"vulnerabilities_webhook\": {\"destination_url\": \"\", \"host_batch_size\": 0, \"enable_vulnerabilities_webhook\": false}, \"failing_policies_webhook\": {\"policy_ids\": null, \"destination_url\": \"\", \"host_batch_size\": 0, \"enable_failing_policies_webhook\": false}}, \"host_expiry_settings\": {\"host_expiry_window\": 0, \"host_expiry_enabled\": false}, \"vulnerability_settings\": {\"databases_path\": \"\"}, \"activity_expiry_settings\": {\"activity_expiry_window\": 0, \"activity_expiry_enabled\": false}}','2020-01-01 01:01:01','2020-01-01 01:01:01');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Update batch script tables (#31508) for #31505 # Checklist for submitter - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually no changed functionality ## Database migrations - [X] Checked table schema to confirm autoupdate - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ ] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). collate for both these tables is previously set to `utf8mb4_0900_ai_ci`, but they don't contain user-settable strings <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Expanded batch activity tracking with new fields for job ID, status, and activity type. * Enhanced database structure to support more flexible batch activity and host result management. * **Bug Fixes** * Updated references throughout the application to use the new table names for batch activities and host results, ensuring consistency. * **Tests** * Added and updated tests to validate the new batch activity schema and migration behavior. * **Chores** * Applied database migration to rename and restructure relevant tables for improved clarity and extensibility. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-05 18:40:23 +00:00
CREATE TABLE `batch_activities` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
Update batch script tables (#31508) for #31505 # Checklist for submitter - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually no changed functionality ## Database migrations - [X] Checked table schema to confirm autoupdate - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ ] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). collate for both these tables is previously set to `utf8mb4_0900_ai_ci`, but they don't contain user-settable strings <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Expanded batch activity tracking with new fields for job ID, status, and activity type. * Enhanced database structure to support more flexible batch activity and host result management. * **Bug Fixes** * Updated references throughout the application to use the new table names for batch activities and host results, ensuring consistency. * **Tests** * Added and updated tests to validate the new batch activity schema and migration behavior. * **Chores** * Applied database migration to rename and restructure relevant tables for improved clarity and extensibility. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-05 18:40:23 +00:00
`script_id` int unsigned NOT NULL,
`execution_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`user_id` int unsigned DEFAULT NULL,
`job_id` int unsigned DEFAULT NULL,
`status` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`activity_type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`num_targeted` int unsigned DEFAULT NULL,
`num_pending` int unsigned DEFAULT NULL,
`num_ran` int unsigned DEFAULT NULL,
`num_errored` int unsigned DEFAULT NULL,
`num_incompatible` int unsigned DEFAULT NULL,
`num_canceled` int unsigned DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`started_at` datetime DEFAULT NULL,
`finished_at` datetime DEFAULT NULL,
`canceled` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
Update batch script tables (#31508) for #31505 # Checklist for submitter - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually no changed functionality ## Database migrations - [X] Checked table schema to confirm autoupdate - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ ] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). collate for both these tables is previously set to `utf8mb4_0900_ai_ci`, but they don't contain user-settable strings <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Expanded batch activity tracking with new fields for job ID, status, and activity type. * Enhanced database structure to support more flexible batch activity and host result management. * **Bug Fixes** * Updated references throughout the application to use the new table names for batch activities and host results, ensuring consistency. * **Tests** * Added and updated tests to validate the new batch activity schema and migration behavior. * **Chores** * Applied database migration to rename and restructure relevant tables for improved clarity and extensibility. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-05 18:40:23 +00:00
UNIQUE KEY `idx_batch_script_executions_execution_id` (`execution_id`),
KEY `batch_script_executions_script_id` (`script_id`),
KEY `idx_batch_activities_status` (`status`),
CONSTRAINT `batch_script_executions_script_id` FOREIGN KEY (`script_id`) REFERENCES `scripts` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Update batch script tables (#31508) for #31505 # Checklist for submitter - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually no changed functionality ## Database migrations - [X] Checked table schema to confirm autoupdate - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ ] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). collate for both these tables is previously set to `utf8mb4_0900_ai_ci`, but they don't contain user-settable strings <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Expanded batch activity tracking with new fields for job ID, status, and activity type. * Enhanced database structure to support more flexible batch activity and host result management. * **Bug Fixes** * Updated references throughout the application to use the new table names for batch activities and host results, ensuring consistency. * **Tests** * Added and updated tests to validate the new batch activity schema and migration behavior. * **Chores** * Applied database migration to rename and restructure relevant tables for improved clarity and extensibility. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-05 18:40:23 +00:00
CREATE TABLE `batch_activity_host_results` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
Update batch script tables (#31508) for #31505 # Checklist for submitter - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually no changed functionality ## Database migrations - [X] Checked table schema to confirm autoupdate - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ ] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). collate for both these tables is previously set to `utf8mb4_0900_ai_ci`, but they don't contain user-settable strings <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Expanded batch activity tracking with new fields for job ID, status, and activity type. * Enhanced database structure to support more flexible batch activity and host result management. * **Bug Fixes** * Updated references throughout the application to use the new table names for batch activities and host results, ensuring consistency. * **Tests** * Added and updated tests to validate the new batch activity schema and migration behavior. * **Chores** * Applied database migration to rename and restructure relevant tables for improved clarity and extensibility. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-05 18:40:23 +00:00
`batch_execution_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`host_id` int unsigned NOT NULL,
`host_execution_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`error` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `unique_batch_host_results_execution_hostid` (`batch_execution_id`,`host_id`),
Update batch script tables (#31508) for #31505 # Checklist for submitter - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [ ] QA'd all new/changed functionality manually no changed functionality ## Database migrations - [X] Checked table schema to confirm autoupdate - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ ] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). collate for both these tables is previously set to `utf8mb4_0900_ai_ci`, but they don't contain user-settable strings <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Expanded batch activity tracking with new fields for job ID, status, and activity type. * Enhanced database structure to support more flexible batch activity and host result management. * **Bug Fixes** * Updated references throughout the application to use the new table names for batch activities and host results, ensuring consistency. * **Tests** * Added and updated tests to validate the new batch activity schema and migration behavior. * **Chores** * Applied database migration to rename and restructure relevant tables for improved clarity and extensibility. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-05 18:40:23 +00:00
KEY `idx_batch_script_execution_host_result_execution_id` (`batch_execution_id`),
CONSTRAINT `batch_script_batch_id` FOREIGN KEY (`batch_execution_id`) REFERENCES `batch_activities` (`execution_id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `ca_config_assets` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`type` enum('digicert','custom_scep_proxy') COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`value` blob NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_ca_config_assets_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-15 00:15:35 +00:00
CREATE TABLE `calendar_events` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
2024-03-15 00:15:35 +00:00
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`start_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`end_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`event` json NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`timezone` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`uuid_bin` binary(16) NOT NULL,
`uuid` varchar(36) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS (insert(insert(insert(insert(hex(`uuid_bin`),9,0,_utf8mb4'-'),14,0,_utf8mb4'-'),19,0,_utf8mb4'-'),24,0,_utf8mb4'-')) VIRTUAL,
PRIMARY KEY (`id`),
Check for calendar updates after callbacks from Google (#20156) #19352 Video explaining code changes: https://www.loom.com/share/370200a276b84aa388effd6ebd762e01?sid=038508c4-f3c2-40c0-baf6-6b6df682d1f0 In maintenance windows using Google Calendar, calendar event is now recreated within 30 seconds if deleted or moved to the past. - Added new endpoint for Google Calendar: `/api/_version_/fleet/calendar/webhook/{event_uuid}` - Added UUID to `calendar_events` table to make webhook lookup more efficient - webhook endpoint will only recreate event if needed -- it will not fire webhook. Webhook is still done by the cron job. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes:
2024-07-08 15:20:03 +00:00
UNIQUE KEY `idx_one_calendar_event_per_email` (`email`),
UNIQUE KEY `idx_calendar_events_uuid_bin_unique` (`uuid_bin`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2024-03-15 00:15:35 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `carve_blocks` (
`metadata_id` int unsigned NOT NULL,
`block_id` int NOT NULL,
`data` longblob,
PRIMARY KEY (`metadata_id`,`block_id`),
CONSTRAINT `carve_blocks_ibfk_1` FOREIGN KEY (`metadata_id`) REFERENCES `carve_metadata` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `carve_metadata` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`block_count` int unsigned NOT NULL,
`block_size` int unsigned NOT NULL,
`carve_size` bigint unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`carve_id` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`request_id` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`session_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`expired` tinyint DEFAULT '0',
`max_block` int DEFAULT '-1',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`error` text COLLATE utf8mb4_unicode_ci,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_session_id` (`session_id`),
UNIQUE KEY `idx_name` (`name`),
KEY `host_id` (`host_id`),
CONSTRAINT `carve_metadata_ibfk_1` FOREIGN KEY (`host_id`) REFERENCES `hosts` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Hydrant CA Feature Branch (#31807) There are still some TODOs particularly within Gitops test code which will be worked on in a followup PR # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually For unreleased bug fixes in a release candidate, one of: - [x] Confirmed that the fix is not expected to adversely impact load test results - [x] Alerted the release DRI if additional load testing is needed ## Database migrations - [x] Checked table schema to confirm autoupdate - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). ## New Fleet configuration settings - [ ] Setting(s) is/are explicitly excluded from GitOps If you didn't check the box above, follow this checklist for GitOps-enabled settings: - [ ] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled --------- Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Magnus Jensen <magnus@fleetdm.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2025-09-04 16:39:41 +00:00
CREATE TABLE `certificate_authorities` (
`id` int NOT NULL AUTO_INCREMENT,
`type` enum('digicert','ndes_scep_proxy','custom_scep_proxy','hydrant','smallstep') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
Hydrant CA Feature Branch (#31807) There are still some TODOs particularly within Gitops test code which will be worked on in a followup PR # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually For unreleased bug fixes in a release candidate, one of: - [x] Confirmed that the fix is not expected to adversely impact load test results - [x] Alerted the release DRI if additional load testing is needed ## Database migrations - [x] Checked table schema to confirm autoupdate - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). ## New Fleet configuration settings - [ ] Setting(s) is/are explicitly excluded from GitOps If you didn't check the box above, follow this checklist for GitOps-enabled settings: - [ ] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled --------- Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Magnus Jensen <magnus@fleetdm.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2025-09-04 16:39:41 +00:00
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`url` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`api_token_encrypted` blob,
`profile_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`certificate_common_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`certificate_user_principal_names` json DEFAULT NULL,
`certificate_seat_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`admin_url` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`username` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`password_encrypted` blob,
`challenge_url` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
Hydrant CA Feature Branch (#31807) There are still some TODOs particularly within Gitops test code which will be worked on in a followup PR # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually For unreleased bug fixes in a release candidate, one of: - [x] Confirmed that the fix is not expected to adversely impact load test results - [x] Alerted the release DRI if additional load testing is needed ## Database migrations - [x] Checked table schema to confirm autoupdate - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). ## New Fleet configuration settings - [ ] Setting(s) is/are explicitly excluded from GitOps If you didn't check the box above, follow this checklist for GitOps-enabled settings: - [ ] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified the setting is documented in a separate PR to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled --------- Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Magnus Jensen <magnus@fleetdm.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
2025-09-04 16:39:41 +00:00
`challenge_encrypted` blob,
`client_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`client_secret_encrypted` blob,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_ca_type_name` (`type`,`name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `challenges` (
`challenge` char(32) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`challenge`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `cron_stats` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`instance` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`stats_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Monitor and alert on errors in cron jobs (#24347) for #19930 # Checklist for submitter - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added/updated tests - [X] If database migrations are included, checked table schema to confirm autoupdate - [X] Manual QA for all new/changed functionality # Details This PR adds a new feature to the existing monitoring add-on. The add-on will now send an SNS alert whenever a scheduled job like "vulnerabilities" or "apple_mdm_apns_pusher" exits early due to errors. The alert contains the job type and the set of errors (there can be multiple, since jobs can have multiple sub-jobs). By default the SNS topic for this new alert is the same as the one for the existing cron system alerts, but it can be configured to use a separate topic (e.g. dogfood instance will post to a separate slack channel). The actual changes are: **On the server side:** - Add errors field to cron_stats table (json DEFAULT NULL) - Added errors var to `Schedule` struct to collect errors from jobs - In `RunAllJobs`, collect err from job into new errors var - Update `Schedule.updateStats`and `CronStats.UpdateCronStats`to accept errors argument - If provided, update errors field of cron_stats table **On the monitor side:** - Add new SQL query to look for all completed schedules since last run with non-null errors - send SNS with job ID, name, errors # Testing New automated testing was added for the functional code that gathers and stores errors from cron runs in the database. To test the actual Lambda, I added a row in my `cron_stats` table with errors, then compiled and ran the Lambda executable locally, pointing it to my local mysql and localstack instances: ``` 2024/12/03 14:43:54 main.go:258: Lambda execution environment not found. Falling back to local execution. 2024/12/03 14:43:54 main.go:133: Connected to database! 2024/12/03 14:43:54 main.go:161: Row vulnerabilities last updated at 2024-11-27 03:30:03 +0000 UTC 2024/12/03 14:43:54 main.go:163: *** 1h hasn't updated in more than vulnerabilities, alerting! (status completed) 2024/12/03 14:43:54 main.go:70: Sending SNS Message 2024/12/03 14:43:54 main.go:74: Sending 'Environment: dev Message: Fleet cron 'vulnerabilities' hasn't updated in more than 1h. Last status was 'completed' at 2024-11-27 03:30:03 +0000 UTC.' to 'arn:aws:sns:us-east-1:000000000000:topic1' 2024/12/03 14:43:54 main.go:82: { MessageId: "260864ff-4cc9-4951-acea-cef883b2de5f" } 2024/12/03 14:43:54 main.go:198: *** mdm_apple_profile_manager job had errors, alerting! (errors {"something": "wrong"}) 2024/12/03 14:43:54 main.go:70: Sending SNS Message 2024/12/03 14:43:54 main.go:74: Sending 'Environment: dev Message: Fleet cron 'mdm_apple_profile_manager' (last updated 2024-12-03 20:34:14 +0000 UTC) raised errors during its run: {"something": "wrong"}.' to 'arn:aws:sns:us-east-1:000000000000:topic1' 2024/12/03 14:43:54 main.go:82: { MessageId: "5cd085ef-89f6-42c1-8470-d80a22b295f8"
2024-12-19 21:55:29 +00:00
`errors` json DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_cron_stats_name_created_at` (`name`,`created_at`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2022-06-01 16:06:57 +00:00
CREATE TABLE `cve_meta` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`cve` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
`cvss_score` double DEFAULT NULL,
`epss_probability` double DEFAULT NULL,
`cisa_known_exploit` tinyint(1) DEFAULT NULL,
`published` timestamp NULL DEFAULT NULL,
`description` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
PRIMARY KEY (`cve`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Allow configuring webhook policy automations for "No team" (#32129) Fixes #32060 This PR adds: - new default_team_config_json table - caching of config from that table, including deep copy methods -- all of this is not absolutely needed for this change since we are only using `webhook_settings.failing_policies_webhook` here but added for completeness/future - teams/0 API updates - GitOps updates - generate gitops updates Future PRs will add: - ticket automation - primo mode migration - frontend changes - documentation # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [x] Added/updated automated tests - [x] QA'd all new/changed functionality manually ## Database migrations - [x] Checked table schema to confirm autoupdate ## New Fleet configuration settings - [x] Verified that the setting is exported via `fleetctl generate-gitops` - [x] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - New Features - Configure failing-policy webhooks for “No team” via GitOps (no-team.yml) and API, including enable/disable, destination URL, policy IDs, and batch size; settings clear when omitted. - GitOps and CLI now read/apply the real “No team” settings with dry-run support. - Policy automation evaluates hosts without a team and triggers “No team” webhooks when applicable. - GET/PATCH team 0 returns/accepts a minimal, webhook-focused config. - Chores - Added persistence and caching for the default “No team” configuration. - Introduced a database table to store the default configuration. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-08-28 21:38:27 +00:00
CREATE TABLE `default_team_config_json` (
`id` int unsigned NOT NULL DEFAULT '1',
`json_value` json NOT NULL,
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`),
CONSTRAINT `default_team_config_id` CHECK ((`id` = 1))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `default_team_config_json` VALUES (1,'{\"mdm\": {\"macos_setup\": {\"bootstrap_package\": null, \"macos_setup_assistant\": null, \"enable_end_user_authentication\": false, \"enable_release_device_manually\": false}, \"macos_updates\": {\"deadline\": null, \"minimum_version\": null}, \"macos_settings\": {\"custom_settings\": null, \"enable_end_user_authentication\": false}, \"windows_updates\": {\"deadline_days\": null, \"grace_period_days\": null}, \"windows_settings\": {\"custom_settings\": null}, \"enable_disk_encryption\": false}, \"scripts\": null, \"features\": {\"enable_host_users\": true, \"additional_queries\": null, \"detail_query_overrides\": null, \"enable_software_inventory\": true, \"enable_host_software_via_scan\": true, \"enable_host_operating_system_details\": true}, \"software\": null, \"integrations\": {\"jira\": null, \"zendesk\": null, \"google_calendar\": null}, \"agent_options\": null, \"webhook_settings\": {\"host_status_webhook\": null, \"failing_policies_webhook\": {\"policy_ids\": [], \"destination_url\": \"\", \"host_batch_size\": 0, \"enable_failing_policies_webhook\": false}}, \"host_expiry_settings\": {\"jitter_percent\": 0, \"host_expiry_window\": 0, \"host_expiry_enabled\": false}}','2020-01-01 01:01:01.000000','2020-01-01 01:01:01.000000');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `distributed_query_campaign_targets` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`type` int DEFAULT NULL,
`distributed_query_campaign_id` int unsigned DEFAULT NULL,
`target_id` int unsigned DEFAULT NULL,
Add `CleanupCompletedCampaignTargets` to cleanup old campaign targets. (#32385) Fixes #31432 - Added campaign target cleanup: Deletes targets from campaigns completed >24h ago. Uses 10% or 50k min per run, processes in 10k batches. Added DB index, integrated into hourly cron, includes tests. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - New Features - Automatic cleanup of live query campaign targets 24 hours after campaign completion to reduce clutter and storage usage. - Chores - Added a database index to speed up live query target operations for improved performance at scale. - Enhanced scheduled maintenance to log cleanup counts and execution time for better observability. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-08-28 16:04:05 +00:00
PRIMARY KEY (`id`),
KEY `idx_distributed_query_campaign_targets_campaign_id` (`distributed_query_campaign_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `distributed_query_campaigns` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`query_id` int unsigned DEFAULT NULL,
`status` int DEFAULT NULL,
`user_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `email_changes` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`user_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token` varchar(128) COLLATE utf8mb4_unicode_ci NOT NULL,
`new_email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_email_changes_token` (`token`) USING BTREE,
KEY `fk_email_changes_users` (`user_id`),
CONSTRAINT `fk_email_changes_users` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `enroll_secrets` (
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`secret` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`team_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`secret`),
KEY `fk_enroll_secrets_team_id` (`team_id`),
CONSTRAINT `enroll_secrets_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `eulas` (
`id` int unsigned NOT NULL,
`token` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`bytes` longblob,
`created_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`sha256` binary(32) DEFAULT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `fleet_maintained_apps` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`slug` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`unique_identifier` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_fleet_library_apps_token` (`slug`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `fleet_variables` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`is_prefix` tinyint(1) NOT NULL DEFAULT '0',
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_fleet_variables_name_is_prefix` (`name`,`is_prefix`)
) ENGINE=InnoDB AUTO_INCREMENT=17 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `fleet_variables` VALUES (1,'FLEET_VAR_NDES_SCEP_CHALLENGE',0,'2025-04-22 00:00:00.000000'),(2,'FLEET_VAR_NDES_SCEP_PROXY_URL',0,'2025-04-22 00:00:00.000000'),(3,'FLEET_VAR_HOST_END_USER_EMAIL_IDP',0,'2025-04-22 00:00:00.000000'),(4,'FLEET_VAR_HOST_HARDWARE_SERIAL',0,'2025-04-22 00:00:00.000000'),(5,'FLEET_VAR_HOST_END_USER_IDP_USERNAME',0,'2025-04-22 00:00:00.000000'),(6,'FLEET_VAR_HOST_END_USER_IDP_USERNAME_LOCAL_PART',0,'2025-04-22 00:00:00.000000'),(7,'FLEET_VAR_HOST_END_USER_IDP_GROUPS',0,'2025-04-22 00:00:00.000000'),(8,'FLEET_VAR_DIGICERT_DATA_',1,'2025-04-22 00:00:00.000000'),(9,'FLEET_VAR_DIGICERT_PASSWORD_',1,'2025-04-22 00:00:00.000000'),(10,'FLEET_VAR_CUSTOM_SCEP_CHALLENGE_',1,'2025-04-22 00:00:00.000000'),(11,'FLEET_VAR_CUSTOM_SCEP_PROXY_URL_',1,'2025-04-22 00:00:00.000000'),(12,'FLEET_VAR_SCEP_RENEWAL_ID',0,'2025-04-30 00:00:00.000000'),(13,'FLEET_VAR_HOST_END_USER_IDP_DEPARTMENT',0,'2025-06-27 00:00:00.000000'),(14,'FLEET_VAR_HOST_UUID',0,'2025-08-08 00:00:00.000000'),(15,'FLEET_VAR_HOST_END_USER_IDP_FULL_NAME',0,'2025-08-25 00:00:00.000000'),(16,'FLEET_VAR_SCEP_WINDOWS_CERTIFICATE_ID',0,'2025-10-22 00:00:00.000000');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_activities` (
`host_id` int unsigned NOT NULL,
`activity_id` int unsigned NOT NULL,
PRIMARY KEY (`host_id`,`activity_id`),
KEY `fk_host_activities_activity_id` (`activity_id`),
CONSTRAINT `host_activities_ibfk_1` FOREIGN KEY (`activity_id`) REFERENCES `activities` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_additional` (
`host_id` int unsigned NOT NULL,
`additional` json DEFAULT NULL,
PRIMARY KEY (`host_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_batteries` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`serial_number` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`cycle_count` int NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`health` varchar(40) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_batteries_host_id_serial_number` (`host_id`,`serial_number`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-15 00:15:35 +00:00
CREATE TABLE `host_calendar_events` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`calendar_event_id` int unsigned NOT NULL,
`webhook_status` tinyint NOT NULL,
2024-03-15 00:15:35 +00:00
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_one_calendar_event_per_host` (`host_id`),
KEY `calendar_event_id` (`calendar_event_id`),
CONSTRAINT `host_calendar_events_ibfk_1` FOREIGN KEY (`calendar_event_id`) REFERENCES `calendar_events` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_certificate_sources` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_certificate_id` bigint unsigned NOT NULL,
`source` enum('system','user') COLLATE utf8mb4_unicode_ci NOT NULL,
`username` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_certificate_sources_unique` (`host_certificate_id`,`source`,`username`),
CONSTRAINT `fk_host_certificate_sources_host_certificate_id` FOREIGN KEY (`host_certificate_id`) REFERENCES `host_certificates` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_certificates` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`not_valid_after` datetime(6) NOT NULL,
`not_valid_before` datetime(6) NOT NULL,
`certificate_authority` tinyint(1) NOT NULL,
`common_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`key_algorithm` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`key_strength` int NOT NULL,
`key_usage` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`serial` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`signing_algorithm` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`subject_country` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`subject_org` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`subject_org_unit` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`subject_common_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`issuer_country` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`issuer_org` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`issuer_org_unit` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`issuer_common_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sha1_sum` binary(20) NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`deleted_at` datetime(6) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_host_certs_hid_cn` (`host_id`,`common_name`),
KEY `idx_host_certs_not_valid_after` (`host_id`,`not_valid_after`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_dep_assignments` (
`host_id` int unsigned NOT NULL,
`added_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`deleted_at` timestamp NULL DEFAULT NULL,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`assign_profile_response` varchar(15) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`response_updated_at` timestamp NULL DEFAULT NULL,
`retry_job_id` int unsigned NOT NULL DEFAULT '0',
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
`abm_token_id` int unsigned DEFAULT NULL,
Skip setup experience during AxM based migrations (#32822) Fixes #32096 The gist of the fix is that when syncing devices from DEP we save the migration deadline to our host_dep_assignments table. The next enrollment, which we assume should be the migration, looks at host_dep_assignments, sees that mdm_migration_deadline is non-Null and mdm_migration_completed is NULL, and uses that as the signal that a migration is in progress and skips enqueuing setup experience items. It then marks the migration as complete which sets mdm_migration_completed = mdm_migration_deadline. Once this is set setup experience will run as normal unless mdm_migration_completed gets set to NULL and/or mdm_migration_deadline gets set to a value in the future(which e.g. would happen if the customer assigned to another MDM server then assigned to migrate to fleet again) DB test failure is expected here because it won't like the migration timestamp but that is a necessary failure because this fix is going to be backported into 4.73 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually For unreleased bug fixes in a release candidate, one of: - [x] Confirmed that the fix is not expected to adversely impact load test results - [x] Alerted the release DRI if additional load testing is needed ## Database migrations - [x] Checked table schema to confirm autoupdate - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * New Features * Tracks and stores Apple DEP MDM migration deadlines per device/host. * Detects “migration in progress” during DEP sync and check-in. * Automatically marks migration complete and skips Setup Assistant items while migration is in progress to prevent conflicts. * Bug Fixes * Improved DEP compatibility by updating the protocol version and User-Agent used for Apple’s APIs, reducing the chance of blocked or rejected requests. * Migrations * Adds fields to support migration deadlines and completion status (no action required). <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
2025-09-11 13:40:40 +00:00
`mdm_migration_deadline` timestamp(6) NULL DEFAULT NULL,
`mdm_migration_completed` timestamp(6) NULL DEFAULT NULL,
PRIMARY KEY (`host_id`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
KEY `idx_hdep_response` (`assign_profile_response`,`response_updated_at`),
KEY `fk_host_dep_assignments_abm_token_id` (`abm_token_id`),
CONSTRAINT `fk_host_dep_assignments_abm_token_id` FOREIGN KEY (`abm_token_id`) REFERENCES `abm_tokens` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2022-03-08 12:05:53 +00:00
CREATE TABLE `host_device_auth` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
2022-03-08 12:05:53 +00:00
PRIMARY KEY (`host_id`),
UNIQUE KEY `idx_host_device_auth_token` (`token`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2022-03-08 12:05:53 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_disk_encryption_keys` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`base64_encrypted` text COLLATE utf8mb4_unicode_ci NOT NULL,
`base64_encrypted_salt` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`key_slot` tinyint unsigned DEFAULT NULL,
`decryptable` tinyint(1) DEFAULT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`reset_requested` tinyint(1) NOT NULL DEFAULT '0',
`client_error` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`host_id`),
KEY `idx_host_disk_encryption_keys_decryptable` (`decryptable`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_disk_encryption_keys_archive` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`hardware_serial` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`base64_encrypted` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`base64_encrypted_salt` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`key_slot` tinyint unsigned DEFAULT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
KEY `idx_host_disk_encryption_keys_archive_host_created_at` (`host_id`,`created_at` DESC)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_disks` (
`host_id` int unsigned NOT NULL,
`gigs_disk_space_available` decimal(10,2) NOT NULL DEFAULT '0.00',
`percent_disk_space_available` decimal(10,2) NOT NULL DEFAULT '0.00',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`encrypted` tinyint(1) DEFAULT NULL,
`gigs_total_disk_space` decimal(10,2) NOT NULL DEFAULT '0.00',
Implement BitLocker "action required" status (#31451) for #31182 # Details This PR implements the "Action Required" state for Windows host disk encryption. This includes updates to reporting for: * disk encryption summary (`GET /fleet/disk_encryption`) * config profiles summary (`GET /configuration_profiles/summary`) * config profile status ( `GET /configuration_profiles/{profile_uuid}/status`) For disk encryption summary, the statuses are now determined according to [the rules in the Figma](https://www.figma.com/design/XbhlPuEJxQtOgTZW9EOJZp/-28133-Enforce-BitLocker-PIN?node-id=5484-928&t=JB13g8zQ2QDVEmPB-0). TL;DR if the criteria for "verified" or "verifying" are set, but a required PIN is not set, we report a host as "action required". For profiles, I followed what seems to be the existing pattern and set the profile status to "pending" if the disk encryption status is "action required". This is what we do for hosts with the "enforcing" or "removing enforcement" statuses. A lot of the changes in these files are due to the creation of the `fleet.DiskEncryptionConfig` struct to hold info about disk encryption config, and passing variables of that type to various functions instead of passing a `bool` to indicate whether encryption is enabled. Other than that, the functional changes are constrained to a few files. > Note: to get the "require bitlocker pin" UI, compile the front end with: ``` SHOW_BITLOCKER_PIN_OPTION=true NODE_ENV=development yarn run webpack --progress --watch ``` # Checklist for submitter If some of the following don't apply, delete the relevant line. - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. Changelog will be added when feature is complete. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) ## Testing - [X] Added/updated automated tests - [X] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [ ] QA'd all new/changed functionality manually Could use some help testing this end-to-end. I was able to test the banners showing up correctly, but testing the Disk Encryption table requires some Windows-MDM-fu (I just get all zeroes). ## Database migrations - [X] Checked table schema to confirm autoupdate - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`).
2025-08-05 16:23:27 +00:00
`tpm_pin_set` tinyint(1) DEFAULT '0',
`gigs_all_disk_space` decimal(10,2) DEFAULT NULL,
PRIMARY KEY (`host_id`),
KEY `idx_host_disks_gigs_disk_space_available` (`gigs_disk_space_available`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2022-10-08 12:57:46 +00:00
CREATE TABLE `host_display_names` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`display_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
2022-10-08 12:57:46 +00:00
PRIMARY KEY (`host_id`),
KEY `display_name` (`display_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2022-10-08 12:57:46 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_emails` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`source` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `idx_host_emails_host_id_email` (`host_id`,`email`),
KEY `idx_host_emails_email` (`email`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add SCEP endpoint for host identity. (#30589) Fixes #30458 Contributor docs PR: https://github.com/fleetdm/fleet/pull/30651 # Checklist for submitter - We will add changes file later. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Added/updated automated tests - Did not do manual QA since the SCEP client I have doesn't support ECC. Will rely on next subtasks for manual QA. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Introduced Host Identity SCEP (Simple Certificate Enrollment Protocol) support, enabling secure host identity certificate enrollment and management. * Added new API endpoints for Host Identity SCEP, including certificate issuance and retrieval. * Implemented MySQL-backed storage and management for host identity SCEP certificates and serials. * Added new database tables for storing host identity SCEP certificates and serial numbers. * Provided utilities for encoding certificates and keys, and handling ECDSA public keys. * **Bug Fixes** * None. * **Tests** * Added comprehensive integration and unit tests for Host Identity SCEP functionality, including certificate issuance, validation, and error scenarios. * **Chores** * Updated test utilities to support unique test names and new SCEP storage options. * Extended mock datastore and interfaces for new host identity certificate methods. * **Documentation** * Added comments and documentation for new SCEP-related interfaces, methods, and database schema changes. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-11 14:44:07 +00:00
CREATE TABLE `host_identity_scep_certificates` (
`serial` bigint unsigned NOT NULL,
`host_id` int unsigned DEFAULT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`not_valid_before` datetime NOT NULL,
`not_valid_after` datetime NOT NULL,
`certificate_pem` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`public_key_raw` varbinary(100) NOT NULL,
`revoked` tinyint(1) NOT NULL DEFAULT '0',
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`serial`),
KEY `idx_host_id_scep_name` (`name`),
KEY `idx_host_id_scep_host_id` (`host_id`),
CONSTRAINT `host_identity_scep_certificates_ibfk_1` FOREIGN KEY (`serial`) REFERENCES `host_identity_scep_serials` (`serial`),
CONSTRAINT `host_identity_scep_certificates_chk_1` CHECK ((substr(`certificate_pem`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----'))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_identity_scep_serials` (
`serial` bigint unsigned NOT NULL AUTO_INCREMENT,
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`serial`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Added host_issues (#19654) * /api/latest/fleet/hosts now returns `critical_vulnerabilities_count` for premium users. This data is held in the new `host_issues` table. The failing policies total is updated in real-time, while the critical vulnerabilities total is updated every hour after vulnerabilities job. * /api/latest/fleet/hosts can be sorted by total_issues_count by specifying `order_key=issues` query parameter. #18115 Video commentary: https://www.loom.com/share/ff4425e217db43d6b3ba4aa075374808 @getvictor TODO: Replacing `disable_failing_policies` with `disable_issues` will be done in a separate PR. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality
2024-06-17 17:15:42 +00:00
CREATE TABLE `host_issues` (
`host_id` int unsigned NOT NULL,
`failing_policies_count` int unsigned NOT NULL DEFAULT '0',
`critical_vulnerabilities_count` int unsigned NOT NULL DEFAULT '0',
`total_issues_count` int unsigned NOT NULL DEFAULT '0',
Added host_issues (#19654) * /api/latest/fleet/hosts now returns `critical_vulnerabilities_count` for premium users. This data is held in the new `host_issues` table. The failing policies total is updated in real-time, while the critical vulnerabilities total is updated every hour after vulnerabilities job. * /api/latest/fleet/hosts can be sorted by total_issues_count by specifying `order_key=issues` query parameter. #18115 Video commentary: https://www.loom.com/share/ff4425e217db43d6b3ba4aa075374808 @getvictor TODO: Replacing `disable_failing_policies` with `disable_issues` will be done in a separate PR. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality
2024-06-17 17:15:42 +00:00
`created_at` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`updated_at` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`host_id`),
KEY `total_issues_count` (`total_issues_count`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm` (
`host_id` int unsigned NOT NULL,
`enrolled` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`server_url` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`installed_from_dep` tinyint(1) NOT NULL DEFAULT '0',
`mdm_id` int unsigned DEFAULT NULL,
`is_server` tinyint(1) DEFAULT NULL,
`fleet_enroll_ref` varchar(36) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`enrollment_status` enum('On (manual)','On (automatic)','Pending','Off','On (personal)') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS ((case when (`is_server` = 1) then NULL when ((`enrolled` = 1) and (`installed_from_dep` = 0) and (`is_personal_enrollment` = 1)) then _utf8mb4'On (personal)' when ((`enrolled` = 1) and (`installed_from_dep` = 0) and (`is_personal_enrollment` = 0)) then _utf8mb4'On (manual)' when ((`enrolled` = 1) and (`installed_from_dep` = 1) and (`is_personal_enrollment` = 0)) then _utf8mb4'On (automatic)' when ((`enrolled` = 0) and (`installed_from_dep` = 1)) then _utf8mb4'Pending' when ((`enrolled` = 0) and (`installed_from_dep` = 0)) then _utf8mb4'Off' else NULL end)) VIRTUAL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`is_personal_enrollment` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`host_id`),
KEY `host_mdm_mdm_id_idx` (`mdm_id`),
KEY `host_mdm_enrolled_installed_from_dep_is_personal_enrollment_idx` (`enrolled`,`installed_from_dep`,`is_personal_enrollment`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_actions` (
`host_id` int unsigned NOT NULL,
`lock_ref` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`wipe_ref` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`unlock_pin` varchar(6) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`unlock_ref` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`fleet_platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`host_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_android_profiles` (
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`operation_type` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`detail` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`profile_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`policy_request_uuid` varchar(36) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`device_request_uuid` varchar(36) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`request_fail_count` tinyint unsigned NOT NULL DEFAULT '0',
`included_in_policy_version` int DEFAULT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`host_uuid`,`profile_uuid`),
KEY `status` (`status`),
KEY `operation_type` (`operation_type`),
KEY `policy_request_uuid` (`policy_request_uuid`),
KEY `device_request_uuid` (`device_request_uuid`),
CONSTRAINT `host_mdm_android_profiles_ibfk_1` FOREIGN KEY (`status`) REFERENCES `mdm_delivery_status` (`status`) ON UPDATE CASCADE,
CONSTRAINT `host_mdm_android_profiles_ibfk_2` FOREIGN KEY (`operation_type`) REFERENCES `mdm_operation_types` (`operation_type`) ON UPDATE CASCADE,
CONSTRAINT `host_mdm_android_profiles_ibfk_3` FOREIGN KEY (`policy_request_uuid`) REFERENCES `android_policy_requests` (`request_uuid`) ON DELETE SET NULL,
CONSTRAINT `host_mdm_android_profiles_ibfk_4` FOREIGN KEY (`device_request_uuid`) REFERENCES `android_policy_requests` (`request_uuid`) ON DELETE SET NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_apple_awaiting_configuration` (
`host_uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`awaiting_configuration` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`host_uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_apple_bootstrap_packages` (
`host_uuid` varchar(127) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`command_uuid` varchar(127) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`host_uuid`),
KEY `command_uuid` (`command_uuid`),
CONSTRAINT `host_mdm_apple_bootstrap_packages_ibfk_1` FOREIGN KEY (`command_uuid`) REFERENCES `nano_commands` (`command_uuid`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-14 21:08:19 +00:00
CREATE TABLE `host_mdm_apple_declarations` (
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`operation_type` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`detail` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`token` binary(16) NOT NULL,
`declaration_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`declaration_identifier` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`declaration_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`secrets_updated_at` datetime(6) DEFAULT NULL,
Fixing issues with Apple DDM profile status (#29059) For #27979 This PR fixes Apple declarations issues: - P2 issue with hashing the declaration token - When declaration items are requested, mark any outstanding "remove" operations as pending. This prevents "remove" operations from being stuck in pending in some cases because they were actually already processed. - When updating verification status, don't update "remove" operations -- we don't update their status and we just delete them. This prevents the issue where a "remove" operation got the updated status and the "install" operation got stuck in verifying forever. - when adding a declaration that has a matching remove outstanding, mark the declaration verified. This prevents "install" operations from being stuck in pending/verifying. Why? Because there is nothing for the host to do if the same declaration was removed and then immediately added back. - migration to delete "remove" operations with non-nil and non-pending status. These are the only legal statuses for remove operations. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality
2025-05-15 18:05:25 +00:00
`resync` tinyint(1) NOT NULL DEFAULT '0',
`scope` enum('System','User') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'System',
2024-03-14 21:08:19 +00:00
PRIMARY KEY (`host_uuid`,`declaration_uuid`),
KEY `status` (`status`),
KEY `operation_type` (`operation_type`),
Fixing issues with Apple DDM profile status (#29059) For #27979 This PR fixes Apple declarations issues: - P2 issue with hashing the declaration token - When declaration items are requested, mark any outstanding "remove" operations as pending. This prevents "remove" operations from being stuck in pending in some cases because they were actually already processed. - When updating verification status, don't update "remove" operations -- we don't update their status and we just delete them. This prevents the issue where a "remove" operation got the updated status and the "install" operation got stuck in verifying forever. - when adding a declaration that has a matching remove outstanding, mark the declaration verified. This prevents "install" operations from being stuck in pending/verifying. Why? Because there is nothing for the host to do if the same declaration was removed and then immediately added back. - migration to delete "remove" operations with non-nil and non-pending status. These are the only legal statuses for remove operations. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality
2025-05-15 18:05:25 +00:00
KEY `idx_token` (`token`),
2024-03-14 21:08:19 +00:00
CONSTRAINT `host_mdm_apple_declarations_ibfk_1` FOREIGN KEY (`status`) REFERENCES `mdm_delivery_status` (`status`) ON UPDATE CASCADE,
CONSTRAINT `host_mdm_apple_declarations_ibfk_2` FOREIGN KEY (`operation_type`) REFERENCES `mdm_operation_types` (`operation_type`) ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2024-03-14 21:08:19 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_apple_profiles` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`profile_identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`host_uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`operation_type` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`detail` text COLLATE utf8mb4_unicode_ci,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`profile_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
various profile fixes (#11084) ### Related tickets https://github.com/fleetdm/fleet/issues/10775 https://github.com/fleetdm/fleet/issues/10678 https://github.com/fleetdm/fleet/issues/11024 https://github.com/fleetdm/fleet/issues/11026 ### What's happening - Implemented the hashing mechanism defined by @mna in #10678, however this mechanism is mainly relevant for batch profile updates via the CLI, we can't leverage it when a host switches teams. - Modified `BulkSetPendingMDMAppleHostProfiles` so when two profiles with the same identifier are sheduled both for removal and update, the function will now mark only the `install` as `pending` so it's picked by the cron, and will `DELETE` the `remove` entry from the database so it's not picked by the cron and never sent to the user. - `GetHostMDMProfiles` and consequently the profiles returned in `GET /api/_version_/fleet/hosts` return `host_mdm_apple_profiles.state = NULL` as "Enforcing (pending", the distinction between `status = 'pending'` and `status IS NULL` is only useful for the cron, for users both mean the same thing, and all our profile aggregations already behave this way. - Using the solution implemented by @gillespi314 in https://github.com/fleetdm/fleet/pull/10998 we're now deleting the host row from `host_disk_encryption_keys` if a host is moved from a team that enforces disk encryption to a team that doesn't. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-04-09 02:23:36 +00:00
`checksum` binary(16) NOT NULL,
`retries` tinyint unsigned NOT NULL DEFAULT '0',
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`secrets_updated_at` datetime(6) DEFAULT NULL,
`ignore_error` tinyint(1) NOT NULL DEFAULT '0',
Add timestamp to host_mdm_apple_profiles for variable update times (#29074) For #28970 Adds a timestamp to the host-apple profiles table so that we know when the profile should have been installed or reinstalled based on the last date the variables were updated. This overrides the usual date which is just based on profile upload time. It's needed because of how variable-based and specifically CA profiles are used to renew certificates - the checksum never changes but we need to verify that the profile was updated after the expected time. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Added/updated automated tests - [x] Manual QA for all new/changed functionality - [x] For unreleased bug fixes in a release candidate, confirmed that the fix is not expected to adversely impact load test results or alerted the release DRI if additional load testing is needed.
2025-05-13 21:22:27 +00:00
`variables_updated_at` datetime(6) DEFAULT NULL,
`scope` enum('System','User') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'System',
PRIMARY KEY (`host_uuid`,`profile_uuid`),
KEY `status` (`status`),
KEY `operation_type` (`operation_type`),
CONSTRAINT `host_mdm_apple_profiles_ibfk_1` FOREIGN KEY (`status`) REFERENCES `mdm_delivery_status` (`status`) ON UPDATE CASCADE,
CONSTRAINT `host_mdm_apple_profiles_ibfk_2` FOREIGN KEY (`operation_type`) REFERENCES `mdm_operation_types` (`operation_type`) ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Deleting an ABM iOS/iPadOS host keeps it in Fleet. (#21433) #21073 - Deleted iOS/iPadOS host will continue to report to Fleet as long as host is in Apple Business Manager (ABM). - Refetching an offline iOS/iPadOS host will not add new MDM commands to the queue if previous refetch has not completed yet. Video demo: https://www.loom.com/share/2f7ecb22e1924d4cbbbdd7dd297439ef?sid=dbfe1939-cb46-47ca-a7a8-84965ed68a7e I considered `nano_command_results` but could not think of an efficient way to use `nano_command_results` to determine if refetch was already done. The problem is that we're overloading `command_uuid` to include the `REFETCH` identifier. So we can't simply add an index on `(uuid, command_uuid, updated_at)`. We need to do a partial text match on `command_uuid` which then requires a full scan of the matching `updated_at` timestamps. It feels like a search like this could blow up for a large `nano_enrollment_queue` and `nano_command_results` table. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality
2024-08-21 13:51:04 +00:00
CREATE TABLE `host_mdm_commands` (
`host_id` int unsigned NOT NULL,
`command_type` varchar(31) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`host_id`,`command_type`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_idp_accounts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`account_uuid` varchar(36) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_mdm_idp_accounts` (`host_uuid`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_managed_certificates` (
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`type` enum('digicert','custom_scep_proxy','ndes','smallstep') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'ndes',
`ca_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'NDES',
`challenge_retrieved_at` timestamp(6) NULL DEFAULT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`not_valid_after` datetime(6) DEFAULT NULL,
Renewal of DigiCert certificates on macOS (#28449) Adds renewal of Digicert certificates: https://github.com/fleetdm/fleet/issues/26553 . Does not attempt to renew custom SCEP or NDES. Also we aren't actually calling the DigiCert renewal endpoint at this time because we don't believe we need to and we can't necessarily do that as we weren't previously storing the serial number however this change adds storage of the serial number. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it)
2025-04-24 12:35:15 +00:00
`serial` varchar(40) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`not_valid_before` datetime(6) DEFAULT NULL,
PRIMARY KEY (`host_uuid`,`profile_uuid`,`ca_name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_windows_profiles` (
`host_uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`operation_type` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`detail` text COLLATE utf8mb4_unicode_ci,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`profile_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`retries` tinyint unsigned NOT NULL DEFAULT '0',
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`checksum` binary(16) NOT NULL DEFAULT '0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0',
`secrets_updated_at` datetime(6) DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`host_uuid`,`profile_uuid`),
KEY `status` (`status`),
KEY `operation_type` (`operation_type`),
CONSTRAINT `host_mdm_windows_profiles_ibfk_1` FOREIGN KEY (`status`) REFERENCES `mdm_delivery_status` (`status`) ON UPDATE CASCADE,
CONSTRAINT `host_mdm_windows_profiles_ibfk_2` FOREIGN KEY (`operation_type`) REFERENCES `mdm_operation_types` (`operation_type`) ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_munki_info` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`deleted_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`host_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_munki_issues` (
`host_id` int unsigned NOT NULL,
`munki_issue_id` int unsigned NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`host_id`,`munki_issue_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_operating_system` (
`host_id` int unsigned NOT NULL,
`os_id` int unsigned NOT NULL,
PRIMARY KEY (`host_id`),
KEY `idx_host_operating_system_id` (`os_id`),
CONSTRAINT `host_operating_system_ibfk_1` FOREIGN KEY (`os_id`) REFERENCES `operating_systems` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_orbit_info` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`version` varchar(50) COLLATE utf8mb4_unicode_ci NOT NULL,
Added orbit_version, fleet_desktop_version, and scripts_enabled to host details. (#18123) #17361 #17148 In GET fleet/hosts/:id response, added the following fields: - orbit_version - `orbit_version == null` means this agent is not an orbit agent - fleet_desktop_version - `fleet_desktop_version == null` means this agent is not an orbit agent or it is an older version which is not collecting the desktop version - `fleet_desktop_version == ""` means this agent is an orbit agent but does not have fleet desktop - scripts_enabled - `scripts_enabled == null` means this agent is not an orbit agent or it is an older version which is not collecting scripts_enabled In orbit_info table, added the following fields: - desktop_version - scripts_enabled Updated docs for orbit_info PR: https://github.com/fleetdm/fleet/pull/18135 Updated API docs: https://github.com/fleetdm/fleet/pull/17814 MDM lock/unlock/wipe error messages are not part of this PR. They will be in a separate PR. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-04-09 21:33:44 +00:00
`desktop_version` varchar(50) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`scripts_enabled` tinyint(1) DEFAULT NULL,
PRIMARY KEY (`host_id`),
KEY `idx_host_orbit_info_version` (`version`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_scim_user` (
`host_id` int unsigned NOT NULL,
`scim_user_id` int unsigned NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`host_id`),
KEY `fk_host_scim_scim_user_id` (`scim_user_id`),
CONSTRAINT `fk_host_scim_scim_user_id` FOREIGN KEY (`scim_user_id`) REFERENCES `scim_users` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_script_results` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`execution_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`output` text COLLATE utf8mb4_unicode_ci NOT NULL,
`runtime` int unsigned NOT NULL DEFAULT '0',
`exit_code` int DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`script_id` int unsigned DEFAULT NULL,
`user_id` int unsigned DEFAULT NULL,
`sync_request` tinyint(1) NOT NULL DEFAULT '0',
`script_content_id` int unsigned DEFAULT NULL,
`host_deleted_at` timestamp NULL DEFAULT NULL,
`timeout` int DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
2024-10-25 22:11:56 +00:00
`setup_experience_script_id` int unsigned DEFAULT NULL,
`is_internal` tinyint(1) DEFAULT '0',
`canceled` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_script_results_execution_id` (`execution_id`),
KEY `idx_host_script_results_host_exit_created` (`host_id`,`exit_code`,`created_at`),
KEY `fk_host_script_results_script_id` (`script_id`),
KEY `fk_host_script_results_user_id` (`user_id`),
KEY `script_content_id` (`script_content_id`),
KEY `fk_script_result_policy_id` (`policy_id`),
2024-10-25 22:11:56 +00:00
KEY `fk_host_script_results_setup_experience_id` (`setup_experience_script_id`),
Optimized GetHostScriptExecutionResults MySQL query for for large numbers of script results. (#32595) Fixes #32295 The issue was identified/fixed using this performance test: https://gist.github.com/getvictor/b289b7b14981fb7bf77e57c80af117d1 With the fix: - 100 records: 2.6ms (similar) - 1,000 records: ~7ms (32x faster) - 5,000 records: ~10ms (530x faster) - 10,000 records: 13ms (1,430x faster) - 20,000 records: ~25ms (2,960x faster) - 40,000 records: 50ms (6,000x faster) # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] QA'd all new/changed functionality manually <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - New Features - Script details now surface the most relevant “latest” status per script, prioritizing upcoming executions when present. - Performance Improvements - Significantly faster loading of host script results and script details, especially at large scale. - Improved responsiveness when filtering/sorting script results. - Documentation - Added changelog entry describing the optimization to script results handling. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2025-09-04 20:48:18 +00:00
KEY `idx_host_script_canceled_created_at` (`host_id`,`script_id`,`canceled`,`created_at` DESC),
CONSTRAINT `fk_host_script_results_script_id` FOREIGN KEY (`script_id`) REFERENCES `scripts` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_host_script_results_setup_experience_id` FOREIGN KEY (`setup_experience_script_id`) REFERENCES `setup_experience_scripts` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_host_script_results_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `host_script_results_ibfk_1` FOREIGN KEY (`script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE,
CONSTRAINT `host_script_results_ibfk_2` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_seen_times` (
`host_id` int unsigned NOT NULL,
`seen_time` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`host_id`),
KEY `idx_host_seen_times_seen_time` (`seen_time`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_software` (
`host_id` int unsigned NOT NULL,
`software_id` bigint unsigned NOT NULL,
`last_opened_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`host_id`,`software_id`),
KEY `host_software_software_fk` (`software_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_software_installed_paths` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`software_id` bigint unsigned NOT NULL,
`installed_path` text COLLATE utf8mb4_unicode_ci NOT NULL,
Add `team_identifier` to macOS software (#23766) Changes to add `team_identifier` signing information to macOS applications on the `/api/latest/fleet/hosts/:id/software` API endpoint. Docs: https://github.com/fleetdm/fleet/pull/23743 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added/updated tests - [X] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [X] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ X Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [X] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [X] Orbit runs on macOS, Linux and Windows. Check if the orbit feature/bugfix should only apply to one platform (`runtime.GOOS`). - [X] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [X] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)). --------- Co-authored-by: Tim Lee <timlee@fleetdm.com> Co-authored-by: Ian Littman <iansltx@gmail.com>
2024-11-15 17:17:04 +00:00
`team_identifier` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`executable_sha256` char(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `host_id_software_id_idx` (`host_id`,`software_id`)
Add `team_identifier` to macOS software (#23766) Changes to add `team_identifier` signing information to macOS applications on the `/api/latest/fleet/hosts/:id/software` API endpoint. Docs: https://github.com/fleetdm/fleet/pull/23743 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added/updated tests - [X] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [X] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ X Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [X] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [X] Orbit runs on macOS, Linux and Windows. Check if the orbit feature/bugfix should only apply to one platform (`runtime.GOOS`). - [X] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [X] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)). --------- Co-authored-by: Tim Lee <timlee@fleetdm.com> Co-authored-by: Ian Littman <iansltx@gmail.com>
2024-11-15 17:17:04 +00:00
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_software_installs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`execution_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`host_id` int unsigned NOT NULL,
`software_installer_id` int unsigned DEFAULT NULL,
`pre_install_query_output` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`install_script_output` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`install_script_exit_code` int DEFAULT NULL,
`post_install_script_output` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`post_install_script_exit_code` int DEFAULT NULL,
`user_id` int unsigned DEFAULT NULL,
2024-09-08 17:26:26 +00:00
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`self_service` tinyint(1) NOT NULL DEFAULT '0',
2024-09-08 17:26:26 +00:00
`host_deleted_at` timestamp(6) NULL DEFAULT NULL,
`removed` tinyint NOT NULL DEFAULT '0',
2024-09-08 17:26:26 +00:00
`uninstall_script_output` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`uninstall_script_exit_code` int DEFAULT NULL,
`uninstall` tinyint unsigned NOT NULL DEFAULT '0',
`status` enum('pending_install','failed_install','installed','pending_uninstall','failed_uninstall','canceled_install','canceled_uninstall') COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS ((case when (`removed` = 1) then NULL when ((`canceled` = 1) and (`uninstall` = 0)) then _utf8mb4'canceled_install' when ((`canceled` = 1) and (`uninstall` = 1)) then _utf8mb4'canceled_uninstall' when ((`post_install_script_exit_code` is not null) and (`post_install_script_exit_code` = 0)) then _utf8mb4'installed' when ((`post_install_script_exit_code` is not null) and (`post_install_script_exit_code` <> 0)) then _utf8mb4'failed_install' when ((`install_script_exit_code` is not null) and (`install_script_exit_code` = 0)) then _utf8mb4'installed' when ((`install_script_exit_code` is not null) and (`install_script_exit_code` <> 0)) then _utf8mb4'failed_install' when ((`pre_install_query_output` is not null) and (`pre_install_query_output` = _utf8mb4'')) then _utf8mb4'failed_install' when ((`host_id` is not null) and (`uninstall` = 0)) then _utf8mb4'pending_install' when ((`uninstall_script_exit_code` is not null) and (`uninstall_script_exit_code` <> 0)) then _utf8mb4'failed_uninstall' when ((`uninstall_script_exit_code` is not null) and (`uninstall_script_exit_code` = 0)) then NULL when ((`host_id` is not null) and (`uninstall` = 1)) then _utf8mb4'pending_uninstall' else NULL end)) STORED,
`policy_id` int unsigned DEFAULT NULL,
`installer_filename` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '[deleted installer]',
`version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'unknown',
`software_title_id` int unsigned DEFAULT NULL,
`software_title_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '[deleted title]',
`execution_status` enum('pending_install','failed_install','installed','pending_uninstall','failed_uninstall','canceled_install','canceled_uninstall') COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS ((case when ((`canceled` = 1) and (`uninstall` = 0)) then _utf8mb4'canceled_install' when ((`canceled` = 1) and (`uninstall` = 1)) then _utf8mb4'canceled_uninstall' when ((`post_install_script_exit_code` is not null) and (`post_install_script_exit_code` = 0)) then _utf8mb4'installed' when ((`post_install_script_exit_code` is not null) and (`post_install_script_exit_code` <> 0)) then _utf8mb4'failed_install' when ((`install_script_exit_code` is not null) and (`install_script_exit_code` = 0)) then _utf8mb4'installed' when ((`install_script_exit_code` is not null) and (`install_script_exit_code` <> 0)) then _utf8mb4'failed_install' when ((`pre_install_query_output` is not null) and (`pre_install_query_output` = _utf8mb4'')) then _utf8mb4'failed_install' when ((`host_id` is not null) and (`uninstall` = 0)) then _utf8mb4'pending_install' when ((`uninstall_script_exit_code` is not null) and (`uninstall_script_exit_code` <> 0)) then _utf8mb4'failed_uninstall' when ((`uninstall_script_exit_code` is not null) and (`uninstall_script_exit_code` = 0)) then NULL when ((`host_id` is not null) and (`uninstall` = 1)) then _utf8mb4'pending_uninstall' else NULL end)) VIRTUAL,
`canceled` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_software_installs_execution_id` (`execution_id`),
KEY `fk_host_software_installs_user_id` (`user_id`),
KEY `idx_host_software_installs_host_installer` (`host_id`,`software_installer_id`),
KEY `fk_software_install_policy_id` (`policy_id`),
KEY `fk_host_software_installs_installer_id` (`software_installer_id`),
KEY `fk_host_software_installs_software_title_id` (`software_title_id`),
CONSTRAINT `fk_host_software_installs_installer_id` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_host_software_installs_software_title_id` FOREIGN KEY (`software_title_id`) REFERENCES `software_titles` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_host_software_installs_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `host_software_installs_ibfk_1` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_updates` (
`host_id` int unsigned NOT NULL,
`software_updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`host_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_users` (
`host_id` int unsigned NOT NULL,
`uid` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`username` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`groupname` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`removed_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`user_type` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`shell` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT '',
PRIMARY KEY (`host_id`,`uid`,`username`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_vpp_software_installs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`adam_id` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`user_id` int unsigned DEFAULT NULL,
`self_service` tinyint(1) NOT NULL DEFAULT '0',
`associated_event_id` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`platform` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`removed` tinyint NOT NULL DEFAULT '0',
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
`vpp_token_id` int unsigned DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
`canceled` tinyint(1) NOT NULL DEFAULT '0',
`verification_command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`verification_at` datetime(6) DEFAULT NULL,
`verification_failed_at` datetime(6) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_vpp_software_installs_command_uuid` (`command_uuid`),
KEY `user_id` (`user_id`),
KEY `adam_id` (`adam_id`,`platform`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
KEY `fk_host_vpp_software_installs_vpp_token_id` (`vpp_token_id`),
KEY `fk_host_vpp_software_installs_policy_id` (`policy_id`),
KEY `idx_host_vpp_software_installs_verification` ((((`verification_at` is null) and (`verification_failed_at` is null)))),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
CONSTRAINT `fk_host_vpp_software_installs_vpp_token_id` FOREIGN KEY (`vpp_token_id`) REFERENCES `vpp_tokens` (`id`) ON DELETE SET NULL,
CONSTRAINT `host_vpp_software_installs_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `host_vpp_software_installs_ibfk_3` FOREIGN KEY (`adam_id`, `platform`) REFERENCES `vpp_apps` (`adam_id`, `platform`) ON DELETE CASCADE,
CONSTRAINT `host_vpp_software_installs_ibfk_4` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `hosts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`osquery_host_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`detail_updated_at` timestamp NULL DEFAULT NULL,
`node_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL,
`hostname` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`osquery_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`os_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`build` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`platform_like` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`code_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`uptime` bigint NOT NULL DEFAULT '0',
`memory` bigint NOT NULL DEFAULT '0',
`cpu_type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`cpu_subtype` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`cpu_brand` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`cpu_physical_cores` int NOT NULL DEFAULT '0',
`cpu_logical_cores` int NOT NULL DEFAULT '0',
`hardware_vendor` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`hardware_model` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`hardware_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`hardware_serial` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`computer_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`primary_ip_id` int unsigned DEFAULT NULL,
`distributed_interval` int DEFAULT '0',
`logger_tls_period` int DEFAULT '0',
`config_tls_refresh` int DEFAULT '0',
`primary_ip` varchar(45) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`primary_mac` varchar(17) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`label_updated_at` timestamp NOT NULL DEFAULT '2000-01-01 00:00:00',
`last_enrolled_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`refetch_requested` tinyint(1) NOT NULL DEFAULT '0',
`team_id` int unsigned DEFAULT NULL,
`policy_updated_at` timestamp NOT NULL DEFAULT '2000-01-01 00:00:00',
`public_ip` varchar(45) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`orbit_node_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL,
`refetch_critical_queries_until` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_osquery_host_id` (`osquery_host_id`),
UNIQUE KEY `idx_host_unique_nodekey` (`node_key`),
UNIQUE KEY `idx_host_unique_orbitnodekey` (`orbit_node_key`),
KEY `fk_hosts_team_id` (`team_id`),
KEY `hosts_platform_idx` (`platform`),
KEY `idx_hosts_hardware_serial` (`hardware_serial`),
KEY `idx_hosts_uuid` (`uuid`),
CONSTRAINT `hosts_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `invite_teams` (
`invite_id` int unsigned NOT NULL,
`team_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`role` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`invite_id`,`team_id`),
KEY `fk_team_id` (`team_id`),
CONSTRAINT `invite_teams_ibfk_1` FOREIGN KEY (`invite_id`) REFERENCES `invites` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `invite_teams_ibfk_2` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `invites` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`invited_by` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`position` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`token` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sso_enabled` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`global_role` varchar(64) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`mfa_enabled` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_invite_unique_email` (`email`),
UNIQUE KEY `idx_invite_unique_key` (`token`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `jobs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`args` json DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`state` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`retries` int NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`error` text COLLATE utf8mb4_unicode_ci,
`not_before` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
2024-08-06 18:49:01 +00:00
PRIMARY KEY (`id`),
KEY `idx_jobs_state_not_before_updated_at` (`state`,`not_before`,`updated_at`),
KEY `idx_jobs_name_state` (`name`,`state`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `jobs` VALUES (1,'2024-03-20 00:00:00','2024-03-20 00:00:00','macos_setup_assistant','{\"task\": \"update_all_profiles\"}','queued',0,'','2024-03-20 00:00:00');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `kernel_host_counts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`software_title_id` int unsigned DEFAULT NULL,
`software_id` int unsigned DEFAULT NULL,
`os_version_id` int unsigned DEFAULT NULL,
`hosts_count` int unsigned NOT NULL,
`team_id` int unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_kernels_unique_mapping` (`os_version_id`,`team_id`,`software_id`),
KEY `software_title_id` (`software_title_id`),
Fixed MySQL DB performance regressions (#33184) Resolves #33147 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ## Database migrations - [x] Checked table schema to confirm autoupdate <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - Bug Fixes - Resolved MySQL performance regressions from 4.73.0/4.73.1 affecting OS versions and software titles views, improving load times and reducing timeouts. - Refactor - Optimized OS vulnerabilities fetching by batching multiple OS versions in a single request. - Added a supporting database index to speed kernel-related vulnerability queries. - Tests - Added comprehensive tests for multi-OS vulnerability retrieval, CVSS enrichment, team-scoped data, and service endpoint behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-09-19 20:35:05 +00:00
KEY `idx_kernel_host_counts_os_version_software` (`os_version_id`,`software_id`,`hosts_count`),
CONSTRAINT `kernel_host_counts_ibfk_1` FOREIGN KEY (`software_title_id`) REFERENCES `software_titles` (`id`) ON DELETE CASCADE
Fixed MySQL DB performance regressions (#33184) Resolves #33147 # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. ## Testing - [x] Added/updated automated tests - [x] Where appropriate, [automated tests simulate multiple hosts and test for host isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing) (updates to one hosts's records do not affect another) - [x] QA'd all new/changed functionality manually ## Database migrations - [x] Checked table schema to confirm autoupdate <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - Bug Fixes - Resolved MySQL performance regressions from 4.73.0/4.73.1 affecting OS versions and software titles views, improving load times and reducing timeouts. - Refactor - Optimized OS vulnerabilities fetching by batching multiple OS versions in a single request. - Added a supporting database index to speed kernel-related vulnerability queries. - Tests - Added comprehensive tests for multi-OS vulnerability retrieval, CVSS enrichment, team-scoped data, and service endpoint behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-09-19 20:35:05 +00:00
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `label_membership` (
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`label_id` int unsigned NOT NULL,
`host_id` int unsigned NOT NULL,
PRIMARY KEY (`host_id`,`label_id`),
KEY `idx_lm_label_id` (`label_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`query` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`label_type` int unsigned NOT NULL DEFAULT '1',
`label_membership_type` int unsigned NOT NULL DEFAULT '0',
Add author ID to labels (#27055) For #27035 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality ## Details This PR adds an `author_id` column to the `labels` table, and adds the associated properties to the `Label` and `LabelSpec` types. When a new label is created via the UI or API, an author ID is set on the label if one can be inferred from the context. Otherwise, the author ID is set to `null`. ## Authz and Automated testing Additional backend authorization logic is introduced in a follow-on PR, https://github.com/fleetdm/fleet/pull/27089, because rconciling all of the test updates between this PR and https://github.com/fleetdm/fleet/pull/27038 was getting complicated. ## Manual Testing * Tested in the UI by creating a new label on the Hosts page * Tested via Gitops by merging this branch with https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops` with a global config with `labels:` in it.
2025-03-20 21:05:16 +00:00
`author_id` int unsigned DEFAULT NULL,
Add support for host vitals labels (#30278) # Details This PR adds support for a new label membership type, `host_vitals`. Membership for these labels is based on a database query created from user-supplied criteria. In this first iteration, the allowed criteria are very simple: a label can specify either an IdP group or IdP department, and hosts with linked users with a matching group or department. Groundwork is laid here for more complex host vitals queries, including `and` and `or` logic, different data types and different kinds of vitals (rather than just the "foreign" vitals of which IdP is an example). Note that this PR does _not_ include the cron job that will trigger membership updating, and it doesn't include ; for sake of simplicity in review that will be done in a follow-on PR. ## Basic flow ### Creating a host vitals label 1. A new label is created via the API / GitOps with membership type `host_vitals` and a `criteria` property that's a JSON blob. Currently the JSON can only contain `vital` and `value` keys (and must contain those keys) 2. The server validates that the specified `vital` exists in our [set of known host vitals](https://github.com/fleetdm/fleet/pull/30278/files#diff-b6d4c48f2624b82c2567b2b88db1de51c6b152eeb261d40acfd5b63a890839b7R418-R436). 3. The server validates that the [criteria can be parsed into a query](https://github.com/fleetdm/fleet/pull/30278/files?diff=unified&w=1#diff-4ac4cfba8bed490e8ef125a0556f5417156f805017bfe93c6e2c61aa94ba8a8cR81-R86). This also happens during GitOps dry run. 4. The label is saved (criteria is saved as JSON in the db) ### Updating membership for a host vitals label 1. The label's criteria is used to generate a query to run on the _Fleet_ db. 1. For each vital criteria, check the vital type. Currently only foreign vitals are supported. 2. For foreign vitals, add its group to a set we keep track of. 3. Add a `WHERE` clause section for the vital and value, e.g. `end_user_idp_groups = ?` 4. Once we have all the `WHERE` clauses, create the query as `SELECT %s FROM %s` + any joins contributed by foreign vitals groups + `WHERE ` + all the `WHERE` clauses we just calculated. The `%s` provide some flexibility if we want to use these queries in other contexts. 2. Delete all existing label members 3. Do an `INSERT...SELECT` using the query we calculated from the label criteria. The query will be `SELECT <label id> as label_id, hosts.id FROM hosts JOIN ...` ## Future work ### Domestic vitals These can be anything that we already store in the `hosts` table. Domestic vitals won't add any `JOIN`s to the calculated label query, and will simply be e.g. `hosts.hostname = ?` ### Custom vitals We currently support an `additional_queries` config that will cause other queries to run on hosts. The data returned from these queries is stored in a `hosts_additional` table as a JSON blob. We can use MySQL JSON functions to match values in this data, e.g. `JSON_EXTRACT(host_additional, `$.some_custom_vital`) = ?` # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. > I'll add the changelog item when I add the cron job PR - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] If database migrations are included, checked table schema to confirm autoupdate - For new Fleet configuration settings - [X] Verified that the setting can be managed via GitOps, or confirmed that the setting is explicitly being excluded from GitOps. If managing via Gitops: - [X] Verified that the setting is exported via `fleetctl generate-gitops` - [X] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [X] Added/updated automated tests - [X] Manual QA for all new/changed functionality
2025-06-30 14:58:58 +00:00
`criteria` json DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_label_unique_name` (`name`),
Add author ID to labels (#27055) For #27035 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality ## Details This PR adds an `author_id` column to the `labels` table, and adds the associated properties to the `Label` and `LabelSpec` types. When a new label is created via the UI or API, an author ID is set on the label if one can be inferred from the context. Otherwise, the author ID is set to `null`. ## Authz and Automated testing Additional backend authorization logic is introduced in a follow-on PR, https://github.com/fleetdm/fleet/pull/27089, because rconciling all of the test updates between this PR and https://github.com/fleetdm/fleet/pull/27038 was getting complicated. ## Manual Testing * Tested in the UI by creating a new label on the Hosts page * Tested via Gitops by merging this branch with https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops` with a global config with `labels:` in it.
2025-03-20 21:05:16 +00:00
KEY `author_id` (`author_id`),
FULLTEXT KEY `labels_search` (`name`),
CONSTRAINT `labels_ibfk_1` FOREIGN KEY (`author_id`) REFERENCES `users` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `labels` VALUES (1,'2024-04-03 00:00:00','2025-10-09 00:00:00','macOS 14+ (Sonoma+)','macOS hosts with version 14 and above','select 1 from os_version where platform = \'darwin\' and major >= 14;','',1,0,NULL,NULL),(2,'2024-06-28 00:00:00','2025-10-09 00:00:00','iOS','All iOS hosts','','',1,1,NULL,NULL),(3,'2024-06-28 00:00:00','2025-10-09 00:00:00','iPadOS','All iPadOS hosts','','',1,1,NULL,NULL),(4,'2024-09-27 00:00:00','2025-10-09 00:00:00','Fedora Linux','All Fedora hosts','select 1 from os_version where name = \'Fedora Linux\';','',1,0,NULL,NULL),(5,'2025-02-25 00:00:00','2025-10-09 00:00:00','Android','All Android hosts','','',1,1,NULL,NULL);
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Enforce FileVault at login when manually enrolled (#31170) Done by not allowing any deferrals as before one deferral was allowed fixes: #29250 _There is no doc change as we nowhere state that we allow one deferral, let me know if we want to write somewhere that this is now the standard behaviour._ _I also investigated trying to force it directly when the profile arrived but without any luck, so still need a logout/login to get filevault enabled, but it's no longer possible to cancel/defer it._ **Do verify when testing in automatic enrollment that FileVault is still enforced as I can't test Automatic enrollment yet.** # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [x] If database migrations are included, checked table schema to confirm autoupdate **not relevant as it does not update any schema just modifies existing entries.** - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Added/updated automated tests - [x] Manual QA for all new/changed functionality
2025-08-01 13:15:11 +00:00
CREATE TABLE `legacy_host_filevault_profiles` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_uuid` varchar(36) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
`operation_type` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
`profile_uuid` varchar(37) COLLATE utf8mb4_unicode_ci NOT NULL,
`detail` text COLLATE utf8mb4_unicode_ci,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`scope` enum('System','User') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'System',
`created_at` timestamp(6) NOT NULL,
`updated_at` timestamp(6) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `legacy_host_mdm_enroll_refs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_ref` varchar(36) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
KEY `idx_legacy_enroll_refs_host_uuid` (`host_uuid`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `legacy_host_mdm_idp_accounts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`account_uuid` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`host_id` int unsigned DEFAULT NULL,
`email_id` int unsigned DEFAULT NULL,
`email_created_at` datetime DEFAULT NULL,
`email_updated_at` datetime DEFAULT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `locks` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`owner` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`expires_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_name` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_android_configuration_profiles` (
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`team_id` int unsigned NOT NULL DEFAULT '0',
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_json` json NOT NULL,
`auto_increment` bigint NOT NULL AUTO_INCREMENT,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`uploaded_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`profile_uuid`),
UNIQUE KEY `auto_increment` (`auto_increment`),
UNIQUE KEY `idx_mdm_android_configuration_profiles_team_id_name` (`team_id`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_bootstrap_packages` (
`team_id` int unsigned NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`sha256` binary(32) NOT NULL,
`bytes` longblob,
`token` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`team_id`),
UNIQUE KEY `idx_token` (`token`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_configuration_profiles` (
`profile_id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`mobileconfig` mediumblob NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`uploaded_at` timestamp(6) NULL DEFAULT NULL,
various profile fixes (#11084) ### Related tickets https://github.com/fleetdm/fleet/issues/10775 https://github.com/fleetdm/fleet/issues/10678 https://github.com/fleetdm/fleet/issues/11024 https://github.com/fleetdm/fleet/issues/11026 ### What's happening - Implemented the hashing mechanism defined by @mna in #10678, however this mechanism is mainly relevant for batch profile updates via the CLI, we can't leverage it when a host switches teams. - Modified `BulkSetPendingMDMAppleHostProfiles` so when two profiles with the same identifier are sheduled both for removal and update, the function will now mark only the `install` as `pending` so it's picked by the cron, and will `DELETE` the `remove` entry from the database so it's not picked by the cron and never sent to the user. - `GetHostMDMProfiles` and consequently the profiles returned in `GET /api/_version_/fleet/hosts` return `host_mdm_apple_profiles.state = NULL` as "Enforcing (pending", the distinction between `status = 'pending'` and `status IS NULL` is only useful for the cron, for users both mean the same thing, and all our profile aggregations already behave this way. - Using the solution implemented by @gillespi314 in https://github.com/fleetdm/fleet/pull/10998 we're now deleting the host row from `host_disk_encryption_keys` if a host is moved from a team that enforces disk encryption to a team that doesn't. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-04-09 02:23:36 +00:00
`checksum` binary(16) NOT NULL,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`secrets_updated_at` datetime(6) DEFAULT NULL,
`scope` enum('System','User') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'System',
PRIMARY KEY (`profile_uuid`),
UNIQUE KEY `idx_mdm_apple_config_prof_team_identifier` (`team_id`,`identifier`),
UNIQUE KEY `idx_mdm_apple_config_prof_team_name` (`team_id`,`name`),
UNIQUE KEY `idx_mdm_apple_config_prof_id` (`profile_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-14 21:08:19 +00:00
CREATE TABLE `mdm_apple_declaration_activation_references` (
`declaration_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`reference` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
2024-03-14 21:08:19 +00:00
PRIMARY KEY (`declaration_uuid`,`reference`),
KEY `reference` (`reference`),
CONSTRAINT `mdm_apple_declaration_activation_references_ibfk_1` FOREIGN KEY (`declaration_uuid`) REFERENCES `mdm_apple_declarations` (`declaration_uuid`) ON UPDATE CASCADE,
CONSTRAINT `mdm_apple_declaration_activation_references_ibfk_2` FOREIGN KEY (`reference`) REFERENCES `mdm_apple_declarations` (`declaration_uuid`) ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-14 21:08:19 +00:00
CREATE TABLE `mdm_apple_declarations` (
`declaration_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`team_id` int unsigned NOT NULL DEFAULT '0',
`identifier` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_json` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`uploaded_at` timestamp(6) NULL DEFAULT NULL,
`auto_increment` bigint NOT NULL AUTO_INCREMENT,
`secrets_updated_at` datetime(6) DEFAULT NULL,
`token` binary(16) GENERATED ALWAYS AS (unhex(md5(concat(`raw_json`,ifnull(`secrets_updated_at`,_utf8mb4''))))) STORED,
`scope` enum('System','User') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'System',
2024-03-14 21:08:19 +00:00
PRIMARY KEY (`declaration_uuid`),
UNIQUE KEY `idx_mdm_apple_declaration_team_identifier` (`team_id`,`identifier`),
UNIQUE KEY `idx_mdm_apple_declaration_team_name` (`team_id`,`name`),
UNIQUE KEY `auto_increment` (`auto_increment`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2024-03-14 21:08:19 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_declarative_requests` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`enrollment_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`message_type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_json` text COLLATE utf8mb4_unicode_ci,
PRIMARY KEY (`id`),
KEY `mdm_apple_declarative_requests_enrollment_id` (`enrollment_id`),
CONSTRAINT `mdm_apple_declarative_requests_enrollment_id` FOREIGN KEY (`enrollment_id`) REFERENCES `nano_enrollments` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_default_setup_assistants` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`profile_uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
`abm_token_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
UNIQUE KEY `idx_mdm_default_setup_assistant_global_or_team_id_abm_token_id` (`global_or_team_id`,`abm_token_id`),
KEY `fk_mdm_default_setup_assistant_team_id` (`team_id`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
KEY `fk_mdm_default_setup_assistant_abm_token_id` (`abm_token_id`),
CONSTRAINT `fk_mdm_default_setup_assistant_abm_token_id` FOREIGN KEY (`abm_token_id`) REFERENCES `abm_tokens` (`id`) ON DELETE CASCADE,
CONSTRAINT `mdm_apple_default_setup_assistants_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `mdm_apple_enrollment_profiles` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`type` varchar(10) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'automatic',
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`dep_profile` json DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_type` (`type`),
UNIQUE KEY `idx_token` (`token`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `mdm_apple_installers` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`size` bigint NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`manifest` text COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`installer` longblob,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`url_token` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
CREATE TABLE `mdm_apple_setup_assistant_profiles` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`setup_assistant_id` int unsigned NOT NULL,
`abm_token_id` int unsigned NOT NULL,
`profile_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_apple_setup_assistant_profiles_asst_id_tok_id` (`setup_assistant_id`,`abm_token_id`),
KEY `fk_mdm_apple_setup_assistant_profiles_abm_token_id` (`abm_token_id`),
CONSTRAINT `fk_mdm_apple_setup_assistant_profiles_abm_token_id` FOREIGN KEY (`abm_token_id`) REFERENCES `abm_tokens` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_mdm_apple_setup_assistant_profiles_setup_assistant_id` FOREIGN KEY (`setup_assistant_id`) REFERENCES `mdm_apple_setup_assistants` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_setup_assistants` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`name` text COLLATE utf8mb4_unicode_ci NOT NULL,
`profile` json NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_setup_assistant_global_or_team_id` (`global_or_team_id`),
KEY `fk_mdm_setup_assistant_team_id` (`team_id`),
CONSTRAINT `mdm_apple_setup_assistants_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_config_assets` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`value` longblob NOT NULL,
`deleted_at` timestamp NULL DEFAULT NULL,
`deletion_uuid` varchar(127) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`md5_checksum` binary(16) NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_config_assets_name_deletion_uuid` (`name`,`deletion_uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_configuration_profile_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`apple_profile_uuid` varchar(37) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`windows_profile_uuid` varchar(37) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`label_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`label_id` int unsigned DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`require_all` tinyint(1) NOT NULL DEFAULT '0',
`android_profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_configuration_profile_labels_apple_label_name` (`apple_profile_uuid`,`label_name`),
UNIQUE KEY `idx_mdm_configuration_profile_labels_windows_label_name` (`windows_profile_uuid`,`label_name`),
UNIQUE KEY `idx_mdm_configuration_profile_labels_android_label_name` (`android_profile_uuid`,`label_name`),
KEY `label_id` (`label_id`),
CONSTRAINT `mdm_configuration_profile_labels_ibfk_1` FOREIGN KEY (`apple_profile_uuid`) REFERENCES `mdm_apple_configuration_profiles` (`profile_uuid`) ON DELETE CASCADE,
CONSTRAINT `mdm_configuration_profile_labels_ibfk_2` FOREIGN KEY (`windows_profile_uuid`) REFERENCES `mdm_windows_configuration_profiles` (`profile_uuid`) ON DELETE CASCADE,
CONSTRAINT `mdm_configuration_profile_labels_ibfk_3` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE SET NULL,
CONSTRAINT `mdm_configuration_profile_labels_ibfk_4` FOREIGN KEY (`android_profile_uuid`) REFERENCES `mdm_android_configuration_profiles` (`profile_uuid`) ON DELETE CASCADE,
CONSTRAINT `ck_mdm_configuration_profile_labels_profile_uuid` CHECK ((((if((`apple_profile_uuid` is null),0,1) + if((`windows_profile_uuid` is null),0,1)) + if((`android_profile_uuid` is null),0,1)) = 1))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_configuration_profile_variables` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`apple_profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`windows_profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`fleet_variable_id` int unsigned NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_configuration_profile_variables_apple_variable` (`apple_profile_uuid`,`fleet_variable_id`),
UNIQUE KEY `idx_mdm_configuration_profile_variables_windows_label_name` (`windows_profile_uuid`,`fleet_variable_id`),
KEY `mdm_configuration_profile_variables_fleet_variable_id` (`fleet_variable_id`),
CONSTRAINT `fk_mdm_configuration_profile_variables_apple_profile_uuid` FOREIGN KEY (`apple_profile_uuid`) REFERENCES `mdm_apple_configuration_profiles` (`profile_uuid`) ON DELETE CASCADE,
CONSTRAINT `fk_mdm_configuration_profile_variables_windows_profile_uuid` FOREIGN KEY (`windows_profile_uuid`) REFERENCES `mdm_windows_configuration_profiles` (`profile_uuid`) ON DELETE CASCADE,
CONSTRAINT `mdm_configuration_profile_variables_fleet_variable_id` FOREIGN KEY (`fleet_variable_id`) REFERENCES `fleet_variables` (`id`) ON DELETE CASCADE,
CONSTRAINT `ck_mdm_configuration_profile_variables_apple_or_windows` CHECK (((`apple_profile_uuid` is null) <> (`windows_profile_uuid` is null)))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-14 21:08:19 +00:00
CREATE TABLE `mdm_declaration_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`apple_declaration_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`label_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`label_id` int unsigned DEFAULT NULL,
2024-03-14 21:08:19 +00:00
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`uploaded_at` timestamp NULL DEFAULT NULL,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`require_all` tinyint(1) NOT NULL DEFAULT '0',
2024-03-14 21:08:19 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_declaration_labels_label_name` (`apple_declaration_uuid`,`label_name`),
2024-03-14 21:08:19 +00:00
KEY `label_id` (`label_id`),
CONSTRAINT `mdm_declaration_labels_ibfk_1` FOREIGN KEY (`apple_declaration_uuid`) REFERENCES `mdm_apple_declarations` (`declaration_uuid`) ON DELETE CASCADE,
2024-03-14 21:08:19 +00:00
CONSTRAINT `mdm_declaration_labels_ibfk_3` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2024-03-14 21:08:19 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_delivery_status` (
`status` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`status`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `mdm_delivery_status` VALUES ('failed'),('pending'),('verified'),('verifying');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_idp_accounts` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`username` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`fullname` varchar(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`email` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`uuid`),
UNIQUE KEY `unique_idp_email` (`email`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_operation_types` (
`operation_type` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`operation_type`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `mdm_operation_types` VALUES ('install'),('remove');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_windows_configuration_profiles` (
`team_id` int unsigned NOT NULL DEFAULT '0',
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`syncml` mediumblob NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`uploaded_at` timestamp(6) NULL DEFAULT NULL,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`auto_increment` bigint NOT NULL AUTO_INCREMENT,
`checksum` binary(16) GENERATED ALWAYS AS (unhex(md5(`syncml`))) STORED,
`secrets_updated_at` datetime(6) DEFAULT NULL,
PRIMARY KEY (`profile_uuid`),
UNIQUE KEY `idx_mdm_windows_configuration_profiles_team_id_name` (`team_id`,`name`),
UNIQUE KEY `auto_increment` (`auto_increment`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_windows_enrollments` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`mdm_device_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`mdm_hardware_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_state` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_user_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_proto_version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_client_version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`not_in_oobe` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_type` (`mdm_hardware_id`),
KEY `idx_mdm_windows_enrollments_mdm_device_id` (`mdm_device_id`),
KEY `idx_mdm_windows_enrollments_host_uuid` (`host_uuid`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Microsoft Compliance Partner backend changes (#29540) For #27042. Ready for review, just missing integration tests that I will be writing today. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] If database migrations are included, checked table schema to confirm autoupdate - For new Fleet configuration settings - [X] Verified that the setting can be managed via GitOps, or confirmed that the setting is explicitly being excluded from GitOps. If managing via Gitops: - [X] Verified that the setting is exported via `fleetctl generate-gitops` - [X] Added the setting to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [X] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Added/updated automated tests - [X] Manual QA for all new/changed functionality --------- Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com> Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-06-11 17:22:46 +00:00
CREATE TABLE `microsoft_compliance_partner_host_statuses` (
`host_id` int unsigned NOT NULL,
`device_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`user_principal_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`managed` tinyint(1) DEFAULT NULL,
`compliant` tinyint(1) DEFAULT NULL,
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`host_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `microsoft_compliance_partner_integrations` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`tenant_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`proxy_server_secret` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`setup_done` tinyint(1) NOT NULL DEFAULT '0',
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_microsoft_compliance_partner_tenant_id` (`tenant_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `migration_status_tables` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`version_id` bigint NOT NULL,
`is_applied` tinyint(1) NOT NULL,
`tstamp` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=431 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `migration_status_tables` VALUES (1,0,1,'2020-01-01 01:01:01'),(2,20161118193812,1,'2020-01-01 01:01:01'),(3,20161118211713,1,'2020-01-01 01:01:01'),(4,20161118212436,1,'2020-01-01 01:01:01'),(5,20161118212515,1,'2020-01-01 01:01:01'),(6,20161118212528,1,'2020-01-01 01:01:01'),(7,20161118212538,1,'2020-01-01 01:01:01'),(8,20161118212549,1,'2020-01-01 01:01:01'),(9,20161118212557,1,'2020-01-01 01:01:01'),(10,20161118212604,1,'2020-01-01 01:01:01'),(11,20161118212613,1,'2020-01-01 01:01:01'),(12,20161118212621,1,'2020-01-01 01:01:01'),(13,20161118212630,1,'2020-01-01 01:01:01'),(14,20161118212641,1,'2020-01-01 01:01:01'),(15,20161118212649,1,'2020-01-01 01:01:01'),(16,20161118212656,1,'2020-01-01 01:01:01'),(17,20161118212758,1,'2020-01-01 01:01:01'),(18,20161128234849,1,'2020-01-01 01:01:01'),(19,20161230162221,1,'2020-01-01 01:01:01'),(20,20170104113816,1,'2020-01-01 01:01:01'),(21,20170105151732,1,'2020-01-01 01:01:01'),(22,20170108191242,1,'2020-01-01 01:01:01'),(23,20170109094020,1,'2020-01-01 01:01:01'),(24,20170109130438,1,'2020-01-01 01:01:01'),(25,20170110202752,1,'2020-01-01 01:01:01'),(26,20170111133013,1,'2020-01-01 01:01:01'),(27,20170117025759,1,'2020-01-01 01:01:01'),(28,20170118191001,1,'2020-01-01 01:01:01'),(29,20170119234632,1,'2020-01-01 01:01:01'),(30,20170124230432,1,'2020-01-01 01:01:01'),(31,20170127014618,1,'2020-01-01 01:01:01'),(32,20170131232841,1,'2020-01-01 01:01:01'),(33,20170223094154,1,'2020-01-01 01:01:01'),(34,20170306075207,1,'2020-01-01 01:01:01'),(35,20170309100733,1,'2020-01-01 01:01:01'),(36,20170331111922,1,'2020-01-01 01:01:01'),(37,20170502143928,1,'2020-01-01 01:01:01'),(38,20170504130602,1,'2020-01-01 01:01:01'),(39,20170509132100,1,'2020-01-01 01:01:01'),(40,20170519105647,1,'2020-01-01 01:01:01'),(41,20170519105648,1,'2020-01-01 01:01:01'),(42,20170831234300,1,'2020-01-01 01:01:01'),(43,20170831234301,1,'2020-01-01 01:01:01'),(44,20170831234303,1,'2020-01-01 01:01:01'),(45,20171116163618,1,'2020-01-01 01:01:01'),(46,20171219164727,1,'2020-01-01 01:01:01'),(47,20180620164811,1,'2020-01-01 01:01:01'),(48,20180620175054,1,'2020-01-01 01:01:01'),(49,20180620175055,1,'2020-01-01 01:01:01'),(50,20191010101639,1,'2020-01-01 01:01:01'),(51,20191010155147,1,'2020-01-01 01:01:01'),(52,20191220130734,1,'2020-01-01 01:01:01'),(53,20200311140000,1,'2020-01-01 01:01:01'),(54,20200405120000,1,'2020-01-01 01:01:01'),(55,20200407120000,1,'2020-01-01 01:01:01'),(56,20200420120000,1,'2020-01-01 01:01:01'),(57,20200504120000,1,'2020-01-01 01:01:01'),(58,20200512120000,1,'2020-01-01 01:01:01'),(59,20200707120000,1,'2020-01-01 01:01:01'),(60,20201011162341,1,'2020-01-01 01:01:01'),(61,20201021104586,1,'2020-01-01 01:01:01'),(62,20201102112520,1,'2020-01-01 01:01:01'),(63,20201208121729,1,'2020-01-01 01:01:01'),(64,20201215091637,1,'2020-01-01 01:01:01'),(65,20210119174155,1,'2020-01-01 01:01:01'),(66,20210326182902,1,'2020-01-01 01:01:01'),(67,20210421112652,1,'2020-01-01 01:01:01'),(68,20210506095025,1,'2020-01-01 01:01:01'),(69,20210513115729,1,'2020-01-01 01:01:01'),(70,20210526113559,1,'2020-01-01 01:01:01'),(71,20210601000001,1,'2020-01-01 01:01:01'),(72,20210601000002,1,'2020-01-01 01:01:01'),(73,20210601000003,1,'2020-01-01 01:01:01'),(74,20210601000004,1,'2020-01-01 01:01:01'),(75,20210601000005,1,'2020-01-01 01:01:01'),(76,20210601000006,1,'2020-01-01 01:01:01'),(77,20210601000007,1,'2020-01-01 01:01:01'),(78,20210601000008,1,'2020-01-01 01:01:01'),(79,20210606151329,1,'2020-01-01 01:01:01'),(80,20210616163757,1,'2020-01-01 01:01:01'),(81,20210617174723,1,'2020-01-01 01:01:01'),(82,20210622160235,1,'2020-01-01 01:01:01'),(83,20210623100031,1,'2020-01-01 01:01:01'),(84,20210623133615,1,'2020-01-01 01:01:01'),(85,20210708143152,1,'2020-01-01 01:01:01'),(86,20210709124443,1,'2020-01-01 01:01:01'),(87,20210712155608,1,'2020-01-01 01:01:01'),(88,20210714102108,1,'2020-01-01 01:01:01'),(89,20210719153709,1,'2020-01-01 01:01:01'),(90,20210721171531,1,'2020-01-01 01:01:01'),(91,20210723135713,1,'2020-01-01 01:01:01'),(92,20210802135933,1,'2020-01-01 01:01:01'),(93,20210806112844,1,'20
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mobile_device_management_solutions` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL,
`server_url` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mobile_device_management_solutions_name` (`name`,`server_url`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `munki_issues` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`issue_type` varchar(10) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_munki_issues_name` (`name`,`issue_type`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_cert_auth_associations` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sha256` char(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`cert_not_valid_after` timestamp NULL DEFAULT NULL,
`renew_command_uuid` varchar(127) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`,`sha256`),
KEY `renew_command_uuid_fk` (`renew_command_uuid`),
CONSTRAINT `renew_command_uuid_fk` FOREIGN KEY (`renew_command_uuid`) REFERENCES `nano_commands` (`command_uuid`),
CONSTRAINT `nano_cert_auth_associations_chk_1` CHECK ((`id` <> _utf8mb4'')),
CONSTRAINT `nano_cert_auth_associations_chk_2` CHECK ((`sha256` <> _utf8mb4''))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_command_results` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(31) COLLATE utf8mb4_unicode_ci NOT NULL,
`result` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`not_now_at` timestamp NULL DEFAULT NULL,
`not_now_tally` int NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`,`command_uuid`),
KEY `command_uuid` (`command_uuid`),
KEY `status` (`status`),
CONSTRAINT `nano_command_results_ibfk_1` FOREIGN KEY (`id`) REFERENCES `nano_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_command_results_ibfk_2` FOREIGN KEY (`command_uuid`) REFERENCES `nano_commands` (`command_uuid`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_command_results_chk_1` CHECK ((`status` <> _utf8mb4'')),
CONSTRAINT `nano_command_results_chk_2` CHECK ((substr(`result`,1,5) = _utf8mb4'<?xml'))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_commands` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`request_type` varchar(63) COLLATE utf8mb4_unicode_ci NOT NULL,
`command` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`subtype` enum('None','ProfileWithSecrets') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'None',
PRIMARY KEY (`command_uuid`),
CONSTRAINT `nano_commands_chk_1` CHECK ((`command_uuid` <> _utf8mb4'')),
CONSTRAINT `nano_commands_chk_2` CHECK ((`request_type` <> _utf8mb4''))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_dep_names` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`consumer_key` text COLLATE utf8mb4_unicode_ci,
`consumer_secret` text COLLATE utf8mb4_unicode_ci,
`access_token` text COLLATE utf8mb4_unicode_ci,
`access_secret` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`access_token_expiry` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`config_base_url` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`tokenpki_cert_pem` text COLLATE utf8mb4_unicode_ci,
`tokenpki_key_pem` text COLLATE utf8mb4_unicode_ci,
`syncer_cursor` varchar(1024) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`syncer_cursor_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`assigner_profile_uuid` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`assigner_profile_uuid_at` timestamp NULL DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`name`),
CONSTRAINT `nano_dep_names_chk_1` CHECK (((`tokenpki_cert_pem` is null) or (substr(`tokenpki_cert_pem`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----'))),
CONSTRAINT `nano_dep_names_chk_2` CHECK (((`tokenpki_key_pem` is null) or (substr(`tokenpki_key_pem`,1,5) = _utf8mb4'-----')))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_devices` (
`id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`identity_cert` text COLLATE utf8mb4_unicode_ci,
`serial_number` varchar(127) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`unlock_token` mediumblob,
`unlock_token_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`authenticate` text COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`authenticate_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token_update` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`token_update_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`bootstrap_token_b64` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`bootstrap_token_at` timestamp NULL DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`enroll_team_id` int unsigned DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`),
KEY `serial_number` (`serial_number`),
KEY `fk_nano_devices_team_id` (`enroll_team_id`),
CONSTRAINT `fk_nano_devices_team_id` FOREIGN KEY (`enroll_team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL,
CONSTRAINT `nano_devices_chk_1` CHECK (((`identity_cert` is null) or (substr(`identity_cert`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----'))),
CONSTRAINT `nano_devices_chk_2` CHECK (((`serial_number` is null) or (`serial_number` <> _utf8mb4''))),
CONSTRAINT `nano_devices_chk_3` CHECK (((`unlock_token` is null) or (length(`unlock_token`) > 0))),
CONSTRAINT `nano_devices_chk_4` CHECK ((`authenticate` <> _utf8mb4'')),
CONSTRAINT `nano_devices_chk_5` CHECK (((`token_update` is null) or (`token_update` <> _utf8mb4''))),
CONSTRAINT `nano_devices_chk_6` CHECK (((`bootstrap_token_b64` is null) or (`bootstrap_token_b64` <> _utf8mb4'')))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_enrollment_queue` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`active` tinyint(1) NOT NULL DEFAULT '1',
`priority` tinyint NOT NULL DEFAULT '0',
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`,`command_uuid`),
KEY `command_uuid` (`command_uuid`),
KEY `priority` (`priority` DESC,`created_at`),
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CONSTRAINT `nano_enrollment_queue_ibfk_1` FOREIGN KEY (`id`) REFERENCES `nano_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_enrollment_queue_ibfk_2` FOREIGN KEY (`command_uuid`) REFERENCES `nano_commands` (`command_uuid`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_enrollments` (
`id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`device_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`user_id` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`type` varchar(31) COLLATE utf8mb4_unicode_ci NOT NULL,
`topic` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`push_magic` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`token_hex` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`enabled` tinyint(1) NOT NULL DEFAULT '1',
`token_update_tally` int NOT NULL DEFAULT '1',
`last_seen_at` timestamp NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`enrolled_from_migration` tinyint unsigned NOT NULL DEFAULT '0',
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `user_id` (`user_id`),
KEY `device_id` (`device_id`),
KEY `type` (`type`),
CONSTRAINT `nano_enrollments_ibfk_1` FOREIGN KEY (`device_id`) REFERENCES `nano_devices` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_enrollments_ibfk_2` FOREIGN KEY (`user_id`) REFERENCES `nano_users` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_enrollments_chk_1` CHECK ((`id` <> _utf8mb4'')),
CONSTRAINT `nano_enrollments_chk_2` CHECK ((`type` <> _utf8mb4'')),
CONSTRAINT `nano_enrollments_chk_3` CHECK ((`topic` <> _utf8mb4'')),
CONSTRAINT `nano_enrollments_chk_4` CHECK ((`push_magic` <> _utf8mb4'')),
CONSTRAINT `nano_enrollments_chk_5` CHECK ((`token_hex` <> _utf8mb4''))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_push_certs` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`topic` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`cert_pem` text COLLATE utf8mb4_unicode_ci NOT NULL,
`key_pem` text COLLATE utf8mb4_unicode_ci NOT NULL,
`stale_token` int NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`topic`),
CONSTRAINT `nano_push_certs_chk_1` CHECK ((`topic` <> _utf8mb4'')),
CONSTRAINT `nano_push_certs_chk_2` CHECK ((substr(`cert_pem`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----')),
CONSTRAINT `nano_push_certs_chk_3` CHECK ((substr(`key_pem`,1,5) = _utf8mb4'-----'))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_users` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`user_short_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`user_long_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`token_update` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`token_update_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`user_authenticate` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`user_authenticate_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`user_authenticate_digest` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`user_authenticate_digest_at` timestamp NULL DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`,`device_id`),
UNIQUE KEY `idx_unique_id` (`id`),
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
KEY `device_id` (`device_id`),
CONSTRAINT `nano_users_ibfk_1` FOREIGN KEY (`device_id`) REFERENCES `nano_devices` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_users_chk_1` CHECK (((`user_short_name` is null) or (`user_short_name` <> _utf8mb4''))),
CONSTRAINT `nano_users_chk_2` CHECK (((`user_long_name` is null) or (`user_long_name` <> _utf8mb4''))),
CONSTRAINT `nano_users_chk_3` CHECK (((`token_update` is null) or (`token_update` <> _utf8mb4''))),
CONSTRAINT `nano_users_chk_4` CHECK (((`user_authenticate` is null) or (`user_authenticate` <> _utf8mb4''))),
CONSTRAINT `nano_users_chk_5` CHECK (((`user_authenticate_digest` is null) or (`user_authenticate_digest` <> _utf8mb4'')))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
SET @saved_cs_client = @@character_set_client;
/*!50503 SET character_set_client = utf8mb4 */;
/*!50001 CREATE VIEW `nano_view_queue` AS SELECT
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
1 AS `id`,
1 AS `created_at`,
1 AS `active`,
1 AS `priority`,
1 AS `command_uuid`,
1 AS `request_type`,
1 AS `command`,
1 AS `result_updated_at`,
1 AS `status`,
1 AS `result`*/;
SET character_set_client = @saved_cs_client;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `network_interfaces` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`mac` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`ip_address` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`broadcast` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`ibytes` bigint NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`interface` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`ipackets` bigint NOT NULL DEFAULT '0',
`last_change` bigint NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`mask` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`metric` int NOT NULL DEFAULT '0',
`mtu` int NOT NULL DEFAULT '0',
`obytes` bigint NOT NULL DEFAULT '0',
`ierrors` bigint NOT NULL DEFAULT '0',
`oerrors` bigint NOT NULL DEFAULT '0',
`opackets` bigint NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`point_to_point` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`type` int NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_network_interfaces_unique_ip_host_intf` (`ip_address`,`host_id`,`interface`),
KEY `idx_network_interfaces_hosts_fk` (`host_id`),
FULLTEXT KEY `ip_address_search` (`ip_address`),
CONSTRAINT `network_interfaces_ibfk_1` FOREIGN KEY (`host_id`) REFERENCES `hosts` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `operating_system_vulnerabilities` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`operating_system_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`cve` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`source` smallint DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`resolved_in_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_os_vulnerabilities_unq_os_id_cve` (`operating_system_id`,`cve`),
KEY `idx_os_vulnerabilities_cve` (`cve`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `operating_systems` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`version` varchar(150) COLLATE utf8mb4_unicode_ci NOT NULL,
`arch` varchar(150) COLLATE utf8mb4_unicode_ci NOT NULL,
`kernel_version` varchar(150) COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(50) COLLATE utf8mb4_unicode_ci NOT NULL,
`display_version` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`os_version_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_os` (`name`,`version`,`arch`,`kernel_version`,`platform`,`display_version`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `osquery_options` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`override_type` int NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`override_identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`options` json NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `osquery_options` VALUES (1,0,'','{\"options\": {\"logger_plugin\": \"tls\", \"pack_delimiter\": \"/\", \"logger_tls_period\": 10, \"distributed_plugin\": \"tls\", \"disable_distributed\": false, \"logger_tls_endpoint\": \"/api/v1/osquery/log\", \"distributed_interval\": 10, \"distributed_tls_max_attempts\": 3}, \"decorators\": {\"load\": [\"SELECT uuid AS host_uuid FROM system_info;\", \"SELECT hostname AS hostname FROM system_info;\"]}}');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `pack_targets` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`pack_id` int unsigned DEFAULT NULL,
`type` int DEFAULT NULL,
`target_id` int unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `constraint_pack_target_unique` (`pack_id`,`target_id`,`type`),
CONSTRAINT `pack_targets_ibfk_1` FOREIGN KEY (`pack_id`) REFERENCES `packs` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `packs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`disabled` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`platform` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`pack_type` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_pack_unique_name` (`name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `password_reset_requests` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`expires_at` timestamp NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`user_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token` varchar(1024) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `policies` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`team_id` int unsigned DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`resolution` text COLLATE utf8mb4_unicode_ci,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`query` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`description` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`author_id` int unsigned DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`platforms` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`critical` tinyint(1) NOT NULL DEFAULT '0',
`checksum` binary(16) NOT NULL,
`calendar_events_enabled` tinyint unsigned NOT NULL DEFAULT '0',
`software_installer_id` int unsigned DEFAULT NULL,
`script_id` int unsigned DEFAULT NULL,
`vpp_apps_teams_id` int unsigned DEFAULT NULL,
Microsoft Compliance Partner backend changes (#29540) For #27042. Ready for review, just missing integration tests that I will be writing today. - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [X] If database migrations are included, checked table schema to confirm autoupdate - For new Fleet configuration settings - [X] Verified that the setting can be managed via GitOps, or confirmed that the setting is explicitly being excluded from GitOps. If managing via Gitops: - [X] Verified that the setting is exported via `fleetctl generate-gitops` - [X] Added the setting to [the GitOps documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - [X] Verified that the setting is cleared on the server if it is not supplied in a YAML file (or that it is documented as being optional) - [x] Verified that any relevant UI is disabled when GitOps mode is enabled - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Added/updated automated tests - [X] Manual QA for all new/changed functionality --------- Co-authored-by: jacobshandling <61553566+jacobshandling@users.noreply.github.com> Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-06-11 17:22:46 +00:00
`conditional_access_enabled` tinyint unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_policies_checksum` (`checksum`),
KEY `idx_policies_author_id` (`author_id`),
KEY `idx_policies_team_id` (`team_id`),
KEY `fk_policies_software_installer_id` (`software_installer_id`),
KEY `fk_policies_script_id` (`script_id`),
KEY `fk_policies_vpp_apps_team_id` (`vpp_apps_teams_id`),
CONSTRAINT `policies_ibfk_3` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`),
CONSTRAINT `policies_ibfk_4` FOREIGN KEY (`script_id`) REFERENCES `scripts` (`id`),
CONSTRAINT `policies_ibfk_5` FOREIGN KEY (`vpp_apps_teams_id`) REFERENCES `vpp_apps_teams` (`id`),
CONSTRAINT `policies_queries_ibfk_1` FOREIGN KEY (`author_id`) REFERENCES `users` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `policy_automation_iterations` (
`policy_id` int unsigned NOT NULL,
`iteration` int NOT NULL,
PRIMARY KEY (`policy_id`),
CONSTRAINT `policy_automation_iterations_ibfk_1` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `policy_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`policy_id` int unsigned NOT NULL,
`label_id` int unsigned NOT NULL,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_policy_labels_policy_label` (`policy_id`,`label_id`),
KEY `policy_labels_label_id` (`label_id`),
CONSTRAINT `policy_labels_label_id` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE CASCADE,
CONSTRAINT `policy_labels_policy_id` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `policy_membership` (
`policy_id` int unsigned NOT NULL,
`host_id` int unsigned NOT NULL,
`passes` tinyint(1) DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`automation_iteration` int DEFAULT NULL,
PRIMARY KEY (`policy_id`,`host_id`),
KEY `idx_policy_membership_passes` (`passes`),
KEY `idx_policy_membership_host_id_passes` (`host_id`,`passes`),
CONSTRAINT `policy_membership_ibfk_1` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2023-11-21 19:52:06 +00:00
CREATE TABLE `policy_stats` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`policy_id` int unsigned NOT NULL,
`inherited_team_id` int unsigned DEFAULT NULL,
`passing_host_count` mediumint unsigned NOT NULL DEFAULT '0',
`failing_host_count` mediumint unsigned NOT NULL DEFAULT '0',
2023-11-21 19:52:06 +00:00
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`inherited_team_id_char` char(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS (if((`inherited_team_id` is null),_utf8mb4'global',cast(`inherited_team_id` as char charset utf8mb4))) VIRTUAL,
2023-11-21 19:52:06 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `policy_id` (`policy_id`,`inherited_team_id_char`),
2023-11-21 19:52:06 +00:00
CONSTRAINT `policy_stats_ibfk_1` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2023-11-21 19:52:06 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `queries` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`saved` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`query` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`author_id` int unsigned DEFAULT NULL,
`observer_can_run` tinyint(1) NOT NULL DEFAULT '0',
`team_id` int unsigned DEFAULT NULL,
`team_id_char` char(10) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`min_osquery_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`schedule_interval` int unsigned NOT NULL DEFAULT '0',
`automations_enabled` tinyint unsigned NOT NULL DEFAULT '0',
`logging_type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'snapshot',
`discard_data` tinyint(1) NOT NULL DEFAULT '1',
`is_scheduled` tinyint(1) GENERATED ALWAYS AS ((`schedule_interval` > 0)) STORED NOT NULL,
PRIMARY KEY (`id`),
2023-07-06 21:28:25 +00:00
UNIQUE KEY `idx_team_id_name_unq` (`team_id_char`,`name`),
UNIQUE KEY `idx_name_team_id_unq` (`name`,`team_id_char`),
KEY `author_id` (`author_id`),
KEY `idx_team_id_saved_auto_interval` (`team_id`,`saved`,`automations_enabled`,`schedule_interval`),
KEY `idx_queries_schedule_automations` (`is_scheduled`,`automations_enabled`),
2023-07-06 21:28:25 +00:00
CONSTRAINT `queries_ibfk_1` FOREIGN KEY (`author_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `queries_ibfk_2` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `query_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`query_id` int unsigned NOT NULL,
`label_id` int unsigned NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_query_labels_query_label` (`query_id`,`label_id`),
KEY `query_labels_label_id` (`label_id`),
CONSTRAINT `query_labels_label_id` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE CASCADE,
CONSTRAINT `query_labels_query_id` FOREIGN KEY (`query_id`) REFERENCES `queries` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2023-10-05 02:56:17 +00:00
CREATE TABLE `query_results` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`query_id` int unsigned NOT NULL,
`host_id` int unsigned NOT NULL,
2023-10-05 02:56:17 +00:00
`osquery_version` varchar(50) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`error` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`last_fetched` timestamp NOT NULL,
2023-10-05 02:56:17 +00:00
`data` json DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_query_id_host_id_last_fetched` (`query_id`,`host_id`,`last_fetched`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2023-10-05 02:56:17 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `scep_certificates` (
`serial` bigint NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(1024) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`not_valid_before` datetime NOT NULL,
`not_valid_after` datetime NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`certificate_pem` text COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`revoked` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`serial`),
CONSTRAINT `scep_certificates_ibfk_1` FOREIGN KEY (`serial`) REFERENCES `scep_serials` (`serial`),
CONSTRAINT `scep_certificates_chk_1` CHECK ((substr(`certificate_pem`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----')),
CONSTRAINT `scep_certificates_chk_2` CHECK (((`name` is null) or (`name` <> _utf8mb4'')))
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `scep_serials` (
`serial` bigint NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`serial`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scheduled_queries` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`pack_id` int unsigned DEFAULT NULL,
`query_id` int unsigned DEFAULT NULL,
`interval` int unsigned DEFAULT NULL,
`snapshot` tinyint(1) DEFAULT NULL,
`removed` tinyint(1) DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`platform` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT '',
`version` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT '',
`shard` int unsigned DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`query_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` varchar(1023) COLLATE utf8mb4_unicode_ci DEFAULT '',
`denylist` tinyint(1) DEFAULT NULL,
2023-07-10 18:56:44 +00:00
`team_id_char` char(10) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
UNIQUE KEY `unique_names_in_packs` (`name`,`pack_id`),
KEY `scheduled_queries_pack_id` (`pack_id`),
KEY `scheduled_queries_query_name` (`query_name`),
KEY `fk_scheduled_queries_queries` (`team_id_char`,`query_name`),
2023-07-10 20:01:46 +00:00
CONSTRAINT `scheduled_queries_ibfk_1` FOREIGN KEY (`team_id_char`, `query_name`) REFERENCES `queries` (`team_id_char`, `name`) ON DELETE CASCADE ON UPDATE CASCADE,
2023-07-06 21:28:25 +00:00
CONSTRAINT `scheduled_queries_pack_id` FOREIGN KEY (`pack_id`) REFERENCES `packs` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scheduled_query_stats` (
`host_id` int unsigned NOT NULL,
`scheduled_query_id` int unsigned NOT NULL,
`average_memory` bigint unsigned NOT NULL,
`denylisted` tinyint(1) DEFAULT NULL,
`executions` bigint unsigned NOT NULL,
`schedule_interval` int DEFAULT NULL,
`last_executed` timestamp NULL DEFAULT NULL,
`output_size` bigint unsigned NOT NULL,
`system_time` bigint unsigned NOT NULL,
`user_time` bigint unsigned NOT NULL,
`wall_time` bigint unsigned NOT NULL,
`query_type` tinyint NOT NULL DEFAULT '0',
PRIMARY KEY (`host_id`,`scheduled_query_id`,`query_type`),
KEY `scheduled_query_id` (`scheduled_query_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scim_groups` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`external_id` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`display_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_scim_groups_display_name` (`display_name`),
KEY `idx_scim_groups_external_id` (`external_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scim_last_request` (
`id` tinyint unsigned NOT NULL DEFAULT '1',
`status` varchar(31) COLLATE utf8mb4_unicode_ci NOT NULL,
`details` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scim_user_emails` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`scim_user_id` int unsigned NOT NULL,
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`primary` tinyint(1) DEFAULT NULL,
`type` varchar(31) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
KEY `idx_scim_user_emails_email_type` (`type`,`email`),
KEY `fk_scim_user_emails_scim_user_id` (`scim_user_id`),
CONSTRAINT `fk_scim_user_emails_scim_user_id` FOREIGN KEY (`scim_user_id`) REFERENCES `scim_users` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scim_user_group` (
`scim_user_id` int unsigned NOT NULL,
`group_id` int unsigned NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`scim_user_id`,`group_id`),
KEY `fk_scim_user_group_group_id` (`group_id`),
CONSTRAINT `fk_scim_user_group_group_id` FOREIGN KEY (`group_id`) REFERENCES `scim_groups` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_scim_user_group_scim_user_id` FOREIGN KEY (`scim_user_id`) REFERENCES `scim_users` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scim_users` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`external_id` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`user_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`given_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`family_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`active` tinyint(1) DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
Add support for IdP department to SCIM and add `FLEET_VAR_HOST_END_USER_IDP_DEPARTMENT` fleet variable (#30375) #29609 Verified the changes with [Entra ID's validator](https://scimvalidator.microsoft.com/) and adding the department attribute to the tester: <img width="1312" alt="Screenshot 2025-06-27 at 8 54 32 AM" src="https://github.com/user-attachments/assets/45a5deb8-7c65-49df-b3e8-eb05bea11f6b" /> <img width="1312" alt="Screenshot 2025-06-27 at 8 54 21 AM" src="https://github.com/user-attachments/assets/91b554b5-b0b9-4bb6-a0cf-4e3b40e6ce21" /> - Tested with Okta - TODO: Test with Entra ID and Google Workspace. - I decided to not fail profile deployment if a user has no department because it's not a required attribute, instead the `FLEET_VAR_HOST_END_USER_IDP_DEPARTMENT` will be replaced with the empty string. --- - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] If database migrations are included, checked table schema to confirm autoupdate (https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [X] Added/updated automated tests - [X] Manual QA for all new/changed functionality
2025-06-29 18:23:03 +00:00
`department` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_scim_users_user_name` (`user_name`),
KEY `idx_scim_users_external_id` (`external_id`)
Add support for IdP department to SCIM and add `FLEET_VAR_HOST_END_USER_IDP_DEPARTMENT` fleet variable (#30375) #29609 Verified the changes with [Entra ID's validator](https://scimvalidator.microsoft.com/) and adding the department attribute to the tester: <img width="1312" alt="Screenshot 2025-06-27 at 8 54 32 AM" src="https://github.com/user-attachments/assets/45a5deb8-7c65-49df-b3e8-eb05bea11f6b" /> <img width="1312" alt="Screenshot 2025-06-27 at 8 54 21 AM" src="https://github.com/user-attachments/assets/91b554b5-b0b9-4bb6-a0cf-4e3b40e6ce21" /> - Tested with Okta - TODO: Test with Entra ID and Google Workspace. - I decided to not fail profile deployment if a user has no department because it's not a required attribute, instead the `FLEET_VAR_HOST_END_USER_IDP_DEPARTMENT` will be replaced with the empty string. --- - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] If database migrations are included, checked table schema to confirm autoupdate (https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485) - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [X] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [X] Added/updated automated tests - [X] Manual QA for all new/changed functionality
2025-06-29 18:23:03 +00:00
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `script_contents` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`md5_checksum` binary(16) NOT NULL,
`contents` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_script_contents_md5_checksum` (`md5_checksum`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `script_upcoming_activities` (
`upcoming_activity_id` bigint unsigned NOT NULL,
`script_id` int unsigned DEFAULT NULL,
`script_content_id` int unsigned DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
`setup_experience_script_id` int unsigned DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`upcoming_activity_id`),
KEY `fk_script_upcoming_activities_script_id` (`script_id`),
KEY `fk_script_upcoming_activities_script_content_id` (`script_content_id`),
KEY `fk_script_upcoming_activities_policy_id` (`policy_id`),
KEY `fk_script_upcoming_activities_setup_experience_script_id` (`setup_experience_script_id`),
CONSTRAINT `fk_script_upcoming_activities_policy_id` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_script_upcoming_activities_script_content_id` FOREIGN KEY (`script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_script_upcoming_activities_script_id` FOREIGN KEY (`script_id`) REFERENCES `scripts` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_script_upcoming_activities_setup_experience_script_id` FOREIGN KEY (`setup_experience_script_id`) REFERENCES `setup_experience_scripts` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_script_upcoming_activities_upcoming_activity_id` FOREIGN KEY (`upcoming_activity_id`) REFERENCES `upcoming_activities` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scripts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`script_content_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_scripts_global_or_team_id_name` (`global_or_team_id`,`name`),
2023-10-10 22:00:45 +00:00
UNIQUE KEY `idx_scripts_team_name` (`team_id`,`name`),
KEY `script_content_id` (`script_content_id`),
CONSTRAINT `scripts_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `scripts_ibfk_2` FOREIGN KEY (`script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `secret_variables` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`value` blob NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
2024-12-10 23:17:21 +00:00
UNIQUE KEY `idx_secret_variables_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `sessions` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`accessed_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`user_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`key` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_session_unique_key` (`key`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `setup_experience_scripts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`script_content_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_setup_experience_scripts_global_or_team_id` (`global_or_team_id`),
KEY `idx_script_content_id` (`script_content_id`),
KEY `fk_setup_experience_scripts_ibfk_1` (`team_id`),
CONSTRAINT `fk_setup_experience_scripts_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_setup_experience_scripts_ibfk_2` FOREIGN KEY (`script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `setup_experience_status_results` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
Mark setup experience installs as "cancelled" and later fail them when certain bulk actions happen (#29355) Still adding tests but wanted to get this up for review of the overall "shape" of the fix When certain things happen like installer updates we delete pending upcoming_activities(UA) and host_software_install(HSI) entries and need to mark setup_experience_status_results(SESR) cancelled. When this happens if that UA/HSI are being depended on by setup experience we need to make sure that that setup experience result eventually gets marked failed. I kind of went back and forth a few times on how best to do this and avoid race conditions. One thing I tried was looking at existence of the UA/HSI but found that naively just trying to look at that in relation to the SESR entry seemed to have a few race conditions that were hard to resolve. There are a few possible states here we need to account for such as: un-activated, totally not yet running software install cancelled activated but not yet running on the host software install cancelled activated and running on the host software install cancelled before results are completely reported back What I eventually came around to was essentially that we want to mark the SESR cancelled in the same transaction we delete the HSI/UA in. We then finalize it by marking it failed and sending the activity the next time the host fetches setupm experience results. The new cancelled status never leaves fleet. This is a bit ugly but in my testing avoided the race conditions and works well. Note that to actually avoid setup experience hanging entirely we still need to fix #29357 which encompasses several cases where the unified queue can get completely stuck for a host # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [ ] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [ ] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [ ] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ ] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [ ] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [ ] Make sure fleetd is compatible with the latest released version of Fleet (see [Must rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)). - [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit feature/bugfix should only apply to one platform (`runtime.GOOS`). - [ ] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [ ] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)). - [ ] For unreleased bug fixes in a release candidate, confirmed that the fix is not expected to adversely impact load test results or alerted the release DRI if additional load testing is needed.
2025-05-27 20:52:51 +00:00
`status` enum('pending','running','success','failure','cancelled') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`software_installer_id` int unsigned DEFAULT NULL,
`host_software_installs_execution_id` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`vpp_app_team_id` int unsigned DEFAULT NULL,
`nano_command_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`setup_experience_script_id` int unsigned DEFAULT NULL,
`script_execution_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`error` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_setup_experience_scripts_host_uuid` (`host_uuid`),
KEY `idx_setup_experience_scripts_hsi_id` (`host_software_installs_execution_id`),
KEY `idx_setup_experience_scripts_nano_command_uuid` (`nano_command_uuid`),
KEY `idx_setup_experience_scripts_script_execution_id` (`script_execution_id`),
KEY `fk_setup_experience_status_results_si_id` (`software_installer_id`),
KEY `fk_setup_experience_status_results_va_id` (`vpp_app_team_id`),
KEY `fk_setup_experience_status_results_ses_id` (`setup_experience_script_id`),
CONSTRAINT `fk_setup_experience_status_results_ses_id` FOREIGN KEY (`setup_experience_script_id`) REFERENCES `setup_experience_scripts` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_setup_experience_status_results_si_id` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_setup_experience_status_results_va_id` FOREIGN KEY (`vpp_app_team_id`) REFERENCES `vpp_apps_teams` (`id`) ON DELETE CASCADE
Mark setup experience installs as "cancelled" and later fail them when certain bulk actions happen (#29355) Still adding tests but wanted to get this up for review of the overall "shape" of the fix When certain things happen like installer updates we delete pending upcoming_activities(UA) and host_software_install(HSI) entries and need to mark setup_experience_status_results(SESR) cancelled. When this happens if that UA/HSI are being depended on by setup experience we need to make sure that that setup experience result eventually gets marked failed. I kind of went back and forth a few times on how best to do this and avoid race conditions. One thing I tried was looking at existence of the UA/HSI but found that naively just trying to look at that in relation to the SESR entry seemed to have a few race conditions that were hard to resolve. There are a few possible states here we need to account for such as: un-activated, totally not yet running software install cancelled activated but not yet running on the host software install cancelled activated and running on the host software install cancelled before results are completely reported back What I eventually came around to was essentially that we want to mark the SESR cancelled in the same transaction we delete the HSI/UA in. We then finalize it by marking it failed and sending the activity the next time the host fetches setupm experience results. The new cancelled status never leaves fleet. This is a bit ugly but in my testing avoided the race conditions and works well. Note that to actually avoid setup experience hanging entirely we still need to fix #29357 which encompasses several cases where the unified queue can get completely stuck for a host # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [ ] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files) for more information. - [ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [ ] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [ ] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [ ] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [ ] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ ] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [ ] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [ ] Make sure fleetd is compatible with the latest released version of Fleet (see [Must rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)). - [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit feature/bugfix should only apply to one platform (`runtime.GOOS`). - [ ] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [ ] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)). - [ ] For unreleased bug fixes in a release candidate, confirmed that the fix is not expected to adversely impact load test results or alerted the release DRI if additional load testing is needed.
2025-05-27 20:52:51 +00:00
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`source` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`bundle_identifier` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT '',
`release` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`vendor_old` varchar(32) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`arch` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`vendor` varchar(114) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`extension_for` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`extension_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`title_id` int unsigned DEFAULT NULL,
`checksum` binary(16) NOT NULL,
`name_source` enum('basic','bundle_4.67') COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'basic',
`application_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_checksum` (`checksum`),
KEY `software_source_vendor_idx` (`source`,`vendor_old`),
KEY `title_id` (`title_id`),
KEY `idx_sw_name_source_browser` (`name`,`source`,`extension_for`),
KEY `software_listing_idx` (`name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_categories` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(63) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_categories_name` (`name`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `software_categories` VALUES (2,'Browsers'),(3,'Communication'),(4,'Developer tools'),(1,'Productivity');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_cpe` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`software_id` bigint unsigned DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`cpe` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `unq_software_id` (`software_id`),
KEY `software_cpe_cpe_idx` (`cpe`),
CONSTRAINT `software_cpe_ibfk_1` FOREIGN KEY (`software_id`) REFERENCES `software` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_cve` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`cve` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`source` int DEFAULT '0',
`software_id` bigint unsigned DEFAULT NULL,
`resolved_in_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
fix: add an index to mitigate vuln processing resource spikes (#26331) > For #26178 # Checklist for submitter ## Data from testing We tested the changes out by directly creating the index in our dogfood env. We saw a decrease in resource usage, captured in these screenshots: DB load (taller spike towards the left is without the index, subsequent spikes are after the index was added) ![image (1)](https://github.com/user-attachments/assets/141f8066-89e9-4a8e-ba7c-9fd3a09afea4) DB load over a longer period (overnight 2025-02-12 -> 2025-02-13) ![Screenshot 2025-02-13 at 9 05 05 AM](https://github.com/user-attachments/assets/bb0c6744-537d-4aec-960b-d100c4285d00) CPU utilization ![Screenshot 2025-02-13 at 10 25 17 AM](https://github.com/user-attachments/assets/eeea9ae5-5a10-4d50-91bc-3a806b359b39) Memory usage ![Screenshot 2025-02-13 at 10 26 52 AM](https://github.com/user-attachments/assets/bb77cdd8-41ef-4d90-a707-70b6d2976a59) --- If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality
2025-02-14 17:53:41 +00:00
UNIQUE KEY `unq_software_id_cve` (`software_id`,`cve`),
KEY `idx_software_cve_cve` (`cve`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_host_counts` (
`software_id` bigint unsigned NOT NULL,
`hosts_count` int unsigned NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`team_id` int unsigned NOT NULL DEFAULT '0',
`global_stats` tinyint unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`software_id`,`team_id`,`global_stats`),
KEY `idx_software_host_counts_updated_at_software_id` (`updated_at`,`software_id`),
KEY `idx_software_host_counts_team_id_hosts_count_software_id` (`team_id`,`hosts_count`,`software_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_install_upcoming_activities` (
`upcoming_activity_id` bigint unsigned NOT NULL,
`software_installer_id` int unsigned DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
`software_title_id` int unsigned DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`upcoming_activity_id`),
KEY `fk_software_install_upcoming_activities_software_installer_id` (`software_installer_id`),
KEY `fk_software_install_upcoming_activities_policy_id` (`policy_id`),
KEY `fk_software_install_upcoming_activities_software_title_id` (`software_title_id`),
CONSTRAINT `fk_software_install_upcoming_activities_policy_id` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_software_install_upcoming_activities_software_installer_id` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_software_install_upcoming_activities_software_title_id` FOREIGN KEY (`software_title_id`) REFERENCES `software_titles` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_software_install_upcoming_activities_upcoming_activity_id` FOREIGN KEY (`upcoming_activity_id`) REFERENCES `upcoming_activities` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-12-10 16:40:46 +00:00
CREATE TABLE `software_installer_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`software_installer_id` int unsigned NOT NULL,
`label_id` int unsigned NOT NULL,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
2024-12-10 16:40:46 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_installer_labels_software_installer_id_label_id` (`software_installer_id`,`label_id`),
KEY `label_id` (`label_id`),
CONSTRAINT `software_installer_labels_ibfk_1` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE CASCADE,
CONSTRAINT `software_installer_labels_ibfk_2` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE RESTRICT
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_installer_software_categories` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`software_category_id` int unsigned NOT NULL,
`software_installer_id` int unsigned NOT NULL,
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_software_installer_id_software_category_id` (`software_installer_id`,`software_category_id`),
KEY `software_category_id` (`software_category_id`),
CONSTRAINT `software_installer_software_categories_ibfk_1` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE CASCADE,
CONSTRAINT `software_installer_software_categories_ibfk_2` FOREIGN KEY (`software_category_id`) REFERENCES `software_categories` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_installers` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`title_id` int unsigned DEFAULT NULL,
`filename` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`pre_install_query` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`install_script_content_id` int unsigned NOT NULL,
`post_install_script_content_id` int unsigned DEFAULT NULL,
`storage_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
2024-09-08 17:26:26 +00:00
`uploaded_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`self_service` tinyint(1) NOT NULL DEFAULT '0',
`user_id` int unsigned DEFAULT NULL,
`user_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_email` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`url` varchar(4095) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
2024-09-08 17:26:26 +00:00
`package_ids` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`extension` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
2024-09-08 17:26:26 +00:00
`uninstall_script_content_id` int unsigned NOT NULL,
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`fleet_maintained_app_id` int unsigned DEFAULT NULL,
2024-10-25 22:11:56 +00:00
`install_during_setup` tinyint(1) NOT NULL DEFAULT '0',
`upgrade_code` varchar(48) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_installers_team_id_title_id` (`global_or_team_id`,`title_id`),
KEY `fk_software_installers_title` (`title_id`),
KEY `fk_software_installers_install_script_content_id` (`install_script_content_id`),
KEY `fk_software_installers_post_install_script_content_id` (`post_install_script_content_id`),
KEY `fk_software_installers_team_id` (`team_id`),
KEY `idx_software_installers_platform_title_id` (`platform`,`title_id`),
KEY `fk_software_installers_user_id` (`user_id`),
2024-09-08 17:26:26 +00:00
KEY `fk_uninstall_script_content_id` (`uninstall_script_content_id`),
KEY `fk_software_installers_fleet_library_app_id` (`fleet_maintained_app_id`),
CONSTRAINT `fk_software_installers_install_script_content_id` FOREIGN KEY (`install_script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE RESTRICT ON UPDATE CASCADE,
CONSTRAINT `fk_software_installers_post_install_script_content_id` FOREIGN KEY (`post_install_script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE RESTRICT ON UPDATE CASCADE,
CONSTRAINT `fk_software_installers_team_id` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_software_installers_title` FOREIGN KEY (`title_id`) REFERENCES `software_titles` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
2024-09-08 17:26:26 +00:00
CONSTRAINT `fk_software_installers_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_uninstall_script_content_id` FOREIGN KEY (`uninstall_script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE RESTRICT ON UPDATE CASCADE,
CONSTRAINT `software_installers_ibfk_1` FOREIGN KEY (`fleet_maintained_app_id`) REFERENCES `fleet_maintained_apps` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_title_icons` (
`id` int NOT NULL AUTO_INCREMENT,
`team_id` int unsigned NOT NULL,
`software_title_id` int unsigned NOT NULL,
`storage_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`filename` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_team_id_title_id_storage_id` (`team_id`,`software_title_id`),
KEY `idx_storage_id_team_id` (`storage_id`,`team_id`),
KEY `software_title_id` (`software_title_id`),
CONSTRAINT `software_title_icons_ibfk_1` FOREIGN KEY (`software_title_id`) REFERENCES `software_titles` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_titles` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`source` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`extension_for` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`bundle_identifier` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`additional_identifier` tinyint unsigned GENERATED ALWAYS AS ((case when (`source` = _utf8mb4'ios_apps') then 1 when (`source` = _utf8mb4'ipados_apps') then 2 when (`bundle_identifier` is not null) then 0 else NULL end)) VIRTUAL,
`is_kernel` tinyint(1) NOT NULL DEFAULT '0',
`application_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`unique_identifier` varchar(255) COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS (coalesce(`bundle_identifier`,`application_id`,`name`)) VIRTUAL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_titles_bundle_identifier` (`bundle_identifier`,`additional_identifier`),
UNIQUE KEY `idx_unique_sw_titles` (`unique_identifier`,`source`,`extension_for`),
KEY `idx_sw_titles` (`name`,`source`,`extension_for`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_titles_host_counts` (
`software_title_id` int unsigned NOT NULL,
`hosts_count` int unsigned NOT NULL,
`team_id` int unsigned NOT NULL DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`global_stats` tinyint unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`software_title_id`,`team_id`,`global_stats`),
KEY `idx_software_titles_host_counts_team_counts_title` (`team_id`,`hosts_count`,`software_title_id`),
KEY `idx_software_titles_host_counts_updated_at_software_title_id` (`updated_at`,`software_title_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `statistics` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`anonymous_identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `teams` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` varchar(1023) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`config` json DEFAULT NULL,
`name_bin` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin GENERATED ALWAYS AS (`name`) VIRTUAL,
`filename` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_teams_filename` (`filename`),
UNIQUE KEY `idx_name_bin` (`name_bin`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `upcoming_activities` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`priority` int NOT NULL DEFAULT '0',
`user_id` int unsigned DEFAULT NULL,
`fleet_initiated` tinyint(1) NOT NULL DEFAULT '0',
`activity_type` enum('script','software_install','software_uninstall','vpp_app_install') COLLATE utf8mb4_unicode_ci NOT NULL,
`execution_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`payload` json NOT NULL,
`activated_at` datetime(6) DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_upcoming_activities_execution_id` (`execution_id`),
KEY `idx_upcoming_activities_host_id_priority_created_at` (`host_id`,`priority`,`created_at`),
KEY `idx_upcoming_activities_host_id_activity_type` (`activity_type`,`host_id`),
KEY `fk_upcoming_activities_user_id` (`user_id`),
CONSTRAINT `fk_upcoming_activities_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `user_teams` (
`user_id` int unsigned NOT NULL,
`team_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`role` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`user_id`,`team_id`),
KEY `fk_user_teams_team_id` (`team_id`),
CONSTRAINT `user_teams_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `user_teams_ibfk_2` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `users` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`password` varbinary(255) NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`salt` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`admin_forced_password_reset` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`gravatar_url` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`position` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`sso_enabled` tinyint NOT NULL DEFAULT '0',
`global_role` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`api_only` tinyint(1) NOT NULL DEFAULT '0',
`mfa_enabled` tinyint(1) NOT NULL DEFAULT '0',
`settings` json NOT NULL DEFAULT (json_object()),
`invite_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_user_unique_email` (`email`),
UNIQUE KEY `invite_id` (`invite_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `users_deleted` (
`id` int unsigned NOT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`email` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `verification_tokens` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`user_id` int unsigned NOT NULL,
`token` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `token` (`token`),
KEY `verification_tokens_users` (`user_id`),
CONSTRAINT `verification_tokens_users` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-12-10 16:40:46 +00:00
CREATE TABLE `vpp_app_team_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`vpp_app_team_id` int unsigned NOT NULL,
`label_id` int unsigned NOT NULL,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
2024-12-10 16:40:46 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `idx_vpp_app_team_labels_vpp_app_team_id_label_id` (`vpp_app_team_id`,`label_id`),
KEY `label_id` (`label_id`),
CONSTRAINT `vpp_app_team_labels_ibfk_1` FOREIGN KEY (`vpp_app_team_id`) REFERENCES `vpp_apps_teams` (`id`) ON DELETE CASCADE,
CONSTRAINT `vpp_app_team_labels_ibfk_2` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE RESTRICT
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_app_team_software_categories` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`software_category_id` int unsigned NOT NULL,
`vpp_app_team_id` int unsigned NOT NULL,
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_vpp_app_team_id_software_category_id` (`vpp_app_team_id`,`software_category_id`),
KEY `software_category_id` (`software_category_id`),
CONSTRAINT `vpp_app_team_software_categories_ibfk_1` FOREIGN KEY (`vpp_app_team_id`) REFERENCES `vpp_apps_teams` (`id`) ON DELETE CASCADE,
CONSTRAINT `vpp_app_team_software_categories_ibfk_2` FOREIGN KEY (`software_category_id`) REFERENCES `software_categories` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_app_upcoming_activities` (
`upcoming_activity_id` bigint unsigned NOT NULL,
`adam_id` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(10) COLLATE utf8mb4_unicode_ci NOT NULL,
`vpp_token_id` int unsigned DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`upcoming_activity_id`),
KEY `fk_vpp_app_upcoming_activities_adam_id_platform` (`adam_id`,`platform`),
KEY `fk_vpp_app_upcoming_activities_vpp_token_id` (`vpp_token_id`),
KEY `fk_vpp_app_upcoming_activities_policy_id` (`policy_id`),
CONSTRAINT `fk_vpp_app_upcoming_activities_adam_id_platform` FOREIGN KEY (`adam_id`, `platform`) REFERENCES `vpp_apps` (`adam_id`, `platform`) ON DELETE CASCADE,
CONSTRAINT `fk_vpp_app_upcoming_activities_policy_id` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_vpp_app_upcoming_activities_upcoming_activity_id` FOREIGN KEY (`upcoming_activity_id`) REFERENCES `upcoming_activities` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_vpp_app_upcoming_activities_vpp_token_id` FOREIGN KEY (`vpp_token_id`) REFERENCES `vpp_tokens` (`id`) ON DELETE SET NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_apps` (
`adam_id` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL,
`title_id` int unsigned DEFAULT NULL,
`bundle_identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`icon_url` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`latest_version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`platform` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`adam_id`,`platform`),
KEY `fk_vpp_apps_title` (`title_id`),
CONSTRAINT `fk_vpp_apps_title` FOREIGN KEY (`title_id`) REFERENCES `software_titles` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_apps_teams` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`adam_id` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int NOT NULL DEFAULT '0',
`platform` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`self_service` tinyint(1) NOT NULL DEFAULT '0',
`vpp_token_id` int unsigned NOT NULL,
`install_during_setup` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_global_or_team_id_adam_id` (`global_or_team_id`,`adam_id`,`platform`),
KEY `team_id` (`team_id`),
KEY `adam_id` (`adam_id`,`platform`),
KEY `fk_vpp_apps_teams_vpp_token_id` (`vpp_token_id`),
CONSTRAINT `fk_vpp_apps_teams_vpp_token_id` FOREIGN KEY (`vpp_token_id`) REFERENCES `vpp_tokens` (`id`) ON DELETE CASCADE,
CONSTRAINT `vpp_apps_teams_ibfk_2` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE,
CONSTRAINT `vpp_apps_teams_ibfk_3` FOREIGN KEY (`adam_id`, `platform`) REFERENCES `vpp_apps` (`adam_id`, `platform`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_token_teams` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`vpp_token_id` int unsigned NOT NULL,
`team_id` int unsigned DEFAULT NULL,
`null_team_type` enum('none','allteams','noteam') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'none',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_vpp_token_teams_team_id` (`team_id`),
KEY `fk_vpp_token_teams_vpp_token_id` (`vpp_token_id`),
CONSTRAINT `fk_vpp_token_teams_team_id` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_vpp_token_teams_vpp_token_id` FOREIGN KEY (`vpp_token_id`) REFERENCES `vpp_tokens` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_tokens` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`organization_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`location` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`renew_at` timestamp NOT NULL,
`token` blob NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_vpp_tokens_location` (`location`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vulnerability_host_counts` (
`cve` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
`team_id` int unsigned NOT NULL DEFAULT '0',
`host_count` int unsigned NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`global_stats` tinyint(1) NOT NULL DEFAULT '0',
UNIQUE KEY `cve_team_id_global_stats` (`cve`,`team_id`,`global_stats`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_mdm_command_queue` (
`enrollment_id` int unsigned NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`enrollment_id`,`command_uuid`),
KEY `command_uuid` (`command_uuid`),
CONSTRAINT `windows_mdm_command_queue_ibfk_1` FOREIGN KEY (`enrollment_id`) REFERENCES `mdm_windows_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `windows_mdm_command_queue_ibfk_2` FOREIGN KEY (`command_uuid`) REFERENCES `windows_mdm_commands` (`command_uuid`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_mdm_command_results` (
`enrollment_id` int unsigned NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_result` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`response_id` int unsigned NOT NULL,
`status_code` varchar(31) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`enrollment_id`,`command_uuid`),
KEY `command_uuid` (`command_uuid`),
KEY `response_id` (`response_id`),
CONSTRAINT `windows_mdm_command_results_ibfk_1` FOREIGN KEY (`enrollment_id`) REFERENCES `mdm_windows_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `windows_mdm_command_results_ibfk_2` FOREIGN KEY (`command_uuid`) REFERENCES `windows_mdm_commands` (`command_uuid`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `windows_mdm_command_results_ibfk_3` FOREIGN KEY (`response_id`) REFERENCES `windows_mdm_responses` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_mdm_commands` (
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_command` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`target_loc_uri` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`command_uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_mdm_responses` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`enrollment_id` int unsigned NOT NULL,
`raw_response` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `enrollment_id` (`enrollment_id`),
CONSTRAINT `windows_mdm_responses_ibfk_1` FOREIGN KEY (`enrollment_id`) REFERENCES `mdm_windows_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_updates` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`date_epoch` int unsigned NOT NULL,
`kb_id` int unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_windows_updates` (`host_id`,`kb_id`),
KEY `idx_update_date` (`host_id`,`date_epoch`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `wstep_cert_auth_associations` (
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sha256` char(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`,`sha256`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `wstep_certificates` (
`serial` bigint unsigned NOT NULL,
`name` varchar(1024) COLLATE utf8mb4_unicode_ci NOT NULL,
`not_valid_before` datetime NOT NULL,
`not_valid_after` datetime NOT NULL,
`certificate_pem` text COLLATE utf8mb4_unicode_ci NOT NULL,
`revoked` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`serial`),
CONSTRAINT `wstep_certificates_ibfk_1` FOREIGN KEY (`serial`) REFERENCES `wstep_serials` (`serial`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `wstep_serials` (
`serial` bigint unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`serial`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `yara_rules` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`contents` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_yara_rules_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!50001 DROP VIEW IF EXISTS `nano_view_queue`*/;
/*!50001 SET @saved_cs_client = @@character_set_client */;
/*!50001 SET @saved_cs_results = @@character_set_results */;
/*!50001 SET @saved_col_connection = @@collation_connection */;
/*!50001 SET character_set_client = utf8mb4 */;
/*!50001 SET character_set_results = utf8mb4 */;
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
/*!50001 SET collation_connection = utf8mb4_unicode_ci */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!50001 CREATE ALGORITHM=UNDEFINED */
/*!50013 DEFINER=`root`@`%` SQL SECURITY INVOKER */
/*!50001 VIEW `nano_view_queue` AS select (`q`.`id` collate utf8mb4_unicode_ci) AS `id`,`q`.`created_at` AS `created_at`,`q`.`active` AS `active`,`q`.`priority` AS `priority`,(`c`.`command_uuid` collate utf8mb4_unicode_ci) AS `command_uuid`,(`c`.`request_type` collate utf8mb4_unicode_ci) AS `request_type`,(`c`.`command` collate utf8mb4_unicode_ci) AS `command`,`r`.`updated_at` AS `result_updated_at`,(`r`.`status` collate utf8mb4_unicode_ci) AS `status`,(`r`.`result` collate utf8mb4_unicode_ci) AS `result` from ((`nano_enrollment_queue` `q` join `nano_commands` `c` on((`q`.`command_uuid` = `c`.`command_uuid`))) left join `nano_command_results` `r` on(((`r`.`command_uuid` = `q`.`command_uuid`) and (`r`.`id` = `q`.`id`)))) order by `q`.`priority` desc,`q`.`created_at` */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!50001 SET character_set_client = @saved_cs_client */;
/*!50001 SET character_set_results = @saved_cs_results */;
/*!50001 SET collation_connection = @saved_col_connection */;