fleet/server/datastore/mysql/schema.sql

2398 lines
170 KiB
MySQL
Raw Normal View History

/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
CREATE TABLE `abm_tokens` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`organization_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`apple_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`terms_expired` tinyint(1) NOT NULL DEFAULT '0',
`renew_at` timestamp NOT NULL,
`token` blob NOT NULL,
`macos_default_team_id` int unsigned DEFAULT NULL,
`ios_default_team_id` int unsigned DEFAULT NULL,
`ipados_default_team_id` int unsigned DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_abm_tokens_organization_name` (`organization_name`),
KEY `fk_abm_tokens_macos_default_team_id` (`macos_default_team_id`),
KEY `fk_abm_tokens_ios_default_team_id` (`ios_default_team_id`),
KEY `fk_abm_tokens_ipados_default_team_id` (`ipados_default_team_id`),
CONSTRAINT `fk_abm_tokens_ios_default_team_id` FOREIGN KEY (`ios_default_team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_abm_tokens_ipados_default_team_id` FOREIGN KEY (`ipados_default_team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_abm_tokens_macos_default_team_id` FOREIGN KEY (`macos_default_team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `activities` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`user_id` int unsigned DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`user_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`activity_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`details` json DEFAULT NULL,
`streamed` tinyint(1) NOT NULL DEFAULT '0',
`user_email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`fleet_initiated` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
KEY `fk_activities_user_id` (`user_id`),
KEY `activities_streamed_idx` (`streamed`),
KEY `activities_created_at_idx` (`created_at`),
CONSTRAINT `activities_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `aggregated_stats` (
`id` bigint unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`json_value` json NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`global_stats` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`,`type`,`global_stats`),
KEY `idx_aggregated_stats_updated_at` (`updated_at`),
KEY `aggregated_stats_type_idx` (`type`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `android_devices` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`device_id` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`enterprise_specific_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`android_policy_id` int unsigned DEFAULT NULL,
`last_policy_sync_time` datetime(3) DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_android_devices_host_id` (`host_id`),
UNIQUE KEY `idx_android_devices_device_id` (`device_id`),
UNIQUE KEY `idx_android_devices_enterprise_specific_id` (`enterprise_specific_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `android_enterprises` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`signup_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`enterprise_id` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`signup_token` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`pubsub_topic_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_id` int unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `app_config_json` (
`id` int unsigned NOT NULL DEFAULT '1',
`json_value` json NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `app_config_json` VALUES (1,'{\"mdm\": {\"ios_updates\": {\"deadline\": null, \"minimum_version\": null}, \"macos_setup\": {\"script\": null, \"software\": null, \"bootstrap_package\": null, \"macos_setup_assistant\": null, \"enable_end_user_authentication\": false, \"enable_release_device_manually\": false}, \"macos_updates\": {\"deadline\": null, \"minimum_version\": null}, \"ipados_updates\": {\"deadline\": null, \"minimum_version\": null}, \"macos_settings\": {\"custom_settings\": null}, \"macos_migration\": {\"mode\": \"\", \"enable\": false, \"webhook_url\": \"\"}, \"windows_updates\": {\"deadline_days\": null, \"grace_period_days\": null}, \"apple_server_url\": \"\", \"windows_settings\": {\"custom_settings\": null}, \"apple_bm_terms_expired\": false, \"apple_business_manager\": null, \"enable_disk_encryption\": false, \"enabled_and_configured\": false, \"end_user_authentication\": {\"idp_name\": \"\", \"metadata\": \"\", \"entity_id\": \"\", \"issuer_uri\": \"\", \"metadata_url\": \"\"}, \"volume_purchasing_program\": null, \"windows_migration_enabled\": false, \"android_enabled_and_configured\": false, \"windows_enabled_and_configured\": false, \"apple_bm_enabled_and_configured\": false}, \"gitops\": {\"repository_url\": \"\", \"gitops_mode_enabled\": false}, \"scripts\": null, \"features\": {\"enable_host_users\": true, \"enable_software_inventory\": false}, \"org_info\": {\"org_name\": \"\", \"contact_url\": \"\", \"org_logo_url\": \"\", \"org_logo_url_light_background\": \"\"}, \"integrations\": {\"jira\": null, \"zendesk\": null, \"digicert\": null, \"google_calendar\": null, \"ndes_scep_proxy\": null, \"custom_scep_proxy\": null}, \"sso_settings\": {\"idp_name\": \"\", \"metadata\": \"\", \"entity_id\": \"\", \"enable_sso\": false, \"issuer_uri\": \"\", \"metadata_url\": \"\", \"idp_image_url\": \"\", \"enable_jit_role_sync\": false, \"enable_sso_idp_login\": false, \"enable_jit_provisioning\": false}, \"agent_options\": {\"config\": {\"options\": {\"logger_plugin\": \"tls\", \"pack_delimiter\": \"/\", \"logger_tls_period\": 10, \"distributed_plugin\": \"tls\", \"disable_distributed\": false, \"logger_tls_endpoint\": \"/api/osquery/log\", \"distributed_interval\": 10, \"distributed_tls_max_attempts\": 3}, \"decorators\": {\"load\": [\"SELECT uuid AS host_uuid FROM system_info;\", \"SELECT hostname AS hostname FROM system_info;\"]}}, \"overrides\": {}}, \"fleet_desktop\": {\"transparency_url\": \"\"}, \"smtp_settings\": {\"port\": 587, \"domain\": \"\", \"server\": \"\", \"password\": \"\", \"user_name\": \"\", \"configured\": false, \"enable_smtp\": false, \"enable_ssl_tls\": true, \"sender_address\": \"\", \"enable_start_tls\": true, \"verify_ssl_certs\": true, \"authentication_type\": \"0\", \"authentication_method\": \"0\"}, \"server_settings\": {\"server_url\": \"\", \"enable_analytics\": false, \"query_report_cap\": 0, \"scripts_disabled\": false, \"deferred_save_host\": false, \"live_query_disabled\": false, \"ai_features_disabled\": false, \"query_reports_disabled\": false}, \"webhook_settings\": {\"interval\": \"0s\", \"activities_webhook\": {\"destination_url\": \"\", \"enable_activities_webhook\": false}, \"host_status_webhook\": {\"days_count\": 0, \"destination_url\": \"\", \"host_percentage\": 0, \"enable_host_status_webhook\": false}, \"vulnerabilities_webhook\": {\"destination_url\": \"\", \"host_batch_size\": 0, \"enable_vulnerabilities_webhook\": false}, \"failing_policies_webhook\": {\"policy_ids\": null, \"destination_url\": \"\", \"host_batch_size\": 0, \"enable_failing_policies_webhook\": false}}, \"host_expiry_settings\": {\"host_expiry_window\": 0, \"host_expiry_enabled\": false}, \"vulnerability_settings\": {\"databases_path\": \"\"}, \"activity_expiry_settings\": {\"activity_expiry_window\": 0, \"activity_expiry_enabled\": false}}','2020-01-01 01:01:01','2020-01-01 01:01:01');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `ca_config_assets` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`type` enum('digicert','custom_scep_proxy') COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`value` blob NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_ca_config_assets_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-15 00:15:35 +00:00
CREATE TABLE `calendar_events` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
2024-03-15 00:15:35 +00:00
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`start_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`end_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`event` json NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`timezone` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`uuid_bin` binary(16) NOT NULL,
`uuid` varchar(36) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS (insert(insert(insert(insert(hex(`uuid_bin`),9,0,_utf8mb4'-'),14,0,_utf8mb4'-'),19,0,_utf8mb4'-'),24,0,_utf8mb4'-')) VIRTUAL,
PRIMARY KEY (`id`),
Check for calendar updates after callbacks from Google (#20156) #19352 Video explaining code changes: https://www.loom.com/share/370200a276b84aa388effd6ebd762e01?sid=038508c4-f3c2-40c0-baf6-6b6df682d1f0 In maintenance windows using Google Calendar, calendar event is now recreated within 30 seconds if deleted or moved to the past. - Added new endpoint for Google Calendar: `/api/_version_/fleet/calendar/webhook/{event_uuid}` - Added UUID to `calendar_events` table to make webhook lookup more efficient - webhook endpoint will only recreate event if needed -- it will not fire webhook. Webhook is still done by the cron job. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes:
2024-07-08 15:20:03 +00:00
UNIQUE KEY `idx_one_calendar_event_per_email` (`email`),
UNIQUE KEY `idx_calendar_events_uuid_bin_unique` (`uuid_bin`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2024-03-15 00:15:35 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `carve_blocks` (
`metadata_id` int unsigned NOT NULL,
`block_id` int NOT NULL,
`data` longblob,
PRIMARY KEY (`metadata_id`,`block_id`),
CONSTRAINT `carve_blocks_ibfk_1` FOREIGN KEY (`metadata_id`) REFERENCES `carve_metadata` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `carve_metadata` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`block_count` int unsigned NOT NULL,
`block_size` int unsigned NOT NULL,
`carve_size` bigint unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`carve_id` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`request_id` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`session_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`expired` tinyint DEFAULT '0',
`max_block` int DEFAULT '-1',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`error` text COLLATE utf8mb4_unicode_ci,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_session_id` (`session_id`),
UNIQUE KEY `idx_name` (`name`),
KEY `host_id` (`host_id`),
CONSTRAINT `carve_metadata_ibfk_1` FOREIGN KEY (`host_id`) REFERENCES `hosts` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `cron_stats` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`instance` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`stats_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Monitor and alert on errors in cron jobs (#24347) for #19930 # Checklist for submitter - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added/updated tests - [X] If database migrations are included, checked table schema to confirm autoupdate - [X] Manual QA for all new/changed functionality # Details This PR adds a new feature to the existing monitoring add-on. The add-on will now send an SNS alert whenever a scheduled job like "vulnerabilities" or "apple_mdm_apns_pusher" exits early due to errors. The alert contains the job type and the set of errors (there can be multiple, since jobs can have multiple sub-jobs). By default the SNS topic for this new alert is the same as the one for the existing cron system alerts, but it can be configured to use a separate topic (e.g. dogfood instance will post to a separate slack channel). The actual changes are: **On the server side:** - Add errors field to cron_stats table (json DEFAULT NULL) - Added errors var to `Schedule` struct to collect errors from jobs - In `RunAllJobs`, collect err from job into new errors var - Update `Schedule.updateStats`and `CronStats.UpdateCronStats`to accept errors argument - If provided, update errors field of cron_stats table **On the monitor side:** - Add new SQL query to look for all completed schedules since last run with non-null errors - send SNS with job ID, name, errors # Testing New automated testing was added for the functional code that gathers and stores errors from cron runs in the database. To test the actual Lambda, I added a row in my `cron_stats` table with errors, then compiled and ran the Lambda executable locally, pointing it to my local mysql and localstack instances: ``` 2024/12/03 14:43:54 main.go:258: Lambda execution environment not found. Falling back to local execution. 2024/12/03 14:43:54 main.go:133: Connected to database! 2024/12/03 14:43:54 main.go:161: Row vulnerabilities last updated at 2024-11-27 03:30:03 +0000 UTC 2024/12/03 14:43:54 main.go:163: *** 1h hasn't updated in more than vulnerabilities, alerting! (status completed) 2024/12/03 14:43:54 main.go:70: Sending SNS Message 2024/12/03 14:43:54 main.go:74: Sending 'Environment: dev Message: Fleet cron 'vulnerabilities' hasn't updated in more than 1h. Last status was 'completed' at 2024-11-27 03:30:03 +0000 UTC.' to 'arn:aws:sns:us-east-1:000000000000:topic1' 2024/12/03 14:43:54 main.go:82: { MessageId: "260864ff-4cc9-4951-acea-cef883b2de5f" } 2024/12/03 14:43:54 main.go:198: *** mdm_apple_profile_manager job had errors, alerting! (errors {"something": "wrong"}) 2024/12/03 14:43:54 main.go:70: Sending SNS Message 2024/12/03 14:43:54 main.go:74: Sending 'Environment: dev Message: Fleet cron 'mdm_apple_profile_manager' (last updated 2024-12-03 20:34:14 +0000 UTC) raised errors during its run: {"something": "wrong"}.' to 'arn:aws:sns:us-east-1:000000000000:topic1' 2024/12/03 14:43:54 main.go:82: { MessageId: "5cd085ef-89f6-42c1-8470-d80a22b295f8"
2024-12-19 21:55:29 +00:00
`errors` json DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_cron_stats_name_created_at` (`name`,`created_at`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2022-06-01 16:06:57 +00:00
CREATE TABLE `cve_meta` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`cve` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
`cvss_score` double DEFAULT NULL,
`epss_probability` double DEFAULT NULL,
`cisa_known_exploit` tinyint(1) DEFAULT NULL,
`published` timestamp NULL DEFAULT NULL,
`description` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
PRIMARY KEY (`cve`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `distributed_query_campaign_targets` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`type` int DEFAULT NULL,
`distributed_query_campaign_id` int unsigned DEFAULT NULL,
`target_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `distributed_query_campaigns` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`query_id` int unsigned DEFAULT NULL,
`status` int DEFAULT NULL,
`user_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `email_changes` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`user_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token` varchar(128) COLLATE utf8mb4_unicode_ci NOT NULL,
`new_email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_email_changes_token` (`token`) USING BTREE,
KEY `fk_email_changes_users` (`user_id`),
CONSTRAINT `fk_email_changes_users` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `enroll_secrets` (
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`secret` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
`team_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`secret`),
KEY `fk_enroll_secrets_team_id` (`team_id`),
CONSTRAINT `enroll_secrets_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `eulas` (
`id` int unsigned NOT NULL,
`token` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`bytes` longblob,
`created_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `fleet_library_apps` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`token` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`installer_url` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`sha256` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`bundle_identifier` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`install_script_content_id` int unsigned NOT NULL,
`uninstall_script_content_id` int unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_fleet_library_apps_token` (`token`),
KEY `fk_fleet_library_apps_install_script_content` (`install_script_content_id`),
KEY `fk_fleet_library_apps_uninstall_script_content` (`uninstall_script_content_id`),
CONSTRAINT `fk_fleet_library_apps_install_script_content` FOREIGN KEY (`install_script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_fleet_library_apps_uninstall_script_content` FOREIGN KEY (`uninstall_script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_activities` (
`host_id` int unsigned NOT NULL,
`activity_id` int unsigned NOT NULL,
PRIMARY KEY (`host_id`,`activity_id`),
KEY `fk_host_activities_activity_id` (`activity_id`),
CONSTRAINT `host_activities_ibfk_1` FOREIGN KEY (`activity_id`) REFERENCES `activities` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_additional` (
`host_id` int unsigned NOT NULL,
`additional` json DEFAULT NULL,
PRIMARY KEY (`host_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_batteries` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`serial_number` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`cycle_count` int NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`health` varchar(40) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_batteries_host_id_serial_number` (`host_id`,`serial_number`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-15 00:15:35 +00:00
CREATE TABLE `host_calendar_events` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`calendar_event_id` int unsigned NOT NULL,
`webhook_status` tinyint NOT NULL,
2024-03-15 00:15:35 +00:00
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_one_calendar_event_per_host` (`host_id`),
KEY `calendar_event_id` (`calendar_event_id`),
CONSTRAINT `host_calendar_events_ibfk_1` FOREIGN KEY (`calendar_event_id`) REFERENCES `calendar_events` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_certificates` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`not_valid_after` datetime(6) NOT NULL,
`not_valid_before` datetime(6) NOT NULL,
`certificate_authority` tinyint(1) NOT NULL,
`common_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`key_algorithm` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`key_strength` int NOT NULL,
`key_usage` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`serial` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`signing_algorithm` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`subject_country` varchar(2) COLLATE utf8mb4_unicode_ci NOT NULL,
`subject_org` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`subject_org_unit` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`subject_common_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`issuer_country` varchar(2) COLLATE utf8mb4_unicode_ci NOT NULL,
`issuer_org` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`issuer_org_unit` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`issuer_common_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sha1_sum` binary(20) NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`deleted_at` datetime(6) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_host_certs_hid_cn` (`host_id`,`common_name`),
KEY `idx_host_certs_not_valid_after` (`host_id`,`not_valid_after`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_dep_assignments` (
`host_id` int unsigned NOT NULL,
`added_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`deleted_at` timestamp NULL DEFAULT NULL,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`assign_profile_response` varchar(15) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`response_updated_at` timestamp NULL DEFAULT NULL,
`retry_job_id` int unsigned NOT NULL DEFAULT '0',
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
`abm_token_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`host_id`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
KEY `idx_hdep_response` (`assign_profile_response`,`response_updated_at`),
KEY `fk_host_dep_assignments_abm_token_id` (`abm_token_id`),
CONSTRAINT `fk_host_dep_assignments_abm_token_id` FOREIGN KEY (`abm_token_id`) REFERENCES `abm_tokens` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2022-03-08 12:05:53 +00:00
CREATE TABLE `host_device_auth` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
2022-03-08 12:05:53 +00:00
PRIMARY KEY (`host_id`),
UNIQUE KEY `idx_host_device_auth_token` (`token`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2022-03-08 12:05:53 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_disk_encryption_keys` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`base64_encrypted` text COLLATE utf8mb4_unicode_ci NOT NULL,
`base64_encrypted_salt` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`key_slot` tinyint unsigned DEFAULT NULL,
`decryptable` tinyint(1) DEFAULT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`reset_requested` tinyint(1) NOT NULL DEFAULT '0',
`client_error` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`host_id`),
KEY `idx_host_disk_encryption_keys_decryptable` (`decryptable`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_disk_encryption_keys_archive` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`hardware_serial` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`base64_encrypted` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`base64_encrypted_salt` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`key_slot` tinyint unsigned DEFAULT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
KEY `idx_host_disk_encryption_keys_archive_host_created_at` (`host_id`,`created_at` DESC)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_disks` (
`host_id` int unsigned NOT NULL,
`gigs_disk_space_available` decimal(10,2) NOT NULL DEFAULT '0.00',
`percent_disk_space_available` decimal(10,2) NOT NULL DEFAULT '0.00',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`encrypted` tinyint(1) DEFAULT NULL,
`gigs_total_disk_space` decimal(10,2) NOT NULL DEFAULT '0.00',
PRIMARY KEY (`host_id`),
KEY `idx_host_disks_gigs_disk_space_available` (`gigs_disk_space_available`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2022-10-08 12:57:46 +00:00
CREATE TABLE `host_display_names` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`display_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
2022-10-08 12:57:46 +00:00
PRIMARY KEY (`host_id`),
KEY `display_name` (`display_name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2022-10-08 12:57:46 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_emails` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`source` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `idx_host_emails_host_id_email` (`host_id`,`email`),
KEY `idx_host_emails_email` (`email`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Added host_issues (#19654) * /api/latest/fleet/hosts now returns `critical_vulnerabilities_count` for premium users. This data is held in the new `host_issues` table. The failing policies total is updated in real-time, while the critical vulnerabilities total is updated every hour after vulnerabilities job. * /api/latest/fleet/hosts can be sorted by total_issues_count by specifying `order_key=issues` query parameter. #18115 Video commentary: https://www.loom.com/share/ff4425e217db43d6b3ba4aa075374808 @getvictor TODO: Replacing `disable_failing_policies` with `disable_issues` will be done in a separate PR. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality
2024-06-17 17:15:42 +00:00
CREATE TABLE `host_issues` (
`host_id` int unsigned NOT NULL,
`failing_policies_count` int unsigned NOT NULL DEFAULT '0',
`critical_vulnerabilities_count` int unsigned NOT NULL DEFAULT '0',
`total_issues_count` int unsigned NOT NULL DEFAULT '0',
Added host_issues (#19654) * /api/latest/fleet/hosts now returns `critical_vulnerabilities_count` for premium users. This data is held in the new `host_issues` table. The failing policies total is updated in real-time, while the critical vulnerabilities total is updated every hour after vulnerabilities job. * /api/latest/fleet/hosts can be sorted by total_issues_count by specifying `order_key=issues` query parameter. #18115 Video commentary: https://www.loom.com/share/ff4425e217db43d6b3ba4aa075374808 @getvictor TODO: Replacing `disable_failing_policies` with `disable_issues` will be done in a separate PR. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality
2024-06-17 17:15:42 +00:00
`created_at` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
`updated_at` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
PRIMARY KEY (`host_id`),
KEY `total_issues_count` (`total_issues_count`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm` (
`host_id` int unsigned NOT NULL,
`enrolled` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`server_url` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`installed_from_dep` tinyint(1) NOT NULL DEFAULT '0',
`mdm_id` int unsigned DEFAULT NULL,
`is_server` tinyint(1) DEFAULT NULL,
`fleet_enroll_ref` varchar(36) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`enrollment_status` enum('On (manual)','On (automatic)','Pending','Off') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS ((case when (`is_server` = 1) then NULL when ((`enrolled` = 1) and (`installed_from_dep` = 0)) then _utf8mb4'On (manual)' when ((`enrolled` = 1) and (`installed_from_dep` = 1)) then _utf8mb4'On (automatic)' when ((`enrolled` = 0) and (`installed_from_dep` = 1)) then _utf8mb4'Pending' when ((`enrolled` = 0) and (`installed_from_dep` = 0)) then _utf8mb4'Off' else NULL end)) VIRTUAL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`host_id`),
KEY `host_mdm_mdm_id_idx` (`mdm_id`),
KEY `host_mdm_enrolled_installed_from_dep_idx` (`enrolled`,`installed_from_dep`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_actions` (
`host_id` int unsigned NOT NULL,
`lock_ref` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`wipe_ref` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`unlock_pin` varchar(6) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`unlock_ref` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`fleet_platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`host_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_apple_awaiting_configuration` (
`host_uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`awaiting_configuration` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`host_uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_apple_bootstrap_packages` (
`host_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`host_uuid`),
KEY `command_uuid` (`command_uuid`),
CONSTRAINT `host_mdm_apple_bootstrap_packages_ibfk_1` FOREIGN KEY (`command_uuid`) REFERENCES `nano_commands` (`command_uuid`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-14 21:08:19 +00:00
CREATE TABLE `host_mdm_apple_declarations` (
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`operation_type` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`detail` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`token` binary(16) NOT NULL,
`declaration_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`declaration_identifier` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`declaration_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`secrets_updated_at` datetime(6) DEFAULT NULL,
2024-03-14 21:08:19 +00:00
PRIMARY KEY (`host_uuid`,`declaration_uuid`),
KEY `status` (`status`),
KEY `operation_type` (`operation_type`),
CONSTRAINT `host_mdm_apple_declarations_ibfk_1` FOREIGN KEY (`status`) REFERENCES `mdm_delivery_status` (`status`) ON UPDATE CASCADE,
CONSTRAINT `host_mdm_apple_declarations_ibfk_2` FOREIGN KEY (`operation_type`) REFERENCES `mdm_operation_types` (`operation_type`) ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2024-03-14 21:08:19 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_apple_profiles` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`profile_identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`host_uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`operation_type` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`detail` text COLLATE utf8mb4_unicode_ci,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`profile_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
various profile fixes (#11084) ### Related tickets https://github.com/fleetdm/fleet/issues/10775 https://github.com/fleetdm/fleet/issues/10678 https://github.com/fleetdm/fleet/issues/11024 https://github.com/fleetdm/fleet/issues/11026 ### What's happening - Implemented the hashing mechanism defined by @mna in #10678, however this mechanism is mainly relevant for batch profile updates via the CLI, we can't leverage it when a host switches teams. - Modified `BulkSetPendingMDMAppleHostProfiles` so when two profiles with the same identifier are sheduled both for removal and update, the function will now mark only the `install` as `pending` so it's picked by the cron, and will `DELETE` the `remove` entry from the database so it's not picked by the cron and never sent to the user. - `GetHostMDMProfiles` and consequently the profiles returned in `GET /api/_version_/fleet/hosts` return `host_mdm_apple_profiles.state = NULL` as "Enforcing (pending", the distinction between `status = 'pending'` and `status IS NULL` is only useful for the cron, for users both mean the same thing, and all our profile aggregations already behave this way. - Using the solution implemented by @gillespi314 in https://github.com/fleetdm/fleet/pull/10998 we're now deleting the host row from `host_disk_encryption_keys` if a host is moved from a team that enforces disk encryption to a team that doesn't. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-04-09 02:23:36 +00:00
`checksum` binary(16) NOT NULL,
`retries` tinyint unsigned NOT NULL DEFAULT '0',
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`secrets_updated_at` datetime(6) DEFAULT NULL,
`ignore_error` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`host_uuid`,`profile_uuid`),
KEY `status` (`status`),
KEY `operation_type` (`operation_type`),
CONSTRAINT `host_mdm_apple_profiles_ibfk_1` FOREIGN KEY (`status`) REFERENCES `mdm_delivery_status` (`status`) ON UPDATE CASCADE,
CONSTRAINT `host_mdm_apple_profiles_ibfk_2` FOREIGN KEY (`operation_type`) REFERENCES `mdm_operation_types` (`operation_type`) ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Deleting an ABM iOS/iPadOS host keeps it in Fleet. (#21433) #21073 - Deleted iOS/iPadOS host will continue to report to Fleet as long as host is in Apple Business Manager (ABM). - Refetching an offline iOS/iPadOS host will not add new MDM commands to the queue if previous refetch has not completed yet. Video demo: https://www.loom.com/share/2f7ecb22e1924d4cbbbdd7dd297439ef?sid=dbfe1939-cb46-47ca-a7a8-84965ed68a7e I considered `nano_command_results` but could not think of an efficient way to use `nano_command_results` to determine if refetch was already done. The problem is that we're overloading `command_uuid` to include the `REFETCH` identifier. So we can't simply add an index on `(uuid, command_uuid, updated_at)`. We need to do a partial text match on `command_uuid` which then requires a full scan of the matching `updated_at` timestamps. It feels like a search like this could blow up for a large `nano_enrollment_queue` and `nano_command_results` table. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality
2024-08-21 13:51:04 +00:00
CREATE TABLE `host_mdm_commands` (
`host_id` int unsigned NOT NULL,
`command_type` varchar(31) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`host_id`,`command_type`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_managed_certificates` (
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`type` enum('digicert','custom_scep_proxy','ndes') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'ndes',
`ca_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'NDES',
`challenge_retrieved_at` timestamp(6) NULL DEFAULT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`not_valid_after` datetime(6) DEFAULT NULL,
PRIMARY KEY (`host_uuid`,`profile_uuid`,`ca_name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_mdm_windows_profiles` (
`host_uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`operation_type` varchar(20) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`detail` text COLLATE utf8mb4_unicode_ci,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`profile_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`retries` tinyint unsigned NOT NULL DEFAULT '0',
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`checksum` binary(16) NOT NULL DEFAULT '0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0',
`secrets_updated_at` datetime(6) DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`host_uuid`,`profile_uuid`),
KEY `status` (`status`),
KEY `operation_type` (`operation_type`),
CONSTRAINT `host_mdm_windows_profiles_ibfk_1` FOREIGN KEY (`status`) REFERENCES `mdm_delivery_status` (`status`) ON UPDATE CASCADE,
CONSTRAINT `host_mdm_windows_profiles_ibfk_2` FOREIGN KEY (`operation_type`) REFERENCES `mdm_operation_types` (`operation_type`) ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_munki_info` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`deleted_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`host_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_munki_issues` (
`host_id` int unsigned NOT NULL,
`munki_issue_id` int unsigned NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`host_id`,`munki_issue_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_operating_system` (
`host_id` int unsigned NOT NULL,
`os_id` int unsigned NOT NULL,
PRIMARY KEY (`host_id`),
KEY `idx_host_operating_system_id` (`os_id`),
CONSTRAINT `host_operating_system_ibfk_1` FOREIGN KEY (`os_id`) REFERENCES `operating_systems` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_orbit_info` (
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`version` varchar(50) COLLATE utf8mb4_unicode_ci NOT NULL,
Added orbit_version, fleet_desktop_version, and scripts_enabled to host details. (#18123) #17361 #17148 In GET fleet/hosts/:id response, added the following fields: - orbit_version - `orbit_version == null` means this agent is not an orbit agent - fleet_desktop_version - `fleet_desktop_version == null` means this agent is not an orbit agent or it is an older version which is not collecting the desktop version - `fleet_desktop_version == ""` means this agent is an orbit agent but does not have fleet desktop - scripts_enabled - `scripts_enabled == null` means this agent is not an orbit agent or it is an older version which is not collecting scripts_enabled In orbit_info table, added the following fields: - desktop_version - scripts_enabled Updated docs for orbit_info PR: https://github.com/fleetdm/fleet/pull/18135 Updated API docs: https://github.com/fleetdm/fleet/pull/17814 MDM lock/unlock/wipe error messages are not part of this PR. They will be in a separate PR. # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features. - [x] Added/updated tests - [x] If database migrations are included, checked table schema to confirm autoupdate - [x] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [x] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [x] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2024-04-09 21:33:44 +00:00
`desktop_version` varchar(50) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`scripts_enabled` tinyint(1) DEFAULT NULL,
PRIMARY KEY (`host_id`),
KEY `idx_host_orbit_info_version` (`version`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_script_results` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`execution_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`output` text COLLATE utf8mb4_unicode_ci NOT NULL,
`runtime` int unsigned NOT NULL DEFAULT '0',
`exit_code` int DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`script_id` int unsigned DEFAULT NULL,
`user_id` int unsigned DEFAULT NULL,
`sync_request` tinyint(1) NOT NULL DEFAULT '0',
`script_content_id` int unsigned DEFAULT NULL,
`host_deleted_at` timestamp NULL DEFAULT NULL,
`timeout` int DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
2024-10-25 22:11:56 +00:00
`setup_experience_script_id` int unsigned DEFAULT NULL,
`is_internal` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_script_results_execution_id` (`execution_id`),
KEY `idx_host_script_results_host_exit_created` (`host_id`,`exit_code`,`created_at`),
KEY `fk_host_script_results_script_id` (`script_id`),
2023-10-10 22:00:45 +00:00
KEY `idx_host_script_created_at` (`host_id`,`script_id`,`created_at`),
KEY `fk_host_script_results_user_id` (`user_id`),
KEY `script_content_id` (`script_content_id`),
KEY `fk_script_result_policy_id` (`policy_id`),
2024-10-25 22:11:56 +00:00
KEY `fk_host_script_results_setup_experience_id` (`setup_experience_script_id`),
CONSTRAINT `fk_host_script_results_script_id` FOREIGN KEY (`script_id`) REFERENCES `scripts` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_host_script_results_setup_experience_id` FOREIGN KEY (`setup_experience_script_id`) REFERENCES `setup_experience_scripts` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_host_script_results_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `host_script_results_ibfk_1` FOREIGN KEY (`script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE,
CONSTRAINT `host_script_results_ibfk_2` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_seen_times` (
`host_id` int unsigned NOT NULL,
`seen_time` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`host_id`),
KEY `idx_host_seen_times_seen_time` (`seen_time`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_software` (
`host_id` int unsigned NOT NULL,
`software_id` bigint unsigned NOT NULL,
`last_opened_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`host_id`,`software_id`),
KEY `host_software_software_fk` (`software_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_software_installed_paths` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`software_id` bigint unsigned NOT NULL,
`installed_path` text COLLATE utf8mb4_unicode_ci NOT NULL,
Add `team_identifier` to macOS software (#23766) Changes to add `team_identifier` signing information to macOS applications on the `/api/latest/fleet/hosts/:id/software` API endpoint. Docs: https://github.com/fleetdm/fleet/pull/23743 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added/updated tests - [X] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [X] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ X Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [X] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [X] Orbit runs on macOS, Linux and Windows. Check if the orbit feature/bugfix should only apply to one platform (`runtime.GOOS`). - [X] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [X] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)). --------- Co-authored-by: Tim Lee <timlee@fleetdm.com> Co-authored-by: Ian Littman <iansltx@gmail.com>
2024-11-15 17:17:04 +00:00
`team_identifier` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
KEY `host_id_software_id_idx` (`host_id`,`software_id`)
Add `team_identifier` to macOS software (#23766) Changes to add `team_identifier` signing information to macOS applications on the `/api/latest/fleet/hosts/:id/software` API endpoint. Docs: https://github.com/fleetdm/fleet/pull/23743 - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [X] Added/updated tests - [X] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [X] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [X] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [X] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [ X Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [X] Manual QA for all new/changed functionality - For Orbit and Fleet Desktop changes: - [X] Orbit runs on macOS, Linux and Windows. Check if the orbit feature/bugfix should only apply to one platform (`runtime.GOOS`). - [X] Manual QA must be performed in the three main OSs, macOS, Windows and Linux. - [X] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)). --------- Co-authored-by: Tim Lee <timlee@fleetdm.com> Co-authored-by: Ian Littman <iansltx@gmail.com>
2024-11-15 17:17:04 +00:00
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_software_installs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`execution_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`host_id` int unsigned NOT NULL,
`software_installer_id` int unsigned DEFAULT NULL,
`pre_install_query_output` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`install_script_output` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`install_script_exit_code` int DEFAULT NULL,
`post_install_script_output` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`post_install_script_exit_code` int DEFAULT NULL,
`user_id` int unsigned DEFAULT NULL,
2024-09-08 17:26:26 +00:00
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`self_service` tinyint(1) NOT NULL DEFAULT '0',
2024-09-08 17:26:26 +00:00
`host_deleted_at` timestamp(6) NULL DEFAULT NULL,
`removed` tinyint NOT NULL DEFAULT '0',
2024-09-08 17:26:26 +00:00
`uninstall_script_output` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`uninstall_script_exit_code` int DEFAULT NULL,
`uninstall` tinyint unsigned NOT NULL DEFAULT '0',
`status` enum('pending_install','failed_install','installed','pending_uninstall','failed_uninstall') COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS ((case when (`removed` = 1) then NULL when ((`post_install_script_exit_code` is not null) and (`post_install_script_exit_code` = 0)) then _utf8mb4'installed' when ((`post_install_script_exit_code` is not null) and (`post_install_script_exit_code` <> 0)) then _utf8mb4'failed_install' when ((`install_script_exit_code` is not null) and (`install_script_exit_code` = 0)) then _utf8mb4'installed' when ((`install_script_exit_code` is not null) and (`install_script_exit_code` <> 0)) then _utf8mb4'failed_install' when ((`pre_install_query_output` is not null) and (`pre_install_query_output` = _utf8mb4'')) then _utf8mb4'failed_install' when ((`host_id` is not null) and (`uninstall` = 0)) then _utf8mb4'pending_install' when ((`uninstall_script_exit_code` is not null) and (`uninstall_script_exit_code` <> 0)) then _utf8mb4'failed_uninstall' when ((`uninstall_script_exit_code` is not null) and (`uninstall_script_exit_code` = 0)) then NULL when ((`host_id` is not null) and (`uninstall` = 1)) then _utf8mb4'pending_uninstall' else NULL end)) STORED,
`policy_id` int unsigned DEFAULT NULL,
`installer_filename` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '[deleted installer]',
`version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'unknown',
`software_title_id` int unsigned DEFAULT NULL,
`software_title_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '[deleted title]',
`execution_status` enum('pending_install','failed_install','installed','pending_uninstall','failed_uninstall') COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS ((case when ((`post_install_script_exit_code` is not null) and (`post_install_script_exit_code` = 0)) then _utf8mb4'installed' when ((`post_install_script_exit_code` is not null) and (`post_install_script_exit_code` <> 0)) then _utf8mb4'failed_install' when ((`install_script_exit_code` is not null) and (`install_script_exit_code` = 0)) then _utf8mb4'installed' when ((`install_script_exit_code` is not null) and (`install_script_exit_code` <> 0)) then _utf8mb4'failed_install' when ((`pre_install_query_output` is not null) and (`pre_install_query_output` = _utf8mb4'')) then _utf8mb4'failed_install' when ((`host_id` is not null) and (`uninstall` = 0)) then _utf8mb4'pending_install' when ((`uninstall_script_exit_code` is not null) and (`uninstall_script_exit_code` <> 0)) then _utf8mb4'failed_uninstall' when ((`uninstall_script_exit_code` is not null) and (`uninstall_script_exit_code` = 0)) then NULL when ((`host_id` is not null) and (`uninstall` = 1)) then _utf8mb4'pending_uninstall' else NULL end)) VIRTUAL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_software_installs_execution_id` (`execution_id`),
KEY `fk_host_software_installs_user_id` (`user_id`),
KEY `idx_host_software_installs_host_installer` (`host_id`,`software_installer_id`),
KEY `fk_software_install_policy_id` (`policy_id`),
KEY `fk_host_software_installs_installer_id` (`software_installer_id`),
KEY `fk_host_software_installs_software_title_id` (`software_title_id`),
CONSTRAINT `fk_host_software_installs_installer_id` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_host_software_installs_software_title_id` FOREIGN KEY (`software_title_id`) REFERENCES `software_titles` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_host_software_installs_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `host_software_installs_ibfk_1` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_updates` (
`host_id` int unsigned NOT NULL,
`software_updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`host_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_users` (
`host_id` int unsigned NOT NULL,
`uid` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`username` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`groupname` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`removed_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`user_type` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`shell` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT '',
PRIMARY KEY (`host_id`,`uid`,`username`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `host_vpp_software_installs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`adam_id` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`user_id` int unsigned DEFAULT NULL,
`self_service` tinyint(1) NOT NULL DEFAULT '0',
`associated_event_id` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`platform` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`removed` tinyint NOT NULL DEFAULT '0',
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
`vpp_token_id` int unsigned DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_host_vpp_software_installs_command_uuid` (`command_uuid`),
KEY `user_id` (`user_id`),
KEY `adam_id` (`adam_id`,`platform`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
KEY `fk_host_vpp_software_installs_vpp_token_id` (`vpp_token_id`),
KEY `fk_host_vpp_software_installs_policy_id` (`policy_id`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
CONSTRAINT `fk_host_vpp_software_installs_vpp_token_id` FOREIGN KEY (`vpp_token_id`) REFERENCES `vpp_tokens` (`id`) ON DELETE SET NULL,
CONSTRAINT `host_vpp_software_installs_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `host_vpp_software_installs_ibfk_3` FOREIGN KEY (`adam_id`, `platform`) REFERENCES `vpp_apps` (`adam_id`, `platform`) ON DELETE CASCADE,
CONSTRAINT `host_vpp_software_installs_ibfk_4` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `hosts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`osquery_host_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`detail_updated_at` timestamp NULL DEFAULT NULL,
`node_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL,
`hostname` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`osquery_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`os_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`build` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`platform_like` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`code_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`uptime` bigint NOT NULL DEFAULT '0',
`memory` bigint NOT NULL DEFAULT '0',
`cpu_type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`cpu_subtype` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`cpu_brand` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`cpu_physical_cores` int NOT NULL DEFAULT '0',
`cpu_logical_cores` int NOT NULL DEFAULT '0',
`hardware_vendor` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`hardware_model` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`hardware_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`hardware_serial` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`computer_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`primary_ip_id` int unsigned DEFAULT NULL,
`distributed_interval` int DEFAULT '0',
`logger_tls_period` int DEFAULT '0',
`config_tls_refresh` int DEFAULT '0',
`primary_ip` varchar(45) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`primary_mac` varchar(17) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`label_updated_at` timestamp NOT NULL DEFAULT '2000-01-01 00:00:00',
`last_enrolled_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`refetch_requested` tinyint(1) NOT NULL DEFAULT '0',
`team_id` int unsigned DEFAULT NULL,
`policy_updated_at` timestamp NOT NULL DEFAULT '2000-01-01 00:00:00',
`public_ip` varchar(45) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`orbit_node_key` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin DEFAULT NULL,
`refetch_critical_queries_until` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_osquery_host_id` (`osquery_host_id`),
UNIQUE KEY `idx_host_unique_nodekey` (`node_key`),
UNIQUE KEY `idx_host_unique_orbitnodekey` (`orbit_node_key`),
KEY `fk_hosts_team_id` (`team_id`),
KEY `hosts_platform_idx` (`platform`),
KEY `idx_hosts_hardware_serial` (`hardware_serial`),
KEY `idx_hosts_uuid` (`uuid`),
CONSTRAINT `hosts_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `invite_teams` (
`invite_id` int unsigned NOT NULL,
`team_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`role` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`invite_id`,`team_id`),
KEY `fk_team_id` (`team_id`),
CONSTRAINT `invite_teams_ibfk_1` FOREIGN KEY (`invite_id`) REFERENCES `invites` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `invite_teams_ibfk_2` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `invites` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`invited_by` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`position` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`token` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sso_enabled` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`global_role` varchar(64) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`mfa_enabled` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_invite_unique_email` (`email`),
UNIQUE KEY `idx_invite_unique_key` (`token`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `jobs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`args` json DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`state` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`retries` int NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`error` text COLLATE utf8mb4_unicode_ci,
`not_before` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
2024-08-06 18:49:01 +00:00
PRIMARY KEY (`id`),
KEY `idx_jobs_state_not_before_updated_at` (`state`,`not_before`,`updated_at`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `jobs` VALUES (1,'2024-03-20 00:00:00','2024-03-20 00:00:00','macos_setup_assistant','{\"task\": \"update_all_profiles\"}','queued',0,'','2024-03-20 00:00:00');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `label_membership` (
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`label_id` int unsigned NOT NULL,
`host_id` int unsigned NOT NULL,
PRIMARY KEY (`host_id`,`label_id`),
KEY `idx_lm_label_id` (`label_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`query` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`label_type` int unsigned NOT NULL DEFAULT '1',
`label_membership_type` int unsigned NOT NULL DEFAULT '0',
Add author ID to labels (#27055) For #27035 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality ## Details This PR adds an `author_id` column to the `labels` table, and adds the associated properties to the `Label` and `LabelSpec` types. When a new label is created via the UI or API, an author ID is set on the label if one can be inferred from the context. Otherwise, the author ID is set to `null`. ## Authz and Automated testing Additional backend authorization logic is introduced in a follow-on PR, https://github.com/fleetdm/fleet/pull/27089, because rconciling all of the test updates between this PR and https://github.com/fleetdm/fleet/pull/27038 was getting complicated. ## Manual Testing * Tested in the UI by creating a new label on the Hosts page * Tested via Gitops by merging this branch with https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops` with a global config with `labels:` in it.
2025-03-20 21:05:16 +00:00
`author_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_label_unique_name` (`name`),
Add author ID to labels (#27055) For #27035 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality ## Details This PR adds an `author_id` column to the `labels` table, and adds the associated properties to the `Label` and `LabelSpec` types. When a new label is created via the UI or API, an author ID is set on the label if one can be inferred from the context. Otherwise, the author ID is set to `null`. ## Authz and Automated testing Additional backend authorization logic is introduced in a follow-on PR, https://github.com/fleetdm/fleet/pull/27089, because rconciling all of the test updates between this PR and https://github.com/fleetdm/fleet/pull/27038 was getting complicated. ## Manual Testing * Tested in the UI by creating a new label on the Hosts page * Tested via Gitops by merging this branch with https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops` with a global config with `labels:` in it.
2025-03-20 21:05:16 +00:00
KEY `author_id` (`author_id`),
FULLTEXT KEY `labels_search` (`name`),
CONSTRAINT `labels_ibfk_1` FOREIGN KEY (`author_id`) REFERENCES `users` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
Add author ID to labels (#27055) For #27035 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality ## Details This PR adds an `author_id` column to the `labels` table, and adds the associated properties to the `Label` and `LabelSpec` types. When a new label is created via the UI or API, an author ID is set on the label if one can be inferred from the context. Otherwise, the author ID is set to `null`. ## Authz and Automated testing Additional backend authorization logic is introduced in a follow-on PR, https://github.com/fleetdm/fleet/pull/27089, because rconciling all of the test updates between this PR and https://github.com/fleetdm/fleet/pull/27038 was getting complicated. ## Manual Testing * Tested in the UI by creating a new label on the Hosts page * Tested via Gitops by merging this branch with https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops` with a global config with `labels:` in it.
2025-03-20 21:05:16 +00:00
INSERT INTO `labels` VALUES (1,'2024-04-03 00:00:00','2024-04-03 00:00:00','macOS 14+ (Sonoma+)','macOS hosts with version 14 and above','select 1 from os_version where platform = \'darwin\' and major >= 14;','darwin',1,0,NULL),(2,'2024-06-28 00:00:00','2024-06-28 00:00:00','iOS','All iOS hosts','','ios',1,1,NULL),(3,'2024-06-28 00:00:00','2024-06-28 00:00:00','iPadOS','All iPadOS hosts','','ipados',1,1,NULL),(4,'2024-09-27 00:00:00','2024-09-27 00:00:00','Fedora Linux','All Fedora hosts','select 1 from os_version where name = \'Fedora Linux\';','rhel',1,0,NULL),(5,'2025-02-25 00:00:00','2025-02-25 00:00:00','Android','All Android hosts','','android',1,1,NULL);
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `locks` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`owner` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`expires_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_name` (`name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_bootstrap_packages` (
`team_id` int unsigned NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`sha256` binary(32) NOT NULL,
`bytes` longblob,
`token` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`team_id`),
UNIQUE KEY `idx_token` (`token`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_configuration_profiles` (
`profile_id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`mobileconfig` mediumblob NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`uploaded_at` timestamp(6) NULL DEFAULT NULL,
various profile fixes (#11084) ### Related tickets https://github.com/fleetdm/fleet/issues/10775 https://github.com/fleetdm/fleet/issues/10678 https://github.com/fleetdm/fleet/issues/11024 https://github.com/fleetdm/fleet/issues/11026 ### What's happening - Implemented the hashing mechanism defined by @mna in #10678, however this mechanism is mainly relevant for batch profile updates via the CLI, we can't leverage it when a host switches teams. - Modified `BulkSetPendingMDMAppleHostProfiles` so when two profiles with the same identifier are sheduled both for removal and update, the function will now mark only the `install` as `pending` so it's picked by the cron, and will `DELETE` the `remove` entry from the database so it's not picked by the cron and never sent to the user. - `GetHostMDMProfiles` and consequently the profiles returned in `GET /api/_version_/fleet/hosts` return `host_mdm_apple_profiles.state = NULL` as "Enforcing (pending", the distinction between `status = 'pending'` and `status IS NULL` is only useful for the cron, for users both mean the same thing, and all our profile aggregations already behave this way. - Using the solution implemented by @gillespi314 in https://github.com/fleetdm/fleet/pull/10998 we're now deleting the host row from `host_disk_encryption_keys` if a host is moved from a team that enforces disk encryption to a team that doesn't. # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-04-09 02:23:36 +00:00
`checksum` binary(16) NOT NULL,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`secrets_updated_at` datetime(6) DEFAULT NULL,
PRIMARY KEY (`profile_uuid`),
UNIQUE KEY `idx_mdm_apple_config_prof_team_identifier` (`team_id`,`identifier`),
UNIQUE KEY `idx_mdm_apple_config_prof_team_name` (`team_id`,`name`),
UNIQUE KEY `idx_mdm_apple_config_prof_id` (`profile_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-14 21:08:19 +00:00
CREATE TABLE `mdm_apple_declaration_activation_references` (
`declaration_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`reference` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
2024-03-14 21:08:19 +00:00
PRIMARY KEY (`declaration_uuid`,`reference`),
KEY `reference` (`reference`),
CONSTRAINT `mdm_apple_declaration_activation_references_ibfk_1` FOREIGN KEY (`declaration_uuid`) REFERENCES `mdm_apple_declarations` (`declaration_uuid`) ON UPDATE CASCADE,
CONSTRAINT `mdm_apple_declaration_activation_references_ibfk_2` FOREIGN KEY (`reference`) REFERENCES `mdm_apple_declarations` (`declaration_uuid`) ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-14 21:08:19 +00:00
CREATE TABLE `mdm_apple_declarations` (
`declaration_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`team_id` int unsigned NOT NULL DEFAULT '0',
`identifier` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_json` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`uploaded_at` timestamp(6) NULL DEFAULT NULL,
`auto_increment` bigint NOT NULL AUTO_INCREMENT,
`secrets_updated_at` datetime(6) DEFAULT NULL,
`token` binary(16) GENERATED ALWAYS AS (unhex(md5(concat(`raw_json`,ifnull(`secrets_updated_at`,_utf8mb4''))))) STORED,
2024-03-14 21:08:19 +00:00
PRIMARY KEY (`declaration_uuid`),
UNIQUE KEY `idx_mdm_apple_declaration_team_identifier` (`team_id`,`identifier`),
UNIQUE KEY `idx_mdm_apple_declaration_team_name` (`team_id`,`name`),
UNIQUE KEY `auto_increment` (`auto_increment`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2024-03-14 21:08:19 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_declarative_requests` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`enrollment_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`message_type` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_json` text COLLATE utf8mb4_unicode_ci,
PRIMARY KEY (`id`),
KEY `mdm_apple_declarative_requests_enrollment_id` (`enrollment_id`),
CONSTRAINT `mdm_apple_declarative_requests_enrollment_id` FOREIGN KEY (`enrollment_id`) REFERENCES `nano_enrollments` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_default_setup_assistants` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`profile_uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
`abm_token_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
UNIQUE KEY `idx_mdm_default_setup_assistant_global_or_team_id_abm_token_id` (`global_or_team_id`,`abm_token_id`),
KEY `fk_mdm_default_setup_assistant_team_id` (`team_id`),
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
KEY `fk_mdm_default_setup_assistant_abm_token_id` (`abm_token_id`),
CONSTRAINT `fk_mdm_default_setup_assistant_abm_token_id` FOREIGN KEY (`abm_token_id`) REFERENCES `abm_tokens` (`id`) ON DELETE CASCADE,
CONSTRAINT `mdm_apple_default_setup_assistants_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `mdm_apple_enrollment_profiles` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`type` varchar(10) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'automatic',
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`dep_profile` json DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_type` (`type`),
UNIQUE KEY `idx_token` (`token`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `mdm_apple_installers` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`size` bigint NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`manifest` text COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`installer` longblob,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`url_token` varchar(36) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
CREATE TABLE `mdm_apple_setup_assistant_profiles` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`setup_assistant_id` int unsigned NOT NULL,
`abm_token_id` int unsigned NOT NULL,
`profile_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_apple_setup_assistant_profiles_asst_id_tok_id` (`setup_assistant_id`,`abm_token_id`),
KEY `fk_mdm_apple_setup_assistant_profiles_abm_token_id` (`abm_token_id`),
CONSTRAINT `fk_mdm_apple_setup_assistant_profiles_abm_token_id` FOREIGN KEY (`abm_token_id`) REFERENCES `abm_tokens` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_mdm_apple_setup_assistant_profiles_setup_assistant_id` FOREIGN KEY (`setup_assistant_id`) REFERENCES `mdm_apple_setup_assistants` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_apple_setup_assistants` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`name` text COLLATE utf8mb4_unicode_ci NOT NULL,
`profile` json NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_setup_assistant_global_or_team_id` (`global_or_team_id`),
KEY `fk_mdm_setup_assistant_team_id` (`team_id`),
CONSTRAINT `mdm_apple_setup_assistants_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_config_assets` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(256) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`value` longblob NOT NULL,
`deleted_at` timestamp NULL DEFAULT NULL,
`deletion_uuid` varchar(127) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`md5_checksum` binary(16) NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_config_assets_name_deletion_uuid` (`name`,`deletion_uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_configuration_profile_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`apple_profile_uuid` varchar(37) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`windows_profile_uuid` varchar(37) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`label_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`label_id` int unsigned DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`require_all` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_configuration_profile_labels_apple_label_name` (`apple_profile_uuid`,`label_name`),
UNIQUE KEY `idx_mdm_configuration_profile_labels_windows_label_name` (`windows_profile_uuid`,`label_name`),
KEY `label_id` (`label_id`),
CONSTRAINT `mdm_configuration_profile_labels_ibfk_1` FOREIGN KEY (`apple_profile_uuid`) REFERENCES `mdm_apple_configuration_profiles` (`profile_uuid`) ON DELETE CASCADE,
CONSTRAINT `mdm_configuration_profile_labels_ibfk_2` FOREIGN KEY (`windows_profile_uuid`) REFERENCES `mdm_windows_configuration_profiles` (`profile_uuid`) ON DELETE CASCADE,
CONSTRAINT `mdm_configuration_profile_labels_ibfk_3` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE SET NULL,
CONSTRAINT `ck_mdm_configuration_profile_labels_apple_or_windows` CHECK (((`apple_profile_uuid` is null) <> (`windows_profile_uuid` is null)))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-03-14 21:08:19 +00:00
CREATE TABLE `mdm_declaration_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`apple_declaration_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`label_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`label_id` int unsigned DEFAULT NULL,
2024-03-14 21:08:19 +00:00
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`uploaded_at` timestamp NULL DEFAULT NULL,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`require_all` tinyint(1) NOT NULL DEFAULT '0',
2024-03-14 21:08:19 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mdm_declaration_labels_label_name` (`apple_declaration_uuid`,`label_name`),
2024-03-14 21:08:19 +00:00
KEY `label_id` (`label_id`),
CONSTRAINT `mdm_declaration_labels_ibfk_1` FOREIGN KEY (`apple_declaration_uuid`) REFERENCES `mdm_apple_declarations` (`declaration_uuid`) ON DELETE CASCADE,
2024-03-14 21:08:19 +00:00
CONSTRAINT `mdm_declaration_labels_ibfk_3` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2024-03-14 21:08:19 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_delivery_status` (
`status` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`status`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `mdm_delivery_status` VALUES ('failed'),('pending'),('verified'),('verifying');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_idp_accounts` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`uuid` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`username` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`fullname` varchar(256) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`email` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`uuid`),
UNIQUE KEY `unique_idp_email` (`email`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_operation_types` (
`operation_type` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`operation_type`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `mdm_operation_types` VALUES ('install'),('remove');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_windows_configuration_profiles` (
`team_id` int unsigned NOT NULL DEFAULT '0',
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`syncml` mediumblob NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`uploaded_at` timestamp(6) NULL DEFAULT NULL,
`profile_uuid` varchar(37) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`auto_increment` bigint NOT NULL AUTO_INCREMENT,
`checksum` binary(16) GENERATED ALWAYS AS (unhex(md5(`syncml`))) STORED,
`secrets_updated_at` datetime(6) DEFAULT NULL,
PRIMARY KEY (`profile_uuid`),
UNIQUE KEY `idx_mdm_windows_configuration_profiles_team_id_name` (`team_id`,`name`),
UNIQUE KEY `auto_increment` (`auto_increment`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mdm_windows_enrollments` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`mdm_device_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`mdm_hardware_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_state` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_user_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_proto_version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`enroll_client_version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`not_in_oobe` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_type` (`mdm_hardware_id`),
KEY `idx_mdm_windows_enrollments_mdm_device_id` (`mdm_device_id`),
KEY `idx_mdm_windows_enrollments_host_uuid` (`host_uuid`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `migration_status_tables` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`version_id` bigint NOT NULL,
`is_applied` tinyint(1) NOT NULL,
`tstamp` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
Add author ID to labels (#27055) For #27035 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality ## Details This PR adds an `author_id` column to the `labels` table, and adds the associated properties to the `Label` and `LabelSpec` types. When a new label is created via the UI or API, an author ID is set on the label if one can be inferred from the context. Otherwise, the author ID is set to `null`. ## Authz and Automated testing Additional backend authorization logic is introduced in a follow-on PR, https://github.com/fleetdm/fleet/pull/27089, because rconciling all of the test updates between this PR and https://github.com/fleetdm/fleet/pull/27038 was getting complicated. ## Manual Testing * Tested in the UI by creating a new label on the Hosts page * Tested via Gitops by merging this branch with https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops` with a global config with `labels:` in it.
2025-03-20 21:05:16 +00:00
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=368 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
Add author ID to labels (#27055) For #27035 # Checklist for submitter - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [ ] Added/updated automated tests - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality ## Details This PR adds an `author_id` column to the `labels` table, and adds the associated properties to the `Label` and `LabelSpec` types. When a new label is created via the UI or API, an author ID is set on the label if one can be inferred from the context. Otherwise, the author ID is set to `null`. ## Authz and Automated testing Additional backend authorization logic is introduced in a follow-on PR, https://github.com/fleetdm/fleet/pull/27089, because rconciling all of the test updates between this PR and https://github.com/fleetdm/fleet/pull/27038 was getting complicated. ## Manual Testing * Tested in the UI by creating a new label on the Hosts page * Tested via Gitops by merging this branch with https://github.com/fleetdm/fleet/pull/27038 and doing `fleetctl gitops` with a global config with `labels:` in it.
2025-03-20 21:05:16 +00:00
INSERT INTO `migration_status_tables` VALUES (1,0,1,'2020-01-01 01:01:01'),(2,20161118193812,1,'2020-01-01 01:01:01'),(3,20161118211713,1,'2020-01-01 01:01:01'),(4,20161118212436,1,'2020-01-01 01:01:01'),(5,20161118212515,1,'2020-01-01 01:01:01'),(6,20161118212528,1,'2020-01-01 01:01:01'),(7,20161118212538,1,'2020-01-01 01:01:01'),(8,20161118212549,1,'2020-01-01 01:01:01'),(9,20161118212557,1,'2020-01-01 01:01:01'),(10,20161118212604,1,'2020-01-01 01:01:01'),(11,20161118212613,1,'2020-01-01 01:01:01'),(12,20161118212621,1,'2020-01-01 01:01:01'),(13,20161118212630,1,'2020-01-01 01:01:01'),(14,20161118212641,1,'2020-01-01 01:01:01'),(15,20161118212649,1,'2020-01-01 01:01:01'),(16,20161118212656,1,'2020-01-01 01:01:01'),(17,20161118212758,1,'2020-01-01 01:01:01'),(18,20161128234849,1,'2020-01-01 01:01:01'),(19,20161230162221,1,'2020-01-01 01:01:01'),(20,20170104113816,1,'2020-01-01 01:01:01'),(21,20170105151732,1,'2020-01-01 01:01:01'),(22,20170108191242,1,'2020-01-01 01:01:01'),(23,20170109094020,1,'2020-01-01 01:01:01'),(24,20170109130438,1,'2020-01-01 01:01:01'),(25,20170110202752,1,'2020-01-01 01:01:01'),(26,20170111133013,1,'2020-01-01 01:01:01'),(27,20170117025759,1,'2020-01-01 01:01:01'),(28,20170118191001,1,'2020-01-01 01:01:01'),(29,20170119234632,1,'2020-01-01 01:01:01'),(30,20170124230432,1,'2020-01-01 01:01:01'),(31,20170127014618,1,'2020-01-01 01:01:01'),(32,20170131232841,1,'2020-01-01 01:01:01'),(33,20170223094154,1,'2020-01-01 01:01:01'),(34,20170306075207,1,'2020-01-01 01:01:01'),(35,20170309100733,1,'2020-01-01 01:01:01'),(36,20170331111922,1,'2020-01-01 01:01:01'),(37,20170502143928,1,'2020-01-01 01:01:01'),(38,20170504130602,1,'2020-01-01 01:01:01'),(39,20170509132100,1,'2020-01-01 01:01:01'),(40,20170519105647,1,'2020-01-01 01:01:01'),(41,20170519105648,1,'2020-01-01 01:01:01'),(42,20170831234300,1,'2020-01-01 01:01:01'),(43,20170831234301,1,'2020-01-01 01:01:01'),(44,20170831234303,1,'2020-01-01 01:01:01'),(45,20171116163618,1,'2020-01-01 01:01:01'),(46,20171219164727,1,'2020-01-01 01:01:01'),(47,20180620164811,1,'2020-01-01 01:01:01'),(48,20180620175054,1,'2020-01-01 01:01:01'),(49,20180620175055,1,'2020-01-01 01:01:01'),(50,20191010101639,1,'2020-01-01 01:01:01'),(51,20191010155147,1,'2020-01-01 01:01:01'),(52,20191220130734,1,'2020-01-01 01:01:01'),(53,20200311140000,1,'2020-01-01 01:01:01'),(54,20200405120000,1,'2020-01-01 01:01:01'),(55,20200407120000,1,'2020-01-01 01:01:01'),(56,20200420120000,1,'2020-01-01 01:01:01'),(57,20200504120000,1,'2020-01-01 01:01:01'),(58,20200512120000,1,'2020-01-01 01:01:01'),(59,20200707120000,1,'2020-01-01 01:01:01'),(60,20201011162341,1,'2020-01-01 01:01:01'),(61,20201021104586,1,'2020-01-01 01:01:01'),(62,20201102112520,1,'2020-01-01 01:01:01'),(63,20201208121729,1,'2020-01-01 01:01:01'),(64,20201215091637,1,'2020-01-01 01:01:01'),(65,20210119174155,1,'2020-01-01 01:01:01'),(66,20210326182902,1,'2020-01-01 01:01:01'),(67,20210421112652,1,'2020-01-01 01:01:01'),(68,20210506095025,1,'2020-01-01 01:01:01'),(69,20210513115729,1,'2020-01-01 01:01:01'),(70,20210526113559,1,'2020-01-01 01:01:01'),(71,20210601000001,1,'2020-01-01 01:01:01'),(72,20210601000002,1,'2020-01-01 01:01:01'),(73,20210601000003,1,'2020-01-01 01:01:01'),(74,20210601000004,1,'2020-01-01 01:01:01'),(75,20210601000005,1,'2020-01-01 01:01:01'),(76,20210601000006,1,'2020-01-01 01:01:01'),(77,20210601000007,1,'2020-01-01 01:01:01'),(78,20210601000008,1,'2020-01-01 01:01:01'),(79,20210606151329,1,'2020-01-01 01:01:01'),(80,20210616163757,1,'2020-01-01 01:01:01'),(81,20210617174723,1,'2020-01-01 01:01:01'),(82,20210622160235,1,'2020-01-01 01:01:01'),(83,20210623100031,1,'2020-01-01 01:01:01'),(84,20210623133615,1,'2020-01-01 01:01:01'),(85,20210708143152,1,'2020-01-01 01:01:01'),(86,20210709124443,1,'2020-01-01 01:01:01'),(87,20210712155608,1,'2020-01-01 01:01:01'),(88,20210714102108,1,'2020-01-01 01:01:01'),(89,20210719153709,1,'2020-01-01 01:01:01'),(90,20210721171531,1,'2020-01-01 01:01:01'),(91,20210723135713,1,'2020-01-01 01:01:01'),(92,20210802135933,1,'2020-01-01 01:01:01'),(93,20210806112844,1,'20
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `mobile_device_management_solutions` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(100) COLLATE utf8mb4_unicode_ci NOT NULL,
`server_url` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_mobile_device_management_solutions_name` (`name`,`server_url`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `munki_issues` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`issue_type` varchar(10) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_munki_issues_name` (`name`,`issue_type`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_cert_auth_associations` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sha256` char(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`cert_not_valid_after` timestamp NULL DEFAULT NULL,
`renew_command_uuid` varchar(127) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`,`sha256`),
KEY `renew_command_uuid_fk` (`renew_command_uuid`),
CONSTRAINT `renew_command_uuid_fk` FOREIGN KEY (`renew_command_uuid`) REFERENCES `nano_commands` (`command_uuid`),
CONSTRAINT `nano_cert_auth_associations_chk_1` CHECK ((`id` <> _utf8mb4'')),
CONSTRAINT `nano_cert_auth_associations_chk_2` CHECK ((`sha256` <> _utf8mb4''))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_command_results` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`status` varchar(31) COLLATE utf8mb4_unicode_ci NOT NULL,
`result` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`not_now_at` timestamp NULL DEFAULT NULL,
`not_now_tally` int NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`,`command_uuid`),
KEY `command_uuid` (`command_uuid`),
KEY `status` (`status`),
CONSTRAINT `nano_command_results_ibfk_1` FOREIGN KEY (`id`) REFERENCES `nano_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_command_results_ibfk_2` FOREIGN KEY (`command_uuid`) REFERENCES `nano_commands` (`command_uuid`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_command_results_chk_1` CHECK ((`status` <> _utf8mb4'')),
CONSTRAINT `nano_command_results_chk_2` CHECK ((substr(`result`,1,5) = _utf8mb4'<?xml'))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_commands` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`request_type` varchar(63) COLLATE utf8mb4_unicode_ci NOT NULL,
`command` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`subtype` enum('None','ProfileWithSecrets') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'None',
PRIMARY KEY (`command_uuid`),
CONSTRAINT `nano_commands_chk_1` CHECK ((`command_uuid` <> _utf8mb4'')),
CONSTRAINT `nano_commands_chk_2` CHECK ((`request_type` <> _utf8mb4''))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_dep_names` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`consumer_key` text COLLATE utf8mb4_unicode_ci,
`consumer_secret` text COLLATE utf8mb4_unicode_ci,
`access_token` text COLLATE utf8mb4_unicode_ci,
`access_secret` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`access_token_expiry` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`config_base_url` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`tokenpki_cert_pem` text COLLATE utf8mb4_unicode_ci,
`tokenpki_key_pem` text COLLATE utf8mb4_unicode_ci,
`syncer_cursor` varchar(1024) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`syncer_cursor_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`assigner_profile_uuid` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`assigner_profile_uuid_at` timestamp NULL DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`name`),
CONSTRAINT `nano_dep_names_chk_1` CHECK (((`tokenpki_cert_pem` is null) or (substr(`tokenpki_cert_pem`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----'))),
CONSTRAINT `nano_dep_names_chk_2` CHECK (((`tokenpki_key_pem` is null) or (substr(`tokenpki_key_pem`,1,5) = _utf8mb4'-----')))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_devices` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`identity_cert` text COLLATE utf8mb4_unicode_ci,
`serial_number` varchar(127) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`unlock_token` mediumblob,
`unlock_token_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`authenticate` text COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`authenticate_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token_update` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`token_update_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`bootstrap_token_b64` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`bootstrap_token_at` timestamp NULL DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`),
KEY `serial_number` (`serial_number`),
CONSTRAINT `nano_devices_chk_1` CHECK (((`identity_cert` is null) or (substr(`identity_cert`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----'))),
CONSTRAINT `nano_devices_chk_2` CHECK (((`serial_number` is null) or (`serial_number` <> _utf8mb4''))),
CONSTRAINT `nano_devices_chk_3` CHECK (((`unlock_token` is null) or (length(`unlock_token`) > 0))),
CONSTRAINT `nano_devices_chk_4` CHECK ((`authenticate` <> _utf8mb4'')),
CONSTRAINT `nano_devices_chk_5` CHECK (((`token_update` is null) or (`token_update` <> _utf8mb4''))),
CONSTRAINT `nano_devices_chk_6` CHECK (((`bootstrap_token_b64` is null) or (`bootstrap_token_b64` <> _utf8mb4'')))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_enrollment_queue` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`active` tinyint(1) NOT NULL DEFAULT '1',
`priority` tinyint NOT NULL DEFAULT '0',
`created_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`,`command_uuid`),
KEY `command_uuid` (`command_uuid`),
KEY `priority` (`priority` DESC,`created_at`),
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CONSTRAINT `nano_enrollment_queue_ibfk_1` FOREIGN KEY (`id`) REFERENCES `nano_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_enrollment_queue_ibfk_2` FOREIGN KEY (`command_uuid`) REFERENCES `nano_commands` (`command_uuid`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_enrollments` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`user_id` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`type` varchar(31) COLLATE utf8mb4_unicode_ci NOT NULL,
`topic` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`push_magic` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`token_hex` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`enabled` tinyint(1) NOT NULL DEFAULT '1',
`token_update_tally` int NOT NULL DEFAULT '1',
`last_seen_at` timestamp NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`enrolled_from_migration` tinyint unsigned NOT NULL DEFAULT '0',
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `user_id` (`user_id`),
KEY `device_id` (`device_id`),
KEY `type` (`type`),
CONSTRAINT `nano_enrollments_ibfk_1` FOREIGN KEY (`device_id`) REFERENCES `nano_devices` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_enrollments_ibfk_2` FOREIGN KEY (`user_id`) REFERENCES `nano_users` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_enrollments_chk_1` CHECK ((`id` <> _utf8mb4'')),
CONSTRAINT `nano_enrollments_chk_2` CHECK ((`type` <> _utf8mb4'')),
CONSTRAINT `nano_enrollments_chk_3` CHECK ((`topic` <> _utf8mb4'')),
CONSTRAINT `nano_enrollments_chk_4` CHECK ((`push_magic` <> _utf8mb4'')),
CONSTRAINT `nano_enrollments_chk_5` CHECK ((`token_hex` <> _utf8mb4''))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_push_certs` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`topic` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`cert_pem` text COLLATE utf8mb4_unicode_ci NOT NULL,
`key_pem` text COLLATE utf8mb4_unicode_ci NOT NULL,
`stale_token` int NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`topic`),
CONSTRAINT `nano_push_certs_chk_1` CHECK ((`topic` <> _utf8mb4'')),
CONSTRAINT `nano_push_certs_chk_2` CHECK ((substr(`cert_pem`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----')),
CONSTRAINT `nano_push_certs_chk_3` CHECK ((substr(`key_pem`,1,5) = _utf8mb4'-----'))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `nano_users` (
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`device_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`user_short_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`user_long_name` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`token_update` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`token_update_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`user_authenticate` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`user_authenticate_at` timestamp NULL DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`user_authenticate_digest` text COLLATE utf8mb4_unicode_ci,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`user_authenticate_digest_at` timestamp NULL DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`id`,`device_id`),
UNIQUE KEY `idx_unique_id` (`id`),
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
KEY `device_id` (`device_id`),
CONSTRAINT `nano_users_ibfk_1` FOREIGN KEY (`device_id`) REFERENCES `nano_devices` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `nano_users_chk_1` CHECK (((`user_short_name` is null) or (`user_short_name` <> _utf8mb4''))),
CONSTRAINT `nano_users_chk_2` CHECK (((`user_long_name` is null) or (`user_long_name` <> _utf8mb4''))),
CONSTRAINT `nano_users_chk_3` CHECK (((`token_update` is null) or (`token_update` <> _utf8mb4''))),
CONSTRAINT `nano_users_chk_4` CHECK (((`user_authenticate` is null) or (`user_authenticate` <> _utf8mb4''))),
CONSTRAINT `nano_users_chk_5` CHECK (((`user_authenticate_digest` is null) or (`user_authenticate_digest` <> _utf8mb4'')))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
SET @saved_cs_client = @@character_set_client;
/*!50503 SET character_set_client = utf8mb4 */;
/*!50001 CREATE VIEW `nano_view_queue` AS SELECT
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
1 AS `id`,
1 AS `created_at`,
1 AS `active`,
1 AS `priority`,
1 AS `command_uuid`,
1 AS `request_type`,
1 AS `command`,
1 AS `result_updated_at`,
1 AS `status`,
1 AS `result`*/;
SET character_set_client = @saved_cs_client;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `network_interfaces` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`mac` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`ip_address` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`broadcast` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`ibytes` bigint NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`interface` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`ipackets` bigint NOT NULL DEFAULT '0',
`last_change` bigint NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`mask` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`metric` int NOT NULL DEFAULT '0',
`mtu` int NOT NULL DEFAULT '0',
`obytes` bigint NOT NULL DEFAULT '0',
`ierrors` bigint NOT NULL DEFAULT '0',
`oerrors` bigint NOT NULL DEFAULT '0',
`opackets` bigint NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`point_to_point` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`type` int NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_network_interfaces_unique_ip_host_intf` (`ip_address`,`host_id`,`interface`),
KEY `idx_network_interfaces_hosts_fk` (`host_id`),
FULLTEXT KEY `ip_address_search` (`ip_address`),
CONSTRAINT `network_interfaces_ibfk_1` FOREIGN KEY (`host_id`) REFERENCES `hosts` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `operating_system_vulnerabilities` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`operating_system_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`cve` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`source` smallint DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`resolved_in_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_os_vulnerabilities_unq_os_id_cve` (`operating_system_id`,`cve`),
KEY `idx_os_vulnerabilities_cve` (`cve`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `operating_systems` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`version` varchar(150) COLLATE utf8mb4_unicode_ci NOT NULL,
`arch` varchar(150) COLLATE utf8mb4_unicode_ci NOT NULL,
`kernel_version` varchar(150) COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(50) COLLATE utf8mb4_unicode_ci NOT NULL,
`display_version` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`os_version_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_os` (`name`,`version`,`arch`,`kernel_version`,`platform`,`display_version`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `osquery_options` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`override_type` int NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`override_identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`options` json NOT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
INSERT INTO `osquery_options` VALUES (1,0,'','{\"options\": {\"logger_plugin\": \"tls\", \"pack_delimiter\": \"/\", \"logger_tls_period\": 10, \"distributed_plugin\": \"tls\", \"disable_distributed\": false, \"logger_tls_endpoint\": \"/api/v1/osquery/log\", \"distributed_interval\": 10, \"distributed_tls_max_attempts\": 3}, \"decorators\": {\"load\": [\"SELECT uuid AS host_uuid FROM system_info;\", \"SELECT hostname AS hostname FROM system_info;\"]}}');
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `pack_targets` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`pack_id` int unsigned DEFAULT NULL,
`type` int DEFAULT NULL,
`target_id` int unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `constraint_pack_target_unique` (`pack_id`,`target_id`,`type`),
CONSTRAINT `pack_targets_ibfk_1` FOREIGN KEY (`pack_id`) REFERENCES `packs` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `packs` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`disabled` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`platform` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`pack_type` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_pack_unique_name` (`name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `password_reset_requests` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`expires_at` timestamp NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`user_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`token` varchar(1024) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `policies` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`team_id` int unsigned DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`resolution` text COLLATE utf8mb4_unicode_ci,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`query` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`description` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`author_id` int unsigned DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`platforms` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`critical` tinyint(1) NOT NULL DEFAULT '0',
`checksum` binary(16) NOT NULL,
`calendar_events_enabled` tinyint unsigned NOT NULL DEFAULT '0',
`software_installer_id` int unsigned DEFAULT NULL,
`script_id` int unsigned DEFAULT NULL,
`vpp_apps_teams_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_policies_checksum` (`checksum`),
KEY `idx_policies_author_id` (`author_id`),
KEY `idx_policies_team_id` (`team_id`),
KEY `fk_policies_software_installer_id` (`software_installer_id`),
KEY `fk_policies_script_id` (`script_id`),
KEY `fk_policies_vpp_apps_team_id` (`vpp_apps_teams_id`),
CONSTRAINT `policies_ibfk_3` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`),
CONSTRAINT `policies_ibfk_4` FOREIGN KEY (`script_id`) REFERENCES `scripts` (`id`),
CONSTRAINT `policies_ibfk_5` FOREIGN KEY (`vpp_apps_teams_id`) REFERENCES `vpp_apps_teams` (`id`),
CONSTRAINT `policies_queries_ibfk_1` FOREIGN KEY (`author_id`) REFERENCES `users` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `policy_automation_iterations` (
`policy_id` int unsigned NOT NULL,
`iteration` int NOT NULL,
PRIMARY KEY (`policy_id`),
CONSTRAINT `policy_automation_iterations_ibfk_1` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `policy_membership` (
`policy_id` int unsigned NOT NULL,
`host_id` int unsigned NOT NULL,
`passes` tinyint(1) DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`automation_iteration` int DEFAULT NULL,
PRIMARY KEY (`policy_id`,`host_id`),
KEY `idx_policy_membership_passes` (`passes`),
KEY `idx_policy_membership_host_id_passes` (`host_id`,`passes`),
CONSTRAINT `policy_membership_ibfk_1` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2023-11-21 19:52:06 +00:00
CREATE TABLE `policy_stats` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`policy_id` int unsigned NOT NULL,
`inherited_team_id` int unsigned DEFAULT NULL,
`passing_host_count` mediumint unsigned NOT NULL DEFAULT '0',
`failing_host_count` mediumint unsigned NOT NULL DEFAULT '0',
2023-11-21 19:52:06 +00:00
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`inherited_team_id_char` char(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS (if((`inherited_team_id` is null),_utf8mb4'global',cast(`inherited_team_id` as char charset utf8mb4))) VIRTUAL,
2023-11-21 19:52:06 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `policy_id` (`policy_id`,`inherited_team_id_char`),
2023-11-21 19:52:06 +00:00
CONSTRAINT `policy_stats_ibfk_1` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2023-11-21 19:52:06 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `queries` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`saved` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`query` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`author_id` int unsigned DEFAULT NULL,
`observer_can_run` tinyint(1) NOT NULL DEFAULT '0',
`team_id` int unsigned DEFAULT NULL,
`team_id_char` char(10) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`platform` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`min_osquery_version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`schedule_interval` int unsigned NOT NULL DEFAULT '0',
`automations_enabled` tinyint unsigned NOT NULL DEFAULT '0',
`logging_type` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT 'snapshot',
`discard_data` tinyint(1) NOT NULL DEFAULT '1',
`is_scheduled` tinyint(1) GENERATED ALWAYS AS ((`schedule_interval` > 0)) STORED NOT NULL,
PRIMARY KEY (`id`),
2023-07-06 21:28:25 +00:00
UNIQUE KEY `idx_team_id_name_unq` (`team_id_char`,`name`),
UNIQUE KEY `idx_name_team_id_unq` (`name`,`team_id_char`),
KEY `author_id` (`author_id`),
KEY `idx_team_id_saved_auto_interval` (`team_id`,`saved`,`automations_enabled`,`schedule_interval`),
KEY `idx_queries_schedule_automations` (`is_scheduled`,`automations_enabled`),
2023-07-06 21:28:25 +00:00
CONSTRAINT `queries_ibfk_1` FOREIGN KEY (`author_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `queries_ibfk_2` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `query_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`query_id` int unsigned NOT NULL,
`label_id` int unsigned NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_query_labels_query_label` (`query_id`,`label_id`),
KEY `query_labels_label_id` (`label_id`),
CONSTRAINT `query_labels_label_id` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE CASCADE,
CONSTRAINT `query_labels_query_id` FOREIGN KEY (`query_id`) REFERENCES `queries` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2023-10-05 02:56:17 +00:00
CREATE TABLE `query_results` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`query_id` int unsigned NOT NULL,
`host_id` int unsigned NOT NULL,
2023-10-05 02:56:17 +00:00
`osquery_version` varchar(50) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`error` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`last_fetched` timestamp NOT NULL,
2023-10-05 02:56:17 +00:00
`data` json DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_query_id_host_id_last_fetched` (`query_id`,`host_id`,`last_fetched`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
2023-10-05 02:56:17 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `scep_certificates` (
`serial` bigint NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(1024) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`not_valid_before` datetime NOT NULL,
`not_valid_after` datetime NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`certificate_pem` text COLLATE utf8mb4_unicode_ci NOT NULL,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
`revoked` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`serial`),
CONSTRAINT `scep_certificates_ibfk_1` FOREIGN KEY (`serial`) REFERENCES `scep_serials` (`serial`),
CONSTRAINT `scep_certificates_chk_1` CHECK ((substr(`certificate_pem`,1,27) = _utf8mb4'-----BEGIN CERTIFICATE-----')),
CONSTRAINT `scep_certificates_chk_2` CHECK (((`name` is null) or (`name` <> _utf8mb4'')))
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
CREATE TABLE `scep_serials` (
`serial` bigint NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
PRIMARY KEY (`serial`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scheduled_queries` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`pack_id` int unsigned DEFAULT NULL,
`query_id` int unsigned DEFAULT NULL,
`interval` int unsigned DEFAULT NULL,
`snapshot` tinyint(1) DEFAULT NULL,
`removed` tinyint(1) DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`platform` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT '',
`version` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT '',
`shard` int unsigned DEFAULT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`query_name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` varchar(1023) COLLATE utf8mb4_unicode_ci DEFAULT '',
`denylist` tinyint(1) DEFAULT NULL,
2023-07-10 18:56:44 +00:00
`team_id_char` char(10) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
UNIQUE KEY `unique_names_in_packs` (`name`,`pack_id`),
KEY `scheduled_queries_pack_id` (`pack_id`),
KEY `scheduled_queries_query_name` (`query_name`),
KEY `fk_scheduled_queries_queries` (`team_id_char`,`query_name`),
2023-07-10 20:01:46 +00:00
CONSTRAINT `scheduled_queries_ibfk_1` FOREIGN KEY (`team_id_char`, `query_name`) REFERENCES `queries` (`team_id_char`, `name`) ON DELETE CASCADE ON UPDATE CASCADE,
2023-07-06 21:28:25 +00:00
CONSTRAINT `scheduled_queries_pack_id` FOREIGN KEY (`pack_id`) REFERENCES `packs` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scheduled_query_stats` (
`host_id` int unsigned NOT NULL,
`scheduled_query_id` int unsigned NOT NULL,
`average_memory` bigint unsigned NOT NULL,
`denylisted` tinyint(1) DEFAULT NULL,
`executions` bigint unsigned NOT NULL,
`schedule_interval` int DEFAULT NULL,
`last_executed` timestamp NULL DEFAULT NULL,
`output_size` bigint unsigned NOT NULL,
`system_time` bigint unsigned NOT NULL,
`user_time` bigint unsigned NOT NULL,
`wall_time` bigint unsigned NOT NULL,
`query_type` tinyint NOT NULL DEFAULT '0',
PRIMARY KEY (`host_id`,`scheduled_query_id`,`query_type`),
KEY `scheduled_query_id` (`scheduled_query_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `script_contents` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`md5_checksum` binary(16) NOT NULL,
`contents` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_script_contents_md5_checksum` (`md5_checksum`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `script_upcoming_activities` (
`upcoming_activity_id` bigint unsigned NOT NULL,
`script_id` int unsigned DEFAULT NULL,
`script_content_id` int unsigned DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
`setup_experience_script_id` int unsigned DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`upcoming_activity_id`),
KEY `fk_script_upcoming_activities_script_id` (`script_id`),
KEY `fk_script_upcoming_activities_script_content_id` (`script_content_id`),
KEY `fk_script_upcoming_activities_policy_id` (`policy_id`),
KEY `fk_script_upcoming_activities_setup_experience_script_id` (`setup_experience_script_id`),
CONSTRAINT `fk_script_upcoming_activities_policy_id` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_script_upcoming_activities_script_content_id` FOREIGN KEY (`script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_script_upcoming_activities_script_id` FOREIGN KEY (`script_id`) REFERENCES `scripts` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_script_upcoming_activities_setup_experience_script_id` FOREIGN KEY (`setup_experience_script_id`) REFERENCES `setup_experience_scripts` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_script_upcoming_activities_upcoming_activity_id` FOREIGN KEY (`upcoming_activity_id`) REFERENCES `upcoming_activities` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `scripts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`script_content_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_scripts_global_or_team_id_name` (`global_or_team_id`,`name`),
2023-10-10 22:00:45 +00:00
UNIQUE KEY `idx_scripts_team_name` (`team_id`,`name`),
KEY `script_content_id` (`script_content_id`),
CONSTRAINT `scripts_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `scripts_ibfk_2` FOREIGN KEY (`script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `secret_variables` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`value` blob NOT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
2024-12-10 23:17:21 +00:00
UNIQUE KEY `idx_secret_variables_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `sessions` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`accessed_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`user_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`key` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_session_unique_key` (`key`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `setup_experience_scripts` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`script_content_id` int unsigned DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_setup_experience_scripts_global_or_team_id` (`global_or_team_id`),
KEY `idx_script_content_id` (`script_content_id`),
KEY `fk_setup_experience_scripts_ibfk_1` (`team_id`),
CONSTRAINT `fk_setup_experience_scripts_ibfk_1` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_setup_experience_scripts_ibfk_2` FOREIGN KEY (`script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `setup_experience_status_results` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`status` enum('pending','running','success','failure') COLLATE utf8mb4_unicode_ci NOT NULL,
`software_installer_id` int unsigned DEFAULT NULL,
`host_software_installs_execution_id` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`vpp_app_team_id` int unsigned DEFAULT NULL,
`nano_command_uuid` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`setup_experience_script_id` int unsigned DEFAULT NULL,
`script_execution_id` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`error` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_setup_experience_scripts_host_uuid` (`host_uuid`),
KEY `idx_setup_experience_scripts_hsi_id` (`host_software_installs_execution_id`),
KEY `idx_setup_experience_scripts_nano_command_uuid` (`nano_command_uuid`),
KEY `idx_setup_experience_scripts_script_execution_id` (`script_execution_id`),
KEY `fk_setup_experience_status_results_si_id` (`software_installer_id`),
KEY `fk_setup_experience_status_results_va_id` (`vpp_app_team_id`),
KEY `fk_setup_experience_status_results_ses_id` (`setup_experience_script_id`),
CONSTRAINT `fk_setup_experience_status_results_ses_id` FOREIGN KEY (`setup_experience_script_id`) REFERENCES `setup_experience_scripts` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_setup_experience_status_results_si_id` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_setup_experience_status_results_va_id` FOREIGN KEY (`vpp_app_team_id`) REFERENCES `vpp_apps_teams` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`source` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`bundle_identifier` varchar(255) COLLATE utf8mb4_unicode_ci DEFAULT '',
`release` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`vendor_old` varchar(32) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`arch` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`vendor` varchar(114) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`browser` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`extension_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`title_id` int unsigned DEFAULT NULL,
`checksum` binary(16) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_checksum` (`checksum`),
KEY `software_source_vendor_idx` (`source`,`vendor_old`),
KEY `title_id` (`title_id`),
KEY `idx_sw_name_source_browser` (`name`,`source`,`browser`),
KEY `software_listing_idx` (`name`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_cpe` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`software_id` bigint unsigned DEFAULT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`cpe` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `unq_software_id` (`software_id`),
KEY `software_cpe_cpe_idx` (`cpe`),
CONSTRAINT `software_cpe_ibfk_1` FOREIGN KEY (`software_id`) REFERENCES `software` (`id`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_cve` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`cve` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`source` int DEFAULT '0',
`software_id` bigint unsigned DEFAULT NULL,
`resolved_in_version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
fix: add an index to mitigate vuln processing resource spikes (#26331) > For #26178 # Checklist for submitter ## Data from testing We tested the changes out by directly creating the index in our dogfood env. We saw a decrease in resource usage, captured in these screenshots: DB load (taller spike towards the left is without the index, subsequent spikes are after the index was added) ![image (1)](https://github.com/user-attachments/assets/141f8066-89e9-4a8e-ba7c-9fd3a09afea4) DB load over a longer period (overnight 2025-02-12 -> 2025-02-13) ![Screenshot 2025-02-13 at 9 05 05 AM](https://github.com/user-attachments/assets/bb0c6744-537d-4aec-960b-d100c4285d00) CPU utilization ![Screenshot 2025-02-13 at 10 25 17 AM](https://github.com/user-attachments/assets/eeea9ae5-5a10-4d50-91bc-3a806b359b39) Memory usage ![Screenshot 2025-02-13 at 10 26 52 AM](https://github.com/user-attachments/assets/bb77cdd8-41ef-4d90-a707-70b6d2976a59) --- If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] If database migrations are included, checked table schema to confirm autoupdate - [x] A detailed QA plan exists on the associated ticket (if it isn't there, work with the product group's QA engineer to add it) - [x] Manual QA for all new/changed functionality
2025-02-14 17:53:41 +00:00
UNIQUE KEY `unq_software_id_cve` (`software_id`,`cve`),
KEY `idx_software_cve_cve` (`cve`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_host_counts` (
`software_id` bigint unsigned NOT NULL,
`hosts_count` int unsigned NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`team_id` int unsigned NOT NULL DEFAULT '0',
`global_stats` tinyint unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`software_id`,`team_id`,`global_stats`),
KEY `idx_software_host_counts_updated_at_software_id` (`updated_at`,`software_id`),
KEY `idx_software_host_counts_team_id_hosts_count_software_id` (`team_id`,`hosts_count`,`software_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_install_upcoming_activities` (
`upcoming_activity_id` bigint unsigned NOT NULL,
`software_installer_id` int unsigned DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
`software_title_id` int unsigned DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`upcoming_activity_id`),
KEY `fk_software_install_upcoming_activities_software_installer_id` (`software_installer_id`),
KEY `fk_software_install_upcoming_activities_policy_id` (`policy_id`),
KEY `fk_software_install_upcoming_activities_software_title_id` (`software_title_id`),
CONSTRAINT `fk_software_install_upcoming_activities_policy_id` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_software_install_upcoming_activities_software_installer_id` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_software_install_upcoming_activities_software_title_id` FOREIGN KEY (`software_title_id`) REFERENCES `software_titles` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_software_install_upcoming_activities_upcoming_activity_id` FOREIGN KEY (`upcoming_activity_id`) REFERENCES `upcoming_activities` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-12-10 16:40:46 +00:00
CREATE TABLE `software_installer_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`software_installer_id` int unsigned NOT NULL,
`label_id` int unsigned NOT NULL,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
2024-12-10 16:40:46 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_installer_labels_software_installer_id_label_id` (`software_installer_id`,`label_id`),
KEY `label_id` (`label_id`),
CONSTRAINT `software_installer_labels_ibfk_1` FOREIGN KEY (`software_installer_id`) REFERENCES `software_installers` (`id`) ON DELETE CASCADE,
CONSTRAINT `software_installer_labels_ibfk_2` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE RESTRICT
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_installers` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int unsigned NOT NULL DEFAULT '0',
`title_id` int unsigned DEFAULT NULL,
`filename` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`version` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`pre_install_query` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
`install_script_content_id` int unsigned NOT NULL,
`post_install_script_content_id` int unsigned DEFAULT NULL,
`storage_id` varchar(64) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
2024-09-08 17:26:26 +00:00
`uploaded_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`self_service` tinyint(1) NOT NULL DEFAULT '0',
`user_id` int unsigned DEFAULT NULL,
`user_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`user_email` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`url` varchar(4095) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
2024-09-08 17:26:26 +00:00
`package_ids` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`extension` varchar(32) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
2024-09-08 17:26:26 +00:00
`uninstall_script_content_id` int unsigned NOT NULL,
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
`fleet_library_app_id` int unsigned DEFAULT NULL,
2024-10-25 22:11:56 +00:00
`install_during_setup` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_installers_team_id_title_id` (`global_or_team_id`,`title_id`),
KEY `fk_software_installers_title` (`title_id`),
KEY `fk_software_installers_install_script_content_id` (`install_script_content_id`),
KEY `fk_software_installers_post_install_script_content_id` (`post_install_script_content_id`),
KEY `fk_software_installers_team_id` (`team_id`),
KEY `idx_software_installers_platform_title_id` (`platform`,`title_id`),
KEY `fk_software_installers_user_id` (`user_id`),
2024-09-08 17:26:26 +00:00
KEY `fk_uninstall_script_content_id` (`uninstall_script_content_id`),
KEY `fk_software_installers_fleet_library_app_id` (`fleet_library_app_id`),
CONSTRAINT `fk_software_installers_install_script_content_id` FOREIGN KEY (`install_script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE RESTRICT ON UPDATE CASCADE,
CONSTRAINT `fk_software_installers_post_install_script_content_id` FOREIGN KEY (`post_install_script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE RESTRICT ON UPDATE CASCADE,
CONSTRAINT `fk_software_installers_team_id` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `fk_software_installers_title` FOREIGN KEY (`title_id`) REFERENCES `software_titles` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
2024-09-08 17:26:26 +00:00
CONSTRAINT `fk_software_installers_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_uninstall_script_content_id` FOREIGN KEY (`uninstall_script_content_id`) REFERENCES `script_contents` (`id`) ON DELETE RESTRICT ON UPDATE CASCADE,
CONSTRAINT `software_installers_ibfk_1` FOREIGN KEY (`fleet_library_app_id`) REFERENCES `fleet_library_apps` (`id`) ON DELETE SET NULL
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_titles` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`source` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`browser` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`bundle_identifier` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`additional_identifier` tinyint unsigned GENERATED ALWAYS AS ((case when (`source` = _utf8mb4'ios_apps') then 1 when (`source` = _utf8mb4'ipados_apps') then 2 when (`bundle_identifier` is not null) then 0 else NULL end)) VIRTUAL,
`unique_identifier` varchar(255) COLLATE utf8mb4_unicode_ci GENERATED ALWAYS AS (coalesce(`bundle_identifier`,`name`)) VIRTUAL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_software_titles_bundle_identifier` (`bundle_identifier`,`additional_identifier`),
UNIQUE KEY `idx_unique_sw_titles` (`unique_identifier`,`source`,`browser`),
KEY `idx_sw_titles` (`name`,`source`,`browser`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `software_titles_host_counts` (
`software_title_id` int unsigned NOT NULL,
`hosts_count` int unsigned NOT NULL,
`team_id` int unsigned NOT NULL DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`global_stats` tinyint unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`software_title_id`,`team_id`,`global_stats`),
KEY `idx_software_titles_host_counts_team_counts_title` (`team_id`,`hosts_count`,`software_title_id`),
KEY `idx_software_titles_host_counts_updated_at_software_title_id` (`updated_at`,`software_title_id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `statistics` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`anonymous_identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `teams` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` varchar(1023) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`config` json DEFAULT NULL,
`name_bin` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin GENERATED ALWAYS AS (`name`) VIRTUAL,
`filename` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_teams_filename` (`filename`),
UNIQUE KEY `idx_name_bin` (`name_bin`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `upcoming_activities` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`priority` int NOT NULL DEFAULT '0',
`user_id` int unsigned DEFAULT NULL,
`fleet_initiated` tinyint(1) NOT NULL DEFAULT '0',
`activity_type` enum('script','software_install','software_uninstall','vpp_app_install') COLLATE utf8mb4_unicode_ci NOT NULL,
`execution_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`payload` json NOT NULL,
`activated_at` datetime(6) DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_upcoming_activities_execution_id` (`execution_id`),
KEY `idx_upcoming_activities_host_id_priority_created_at` (`host_id`,`priority`,`created_at`),
KEY `idx_upcoming_activities_host_id_activity_type` (`activity_type`,`host_id`),
KEY `fk_upcoming_activities_user_id` (`user_id`),
CONSTRAINT `fk_upcoming_activities_user_id` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE SET NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `user_teams` (
`user_id` int unsigned NOT NULL,
`team_id` int unsigned NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`role` varchar(64) COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`user_id`,`team_id`),
KEY `fk_user_teams_team_id` (`team_id`),
CONSTRAINT `user_teams_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `user_teams_ibfk_2` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `users` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`password` varbinary(255) NOT NULL,
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`salt` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`email` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`admin_forced_password_reset` tinyint(1) NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`gravatar_url` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`position` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`sso_enabled` tinyint NOT NULL DEFAULT '0',
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
`global_role` varchar(64) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`api_only` tinyint(1) NOT NULL DEFAULT '0',
`mfa_enabled` tinyint(1) NOT NULL DEFAULT '0',
`settings` json NOT NULL DEFAULT (json_object()),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_user_unique_email` (`email`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `users_deleted` (
`id` int unsigned NOT NULL,
`name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`email` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `verification_tokens` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`user_id` int unsigned NOT NULL,
`token` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `token` (`token`),
KEY `verification_tokens_users` (`user_id`),
CONSTRAINT `verification_tokens_users` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
2024-12-10 16:40:46 +00:00
CREATE TABLE `vpp_app_team_labels` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`vpp_app_team_id` int unsigned NOT NULL,
`label_id` int unsigned NOT NULL,
`exclude` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
2024-12-10 16:40:46 +00:00
PRIMARY KEY (`id`),
UNIQUE KEY `idx_vpp_app_team_labels_vpp_app_team_id_label_id` (`vpp_app_team_id`,`label_id`),
KEY `label_id` (`label_id`),
CONSTRAINT `vpp_app_team_labels_ibfk_1` FOREIGN KEY (`vpp_app_team_id`) REFERENCES `vpp_apps_teams` (`id`) ON DELETE CASCADE,
CONSTRAINT `vpp_app_team_labels_ibfk_2` FOREIGN KEY (`label_id`) REFERENCES `labels` (`id`) ON DELETE RESTRICT
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_app_upcoming_activities` (
`upcoming_activity_id` bigint unsigned NOT NULL,
`adam_id` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL,
`platform` varchar(10) COLLATE utf8mb4_unicode_ci NOT NULL,
`vpp_token_id` int unsigned DEFAULT NULL,
`policy_id` int unsigned DEFAULT NULL,
`created_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` datetime(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`upcoming_activity_id`),
KEY `fk_vpp_app_upcoming_activities_adam_id_platform` (`adam_id`,`platform`),
KEY `fk_vpp_app_upcoming_activities_vpp_token_id` (`vpp_token_id`),
KEY `fk_vpp_app_upcoming_activities_policy_id` (`policy_id`),
CONSTRAINT `fk_vpp_app_upcoming_activities_adam_id_platform` FOREIGN KEY (`adam_id`, `platform`) REFERENCES `vpp_apps` (`adam_id`, `platform`) ON DELETE CASCADE,
CONSTRAINT `fk_vpp_app_upcoming_activities_policy_id` FOREIGN KEY (`policy_id`) REFERENCES `policies` (`id`) ON DELETE SET NULL,
CONSTRAINT `fk_vpp_app_upcoming_activities_upcoming_activity_id` FOREIGN KEY (`upcoming_activity_id`) REFERENCES `upcoming_activities` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_vpp_app_upcoming_activities_vpp_token_id` FOREIGN KEY (`vpp_token_id`) REFERENCES `vpp_tokens` (`id`) ON DELETE SET NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_apps` (
`adam_id` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL,
`title_id` int unsigned DEFAULT NULL,
`bundle_identifier` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`icon_url` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`latest_version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`platform` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`adam_id`,`platform`),
KEY `fk_vpp_apps_title` (`title_id`),
CONSTRAINT `fk_vpp_apps_title` FOREIGN KEY (`title_id`) REFERENCES `software_titles` (`id`) ON DELETE SET NULL ON UPDATE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_apps_teams` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`adam_id` varchar(16) COLLATE utf8mb4_unicode_ci NOT NULL,
`team_id` int unsigned DEFAULT NULL,
`global_or_team_id` int NOT NULL DEFAULT '0',
`platform` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`self_service` tinyint(1) NOT NULL DEFAULT '0',
`vpp_token_id` int unsigned NOT NULL,
`install_during_setup` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`updated_at` timestamp(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6) ON UPDATE CURRENT_TIMESTAMP(6),
PRIMARY KEY (`id`),
UNIQUE KEY `idx_global_or_team_id_adam_id` (`global_or_team_id`,`adam_id`,`platform`),
KEY `team_id` (`team_id`),
KEY `adam_id` (`adam_id`,`platform`),
KEY `fk_vpp_apps_teams_vpp_token_id` (`vpp_token_id`),
CONSTRAINT `fk_vpp_apps_teams_vpp_token_id` FOREIGN KEY (`vpp_token_id`) REFERENCES `vpp_tokens` (`id`) ON DELETE CASCADE,
CONSTRAINT `vpp_apps_teams_ibfk_2` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE,
CONSTRAINT `vpp_apps_teams_ibfk_3` FOREIGN KEY (`adam_id`, `platform`) REFERENCES `vpp_apps` (`adam_id`, `platform`) ON DELETE CASCADE
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
feat: enable multiple ABM and VPP tokens (#21693) > Related issue: #9956 # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [x] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [x] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) - [x] Added/updated tests - [x] If paths of existing endpoints are modified without backwards compatibility, checked the frontend/CLI for any necessary changes - [x] If database migrations are included, checked table schema to confirm autoupdate - For database migrations: - [x] Checked schema for all modified table for columns that will auto-update timestamps during migration. - [x] Confirmed that updating the timestamps is acceptable, and will not cause unwanted side effects. - [x] Ensured the correct collation is explicitly set for character columns (`COLLATE utf8mb4_unicode_ci`). - [x] Manual QA for all new/changed functionality --------- Co-authored-by: Martin Angers <martin.n.angers@gmail.com> Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com> Co-authored-by: Roberto Dip <rroperzh@gmail.com> Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com> Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com> Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
2024-08-29 22:51:46 +00:00
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_token_teams` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`vpp_token_id` int unsigned NOT NULL,
`team_id` int unsigned DEFAULT NULL,
`null_team_type` enum('none','allteams','noteam') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT 'none',
PRIMARY KEY (`id`),
UNIQUE KEY `idx_vpp_token_teams_team_id` (`team_id`),
KEY `fk_vpp_token_teams_vpp_token_id` (`vpp_token_id`),
CONSTRAINT `fk_vpp_token_teams_team_id` FOREIGN KEY (`team_id`) REFERENCES `teams` (`id`) ON DELETE CASCADE,
CONSTRAINT `fk_vpp_token_teams_vpp_token_id` FOREIGN KEY (`vpp_token_id`) REFERENCES `vpp_tokens` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vpp_tokens` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`organization_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`location` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,
`renew_at` timestamp NOT NULL,
`token` blob NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_vpp_tokens_location` (`location`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `vulnerability_host_counts` (
`cve` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL,
`team_id` int unsigned NOT NULL DEFAULT '0',
`host_count` int unsigned NOT NULL DEFAULT '0',
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`global_stats` tinyint(1) NOT NULL DEFAULT '0',
UNIQUE KEY `cve_team_id_global_stats` (`cve`,`team_id`,`global_stats`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_mdm_command_queue` (
`enrollment_id` int unsigned NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`enrollment_id`,`command_uuid`),
KEY `command_uuid` (`command_uuid`),
CONSTRAINT `windows_mdm_command_queue_ibfk_1` FOREIGN KEY (`enrollment_id`) REFERENCES `mdm_windows_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `windows_mdm_command_queue_ibfk_2` FOREIGN KEY (`command_uuid`) REFERENCES `windows_mdm_commands` (`command_uuid`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_mdm_command_results` (
`enrollment_id` int unsigned NOT NULL,
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_result` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`response_id` int unsigned NOT NULL,
`status_code` varchar(31) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`enrollment_id`,`command_uuid`),
KEY `command_uuid` (`command_uuid`),
KEY `response_id` (`response_id`),
CONSTRAINT `windows_mdm_command_results_ibfk_1` FOREIGN KEY (`enrollment_id`) REFERENCES `mdm_windows_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `windows_mdm_command_results_ibfk_2` FOREIGN KEY (`command_uuid`) REFERENCES `windows_mdm_commands` (`command_uuid`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `windows_mdm_command_results_ibfk_3` FOREIGN KEY (`response_id`) REFERENCES `windows_mdm_responses` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_mdm_commands` (
`command_uuid` varchar(127) COLLATE utf8mb4_unicode_ci NOT NULL,
`raw_command` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`target_loc_uri` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`command_uuid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_mdm_responses` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`enrollment_id` int unsigned NOT NULL,
`raw_response` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `enrollment_id` (`enrollment_id`),
CONSTRAINT `windows_mdm_responses_ibfk_1` FOREIGN KEY (`enrollment_id`) REFERENCES `mdm_windows_enrollments` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `windows_updates` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`host_id` int unsigned NOT NULL,
`date_epoch` int unsigned NOT NULL,
`kb_id` int unsigned NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_unique_windows_updates` (`host_id`,`kb_id`),
KEY `idx_update_date` (`host_id`,`date_epoch`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `wstep_cert_auth_associations` (
`id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`sha256` char(64) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`,`sha256`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `wstep_certificates` (
`serial` bigint unsigned NOT NULL,
`name` varchar(1024) COLLATE utf8mb4_unicode_ci NOT NULL,
`not_valid_before` datetime NOT NULL,
`not_valid_after` datetime NOT NULL,
`certificate_pem` text COLLATE utf8mb4_unicode_ci NOT NULL,
`revoked` tinyint(1) NOT NULL DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`serial`),
CONSTRAINT `wstep_certificates_ibfk_1` FOREIGN KEY (`serial`) REFERENCES `wstep_serials` (`serial`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `wstep_serials` (
`serial` bigint unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`serial`)
) /*!50100 TABLESPACE `innodb_system` */ ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `yara_rules` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`contents` mediumtext COLLATE utf8mb4_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_yara_rules_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
/*!40101 SET character_set_client = @saved_cs_client */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!50001 DROP VIEW IF EXISTS `nano_view_queue`*/;
/*!50001 SET @saved_cs_client = @@character_set_client */;
/*!50001 SET @saved_cs_results = @@character_set_results */;
/*!50001 SET @saved_col_connection = @@collation_connection */;
/*!50001 SET character_set_client = utf8mb4 */;
/*!50001 SET character_set_results = utf8mb4 */;
enforce an uniform collation for all tables (#10515) related to #10441, inspired by the prior work done in https://github.com/kolide/fleet/pull/1360, this PR: 1. Adds a migration to use `utf8mb4_general_ci` as the default collation for the database and all the tables. From [MySQL's documentation][1]: > To change the table default character set and all character columns > (CHAR, VARCHAR, TEXT) to a new character set, use a statement like > this: > > ``` > ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name; > ``` > The statement also changes the collation of all character columns. If > you specify no COLLATE clause to indicate which collation to use, the > statement uses default collation for the character set. 2. Changes the connection settings to use `utf8mb4_general_ci` as the default collation, from the [driver docs][2]: > Sets the collation used for client-server interaction on connection. In contrast to charset, collation does not issue additional queries. If the specified collation is unavailable on the target server, the connection will fail. [1]: https://dev.mysql.com/doc/refman/5.7/en/alter-table.html [2]: https://github.com/go-sql-driver/mysql **TODO:** discuss how we can enforce this, is setting the database default collation enough? should we add some kind of custom lint rule to all migrations? # Checklist for submitter If some of the following don't apply, delete the relevant line. - [x] Changes file added for user-visible changes in `changes/` or `orbit/changes/`. See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - [x] Added/updated tests - [x] Manual QA for all new/changed functionality
2023-03-16 18:49:24 +00:00
/*!50001 SET collation_connection = utf8mb4_unicode_ci */;
Add Apple MDM functionality (#7940) * WIP * Adding DEP functionality to Fleet * Better organize additional MDM code * Add cmdr.py and amend API paths * Fix lint * Add demo file * Fix demo.md * go mod tidy * Add munki setup to Fleet * Add diagram to demo.md * Add fixes * Update TODOs and demo.md * Fix cmdr.py and add TODO * Add endpoints to demo.md * Add more Munki PoC/demo stuff * WIP * Remove proposals from PoC * Replace prepare commands with fleetctl commands * Update demo.md with current state * Remove config field * Amend demo * Remove Munki setup from MVP-Dogfood * Update demo.md * Add apple mdm commands (#7769) * fleetctl enqueue mdm command * fix deps * Fix build Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> * Add command to upload installers * go mod tidy * fix subcommands help There is a bug in urfave/cli where help text is not generated properly when subcommands are nested too deep. * Add support for installing apps * Add a way to list enrolled devices * Add dep listing * Rearrange endpoints * Move DEP routine to schedule * Define paths globally * Add a way to list enrollments and installers * Parse device-ids as comma-separated string * Remove unused types * Add simple commands and nest under enqueue-command * Fix simple commands * Add help to enqueue-command * merge apple_mdm database * Fix commands * update nanomdm * Split nanomdm and nanodep schemas * Set 512 MB in memory for upload * Remove empty file * Amend profile * Add sample commands * Add delete installers and fix bug in DEP profile assigning * Add dogfood.md deployment guide * Update schema.sql * Dump schema with MySQL 5 * Set default value for authenticate_at * add tokens to enrollment profiles When a device downloads an MDM enrollment profile, verify the token passed as a query parameter. This ensures untrusted devices don't enroll with our MDM server. - Rename enrollments to enrollment profiles. Enrollments is used by nano to refer to devices that are enrolled with MDM - Rename endpoint /api/<version>/fleet/mdm/apple/enrollments to ../enrollmentprofiles - Generate a token for authentication when creating an enrollment profile - Return unauthorized if token is invalid when downloading an enrollment profile from /api/mdm/apple/enroll?token= * remove mdm apple server url * update docs * make dump-test-schema * Update nanomdm with missing prefix table * Add docs and simplify changes * Add changes file * Add method docs * Fix compile and revert prepare.go changes * Revert migration status check change * Amend comments * Add more docs * Clarify storage of installers * Remove TODO * Remove unused * update dogfood.md * remove cmdr.py * Add authorization tests * Add TODO comment * use kitlog for nano logging * Add yaml tags * Remove unused flag * Remove changes file * Only run DEP routine if MDM is enabled * Add docs to all new exported types * Add docs * more nano logging changes * Fix unintentional removal * more nano logging changes * Fix compile test * Use string for configs and fix config test * Add docs and amend changes * revert changes to basicAuthHandler * remove exported BasicAuthHandler * rename rego authz type * Add more information to dep list * add db tag * update deps * Fix schema * Remove unimplemented Co-authored-by: Michal Nicpon <39177923+michalnicp@users.noreply.github.com> Co-authored-by: Michal Nicpon <michal@fleetdm.com>
2022-10-05 22:53:54 +00:00
/*!50001 CREATE ALGORITHM=UNDEFINED */
/*!50013 DEFINER=`root`@`%` SQL SECURITY DEFINER */
/*!50001 VIEW `nano_view_queue` AS select `q`.`id` AS `id`,`q`.`created_at` AS `created_at`,`q`.`active` AS `active`,`q`.`priority` AS `priority`,`c`.`command_uuid` AS `command_uuid`,`c`.`request_type` AS `request_type`,`c`.`command` AS `command`,`r`.`updated_at` AS `result_updated_at`,`r`.`status` AS `status`,`r`.`result` AS `result` from ((`nano_enrollment_queue` `q` join `nano_commands` `c` on((`q`.`command_uuid` = `c`.`command_uuid`))) left join `nano_command_results` `r` on(((`r`.`command_uuid` = `q`.`command_uuid`) and (`r`.`id` = `q`.`id`)))) order by `q`.`priority` desc,`q`.`created_at` */;
/*!50001 SET character_set_client = @saved_cs_client */;
/*!50001 SET character_set_results = @saved_cs_results */;
/*!50001 SET collation_connection = @saved_col_connection */;