mirror of
https://github.com/fleetdm/fleet
synced 2026-05-21 07:58:31 +00:00
6 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Scott Gress
|
c1c078795e
|
Fix macos_setup not always being exported correctly by generate-gitops (#30504)
for #30502 # Details This PR fixes an issue where `fleetctl generate-gitops` would not always add a `macos_setup` setting to a .yml file even if the team had a setup experience configured. This was due to relying on the `MacOSSetup` config returned by app/team config APIs to have this data populated, which turned out to be an incorrect assumption. Instead, we now utilize various APIs to check for the presence of setup software, scripts, bootstrap packages and profiles. Note that for now, `generate-gitops` will only output a `TODO` line if setup experience is detected; https://github.com/fleetdm/fleet/issues/30210 is open to flesh this out. In the meantime `fleetctl gitops` will fail if this TODO is inserted, so that the user must go and fix it manually. # Checklist for submitter If some of the following don't apply, delete the relevant line. <!-- Note that API documentation changes are now addressed by the product design team. --> - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. - [X] Added/updated automated tests - [X] Manual QA for all new/changed functionality # Testing I set up MDM on a local instance and tried the following both on No Team and a regular team: * Turned "End user authentication on", verified that `fleetctl generate-gitops` output a `macos_setup` setting for the team. Turned it back off and verified that `macos_setup` was no longer exported by `fleetctl generate-gitops`. * Did the same for bootstrap package. * Did the same for install software, and additionally verified that having software available but _not_ selected did not cause `macos_setup` to be exported. Same for teams with no software available at all. * Did the same for setup assistant. I also tested that changes to No Team didn't affect the output when exporting a regular team. --------- Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com> |
||
|
Scott Gress
|
d716265641
|
Add "generate-gitops" command (#28555)
For #27476 # Checklist for submitter - [X] Changes file added for user-visible changes in `changes/`, `orbit/changes/` or `ee/fleetd-chrome/changes`. See [Changes files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files) for more information. - [X] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements) # Details This PR adds a new command `generate-gitops` to the `fleetctl` tool. The purpose of this command is to output GitOps-ready files that can then be used with `fleetctl-gitops`. The general usage of the command is: ``` fleectl generate-gitops --dir /path/to/dir/to/add/files/to ``` By default, the outputted files will not contain sensitive data, but will instead add comments where the data needs to be replaced by a user. In cases where sensitive data is redacted, the tool outputs warnings to the user indicating which keys need to be updated. The tool uses existing APIs to gather data for use in generating configuration files. In some cases new API client methods needed to be added to support the tool: * ListConfigurationProfiles * GetProfileContents * GetScriptContents * GetSoftwareTitleByID Additionally, the response for the /api/latest/fleet/software/batch endpoint was updated slightly to return `HashSHA256` for the software installers. This allows policies that automatically install software to refer to that software by hash. Other options that we may or may not choose to document at this time: * `--insecure`: outputs sensitive data in plaintext instead of leaving comments * `--print`: prints the output to stdout instead of writing files * `--key`: outputs the value at a keypath to stdout, e.g. `--key agent_options.config` * `--team`: only generates config for the specified team name * `--force`: overwrites files in the given directory (defaults to false, which errors if the dir is not empty) # Technical notes The command is implemented using a `GenerateGitopsCommand` type which holds some state (like a list of software and scripts encountered) as well as a Fleet client instance (which may be a mock instance for tests) and the CLI context (containing things like flags and output writers). The actual "action" of the CLI command calls the `Run()` method of the `GenerateGitopsCommand` var, which delegates most of the work to other methods like `generateOrgSettings()`, `generateControls()`, etc. Wherever possible, the subroutines use reflection to translate Go struct fields into JSON property names. This guarantees that the correct keys are written to config files, and protects against the unlikely event of keys changing. When sensitive data is encountered, the subroutines call `AddComment()` to get a new token to add to the config files. These tokens are replaced with comments like `# TODO - Add your enrollment secrets here` in the final output. # Known issues / TODOs: * The `macos_setup` configuration is not output by this tool yet. More planning is required for this. In the meantime, if the tool detects that `macos_setup` is configured on the server, it outputs a key with an invalid value and prints a warning to the user that they'll need to configure it themselves. * `yara_rules` are not output yet. The tool adds a warning that if you have Yara rules (which you can only upload via GitOps right now) that you'll have to migrate them manually. Supporting this will require a new API that we'll have to discuss the authz for, so punting on it for now. * Fleet maintained apps are not supported by GitOps yet (coming in https://github.com/fleetdm/fleet/issues/24469). In the meantime, this tool will output a `fleet_maintained_apps` key and trigger a warning, and GitOps will fail if that key is present. --------- Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com> Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com> |
||
|
Martin Angers
|
f5f66ebe63
|
Unify profiles database identifier to profile_uuid for macOS and Windows profiles. (#15297)
|
||
|
Sarah Gillespie
|
0850367734
|
Add GET /mdm/profiles/summary endpoint (#15077)
|
||
|
Juan Fernandez
|
7226b7f087
|
Warnings in fleetctl should use Stderr not Stdout (#12316)
Fixed issue were the expired license banner was being sent to Stdout instead of Stderr |
||
|
Lucas Manuel Rodriguez
|
b3e038fffc
|
Add mdm loadtest script (#11696)
#11531 How to use: ``` go run ./tools/mdm/apple/loadtest -api_token $API_TOKEN -fleet_url https://fleet.example.com -team_count 5 -team_extra_count 2 -loop_count 1 2023-05-15T16:05:06Z: 1. Creating 5 teams... (press enter to proceed) 2023-05-15T16:05:18Z: 1. Duration: 165.020742ms 2023-05-15T16:05:18Z: 2. Transfering one host to each team... (press enter to proceed) 2023-05-15T16:05:20Z: 2. Duration: 370.515741ms 2023-05-15T16:05:20Z: 3a. Add 10 profiles to all teams... (press enter to proceed) 2023-05-15T16:05:22Z: Applying profiles to team Team 0... 2023-05-15T16:05:22Z: Applying profiles to team Team 1... 2023-05-15T16:05:22Z: Applying profiles to team Team 2... 2023-05-15T16:05:22Z: Applying profiles to team Team 3... 2023-05-15T16:05:22Z: Applying profiles to team Team 4... 2023-05-15T16:05:22Z: 3a. Duration: 502.557741ms 2023-05-15T16:05:22Z: 3b. Waiting for all profiles to be applied on all teams... 2023-05-15T16:05:22Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:05:27Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:05:32Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:05:37Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:05:42Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:1 Pending:0 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:05:48Z: 3b. Duration: 25.487993296s 2023-05-15T16:05:48Z: 4a. Modify a profile on all teams... (press enter to proceed) 2023-05-15T16:05:50Z: 4a. Duration: 654.712391ms 2023-05-15T16:05:50Z: 4b. Waiting for all profiles to be applied on all hosts of all teams... 2023-05-15T16:05:50Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:05:55Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:06:00Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:06:05Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:06:10Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0} 45:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:06:15Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:1 Pending:0 Failed:0} 44:{Verifying:1 Pending:0 Failed:0} 45:{Verifying:1 Pending:0 Failed:0}] 2023-05-15T16:06:20Z: 4b. Duration: 30.48166655s 2023-05-15T16:06:20Z: 5. Creating extra 2 teams... (press enter to proceed) 2023-05-15T16:06:22Z: 5. Duration: 72.531251ms 2023-05-15T16:06:22Z: 6a. Moving one host to each new extra 2 teams... (press enter to proceed) 2023-05-15T16:07:23Z: 6a. Duration: 164.446788ms 2023-05-15T16:07:23Z: 6b. Waiting for all profiles to be applied on all hosts of the extra teams... 2023-05-15T16:07:23Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:07:29Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:07:34Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:07:39Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:07:44Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}] 2023-05-15T16:07:49Z: 6b. Duration: 25.210811603s 2023-05-15T16:07:49Z: 7a. Add 10 profiles to all extra 2 teams... (press enter to proceed) 2023-05-15T16:07:50Z: 7a. Duration: 175.694423ms 2023-05-15T16:07:50Z: 7b. Waiting for all profiles to be applied on all hosts of the extra teams... 2023-05-15T16:07:50Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:07:55Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:08:01Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:08:06Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}] 2023-05-15T16:08:11Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}] 2023-05-15T16:08:16Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}] 2023-05-15T16:08:21Z: 7b. Duration: 30.219261468s 2023-05-15T16:08:21Z: 8. Destroy 2 extra teams... (press enter to proceed) 2023-05-15T16:08:31Z: 8. Duration: 134.372194ms ``` - ~[ ] Changes file added for user-visible changes in `changes/` or `orbit/changes/`.~ See [Changes files](https://fleetdm.com/docs/contributing/committing-changes#changes-files) for more information. - ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or docs/Contributing/API-for-contributors.md)~ - ~[ ] Documented any permissions changes~ - ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL injection is prevented (using placeholders for values in statements)~ - ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for new osquery data ingestion features.~ - ~[ ] Added/updated tests~ - [X] Manual QA for all new/changed functionality - ~For Orbit and Fleet Desktop changes:~ - ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows and Linux.~ - ~[ ] Auto-update manual QA, from released version of component to new version (see [tools/tuf/test](../tools/tuf/test/README.md)).~ |