for #21440
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
> Related issue: #9956
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated tests
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Dante Catalfamo <43040593+dantecatalfamo@users.noreply.github.com>
Co-authored-by: Roberto Dip <dip.jesusr@gmail.com>
for #21198
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
> Related issue: #20311
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
> No related issue, updating an existing tool
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
for #19844
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
`go-kit/kit/log` was deprecated and generating warnings
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
Backend changes for #19010.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Manual QA for all new/changed functionality
For #19016
This changes all the places where we previously assumed that certs were
hardcoded when the Fleet server started to query the database instead.
The plan is to loadtest afterwards, but as a first preemptive measure,
this adds a caching layer on top the mysql datastore.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- For database migrations:
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [x] Manual QA for all new/changed functionality
For #16795, this:
- Updates Go to go1.22.3
- Per
https://github.com/fleetdm/fleet/issues/16795#issuecomment-2100450618, I
also ran the following to update the versions requested by @getvictor
```
go get github.com/kataras/golog@v0.1.12
go get github.com/kataras/iris/v12@v12.2.11
go get github.com/sethvargo/go-password@v0.3.0
```
**Notes**
After this is merged people will need to update their Go version. I use
gvm and I did it like:
```
$ gvm install go1.22.3
$ gvm use go1.22.3 --default
```
**Relevant changes**
The release notes mention:
> Previously, the variables declared by a “for” loop were created once
> and updated by each iteration. In Go 1.22, each iteration of the loop
> creates new variables, to avoid accidental sharing bugs.
However, we already have a lint rule (see
https://github.com/fleetdm/fleet/pull/13877) for this scenario, so it
shouldn't affect us.
#16677
Improvements to `fleetctl gitops` command:
- Added the ability to pass multiple files, like `fleetctl gitops -f
file1 -f file2`, where the first file must be the global configuration
- Added the ability to remove teams that were not specified in team
configs using the switch `--delete-other-teams`
- When passing a global config and team config during initial
configuration, the `org_settings.mdm.apple_bm_default_team` value can be
set to match the team that will be created by the provided team config.
After these changes are released to prod, we can update
https://github.com/fleetdm/fleet-gitops to use the new switches: #18692
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Script will build a fleet-osquery package, launch a fresh macos VM and
install it. It will also download the correct MDM profile and prepare it
to be enabled
for #14715
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
final step of #14361 , this integrates the work of the other PRs.
currently branched off `14361-windows-custom-settings-configs`
---------
Co-authored-by: Martin Angers <martin.n.angers@gmail.com>
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [ ] Documented any permissions changes (docs/Using
Fleet/manage-access.md)
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] Added/updated tests
- [ ] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
For #13715, this:
- Upgrades the Go version to `1.21.1`, infrastructure changes are
addressed separately at https://github.com/fleetdm/fleet/pull/13878
- Upgrades the linter version, as the current version doesn't work well
after the Go upgrade
- Fixes new linting errors (we now get errors for memory aliasing in
loops! 🎉 )
After this is merged people will need to:
1. Update their Go version. I use `gvm` and I did it like:
```
$ gvm install go1.21.1
$ gvm use go1.21.1 --default
```
2. Update the local version of `golangci-lint`:
```
$ go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.54.2
```
3. (optional) depending on your setup, you might need to re-install some
packages, for example:
```
# goimports to automatically import libraries
$ go install golang.org/x/tools/cmd/goimports@latest
# gopls for the language server
$ go install golang.org/x/tools/gopls@latest
# etc...
```
This PR requires the Windows MDM configuration changes - This will be
updated next week
- [x] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)
- [x] Documented any permissions changes
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
#11531
How to use:
```
go run ./tools/mdm/apple/loadtest -api_token $API_TOKEN -fleet_url https://fleet.example.com -team_count 5 -team_extra_count 2 -loop_count 1
2023-05-15T16:05:06Z: 1. Creating 5 teams... (press enter to proceed)
2023-05-15T16:05:18Z: 1. Duration: 165.020742ms
2023-05-15T16:05:18Z: 2. Transfering one host to each team... (press enter to proceed)
2023-05-15T16:05:20Z: 2. Duration: 370.515741ms
2023-05-15T16:05:20Z: 3a. Add 10 profiles to all teams... (press enter to proceed)
2023-05-15T16:05:22Z: Applying profiles to team Team 0...
2023-05-15T16:05:22Z: Applying profiles to team Team 1...
2023-05-15T16:05:22Z: Applying profiles to team Team 2...
2023-05-15T16:05:22Z: Applying profiles to team Team 3...
2023-05-15T16:05:22Z: Applying profiles to team Team 4...
2023-05-15T16:05:22Z: 3a. Duration: 502.557741ms
2023-05-15T16:05:22Z: 3b. Waiting for all profiles to be applied on all teams...
2023-05-15T16:05:22Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:27Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:32Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:37Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:42Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:1 Pending:0 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:48Z: 3b. Duration: 25.487993296s
2023-05-15T16:05:48Z: 4a. Modify a profile on all teams... (press enter to proceed)
2023-05-15T16:05:50Z: 4a. Duration: 654.712391ms
2023-05-15T16:05:50Z: 4b. Waiting for all profiles to be applied on all hosts of all teams...
2023-05-15T16:05:50Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:05:55Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:06:00Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:06:05Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:0 Pending:1 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:06:10Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:0 Pending:1 Failed:0} 44:{Verifying:0 Pending:1 Failed:0}
45:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:06:15Z: Waiting for all profiles to be applied on hosts..., summary: map[41:{Verifying:0 Pending:1 Failed:0} 42:{Verifying:1 Pending:0 Failed:0} 43:{Verifying:1 Pending:0 Failed:0} 44:{Verifying:1 Pending:0 Failed:0}
45:{Verifying:1 Pending:0 Failed:0}]
2023-05-15T16:06:20Z: 4b. Duration: 30.48166655s
2023-05-15T16:06:20Z: 5. Creating extra 2 teams... (press enter to proceed)
2023-05-15T16:06:22Z: 5. Duration: 72.531251ms
2023-05-15T16:06:22Z: 6a. Moving one host to each new extra 2 teams... (press enter to proceed)
2023-05-15T16:07:23Z: 6a. Duration: 164.446788ms
2023-05-15T16:07:23Z: 6b. Waiting for all profiles to be applied on all hosts of the extra teams...
2023-05-15T16:07:23Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:29Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:34Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:39Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:44Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}]
2023-05-15T16:07:49Z: 6b. Duration: 25.210811603s
2023-05-15T16:07:49Z: 7a. Add 10 profiles to all extra 2 teams... (press enter to proceed)
2023-05-15T16:07:50Z: 7a. Duration: 175.694423ms
2023-05-15T16:07:50Z: 7b. Waiting for all profiles to be applied on all hosts of the extra teams...
2023-05-15T16:07:50Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:07:55Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:08:01Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:08:06Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:0 Pending:1 Failed:0}]
2023-05-15T16:08:11Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}]
2023-05-15T16:08:16Z: Waiting for all profiles to be applied on hosts..., summary: map[46:{Verifying:0 Pending:1 Failed:0} 47:{Verifying:1 Pending:0 Failed:0}]
2023-05-15T16:08:21Z: 7b. Duration: 30.219261468s
2023-05-15T16:08:21Z: 8. Destroy 2 extra teams... (press enter to proceed)
2023-05-15T16:08:31Z: 8. Duration: 134.372194ms
```
- ~[ ] Changes file added for user-visible changes in `changes/` or
`orbit/changes/`.~
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- ~[ ] Documented any API changes (docs/Using-Fleet/REST-API.md or
docs/Contributing/API-for-contributors.md)~
- ~[ ] Documented any permissions changes~
- ~[ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)~
- ~[ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.~
- ~[ ] Added/updated tests~
- [X] Manual QA for all new/changed functionality
- ~For Orbit and Fleet Desktop changes:~
- ~[ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~
- ~[ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~
