for #31321
# Details
Small updates from [community
PR](https://github.com/fleetdm/fleet/pull/31134):
* Updated config vars to match
[docs](https://github.com/fleetdm/fleet/blob/docs-v4.75.0/docs/Configuration/fleet-server-configuration.md#server_private_key_region)
* Added support for specifying region in config (already documented)
* Removed parsing of ARN for region
* Made retry backoff intervals a bit longer
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
(already added in the community PR
[here](https://github.com/fleetdm/fleet/blob/sgress454/updates-for-private-key-in-aws-sm/changes/private-key-secrets-manager#L0-L1)
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added support for specifying the AWS region for server private key
retrieval from AWS Secrets Manager via server.private_key_region.
- Chores
- Renamed configuration keys:
- server.private_key_secret_arn → server.private_key_arn
- server.private_key_secret_sts_assume_role_arn →
server.private_key_sts_assume_role_arn
- server.private_key_secret_sts_external_id →
server.private_key_sts_external_id
- Update your configuration to use the new keys.
- Adjusted retry backoff for Secrets Manager retrieval to improve
resilience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#31693
Manually forced a run for MySQL 8.4.6 to validate.
# Checklist for submitter
- Changes not needed since this is not a product change.
## Testing
- [x] Added/updated automated tests
Also ensures we run integration tests when docker-compose files used by
`fleetctl preview` are changed, so we don't merge any more test failures
due to those.
For #28902
Modified the GitHub Actions workflow to prevent it from hanging at the
"waiting for mysql..." step. The updated workflow now:
- Times out after 5 minutes of unsuccessful MySQL connection attempts
- Logs all connection attempts with timestamps and error messages when a
timeout occurs
- Dumps MySQL container logs when a timeout occurs, providing valuable
diagnostic information
- Automatically stops and restarts all Docker containers using the same
command as the original "Start Infra Dependencies" step
- Retries this process up to 5 times before failing the job
Replacing MySQL 8.4 with MySQL 9.X in our regular (every commit) tests
to speed up dev feedback. The MySQL 8.4 tests will still run nightly.
The vulnerability feed download CI fails below are not related to this
change.
This PR makes several improvements to test-go CI job
- remove ZSH dependency from all test suites except for a new `scripts`
suite
- add a `fast` suite that does not have out-of-process dependencies,
which completes in ~1.5 minutes
- contributors can add their fast tests to this suite so they can see
the results in CI faster
- Rename `core` to `main` test suite to be consistent with Makefile. It
is the default bucket for tests.
- Cleaned up Makefile so that it is more straightforward to add new test
suites or move Go packages between suites
- Do not stop the test suites on a fail.
- We do not want to be blocked by a test fail that another product team
introduced
- Sometimes, we want to see all test failures so we can fix them all at
once.
- Removed `test-schema` and `mock` prerequisites for `test-go` since
they are not needed and just take up time.
- But also added `test-schema` run to one of the test suites just in
case.
Unfortunately, `fleetctl` is still the bottleneck and needs to be
refactored. New issue filed:
https://github.com/fleetdm/fleet/issues/27927
## Before
<img width="248" alt="image"
src="https://github.com/user-attachments/assets/110ffc1d-f090-4d3e-be77-0419b9577d20"
/>
## After
<img width="320" alt="image"
src="https://github.com/user-attachments/assets/8d01ea11-408f-4eb6-81d8-9c25410b8830"
/>
Release notes: https://tip.golang.org/doc/go1.24
> Go modules can now track executable dependencies using tool directives
in go.mod. This removes the need for the previous workaround of adding
tools as blank imports to a file conventionally named “tools.go”. The go
tool command can now run these tools in addition to tools shipped with
the Go distribution. For more information see [the
documentation](https://tip.golang.org/doc/modules/managing-dependencies#tools).
The new -tool flag for go get causes a tool directive to be added to the
current module for named packages in addition to adding require
directives.
I ran:
```
go get -tool github.com/fleetdm/fleet/v4/server/goose
go get -tool github.com/kevinburke/go-bindata
go get -tool github.com/quasilyte/go-ruleguard/dsl
go rm tools.go
go mod tidy
```
`make deps-go` was failing in CI because of the removal of `tools.go`
(my guess is that `go get .` was a nop because there was nothing in `.`
to download).
So, taking the chance of removing `deps-go` because `go` will download
packages during the build process. AFAICS there's no need to download
everything beforehand.
From what I can tell, continue-on-error has been false for the
integration suites since the suites were renamed to `integration-*`, so
this fixes that issue in addition to continuing to run test suites when
the vulns suite fails (which may be due to vulns feed updates).
This also makes the vulns test more resilient to new CVEs being reported
on Python 3.12.0, which is rather likely to collect new CVEs.
# Checklist for submitter
- [x] Added/updated automated tests
#23905
- Update with upstream nanomdm changes up to
825f2979a2
- Removed PostgeSQL folder from our nanomdm
- Added nanomdm MySQL test job to our CI
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
Also mention that we test with 8.4.2 in a few more places.
Note that while I'm editing release articles, this isn't retconning
minimum requirements; we mention in 4.55.0 release notes further down
that we expect 8.0.36.
Done as part of oncall improvements.
`vars.GO_VERSION` can only be changed by admins and it's not public
(Fleet devs don't know the current value of the variable), this approach
uses the version specified in our `go.mod` file.
This ensures `#help-engineering` is notified when a dogfood deploy is in
progress. It helps set people's expectations about what's going on while
the server is temporarily down.
for #18297
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
#15881
This PR adds a script to test DB migrations with Percona XtraDB 5.7.25.
PS: To run this test before we merge this PR to `main` you will need to
change step 2 (`Make sure to be on latest main`), instead of `main` use
this branch `15881-test-migrations-with-percona`.
Related to: #10209
Changes:
- Updated the go test in the Makefile to have the `--codepkg` flag.
- Added a newline to the `test-go` GH workflow to trigger a run for this
PR
> Note: I'm creating this as a draft PR to see the results of the "Test
Go" workflow
For #13715, this:
- Upgrades the Go version to `1.21.1`, infrastructure changes are
addressed separately at https://github.com/fleetdm/fleet/pull/13878
- Upgrades the linter version, as the current version doesn't work well
after the Go upgrade
- Fixes new linting errors (we now get errors for memory aliasing in
loops! 🎉 )
After this is merged people will need to:
1. Update their Go version. I use `gvm` and I did it like:
```
$ gvm install go1.21.1
$ gvm use go1.21.1 --default
```
2. Update the local version of `golangci-lint`:
```
$ go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.54.2
```
3. (optional) depending on your setup, you might need to re-install some
packages, for example:
```
# goimports to automatically import libraries
$ go install golang.org/x/tools/cmd/goimports@latest
# gopls for the language server
$ go install golang.org/x/tools/gopls@latest
# etc...
```
