<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33110
**Related issue:** Resolves#33109
# Details
This PR implements the new "cancel setup if any software fails on macos"
flag, including both backend and frontend logic.
Half of the file changes are updating test expectations / auto-generated
schema.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
`macos_setup` is still excluded from generate-girtops
- [X] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
Documented [here](https://github.com/fleetdm/fleet/pull/33016/files)
- [X] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [X] Verified that any relevant UI is disabled when GitOps mode is
enabled
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Added a macOS setup option: “Cancel setup if software install fails.”
- Configure at global or team level; team settings override global.
- Toggle available in Setup Experience > Install software > Advanced
options.
- Saved state persists and can be updated without leaving the page.
- Devices honor the resolved setting during provisioning.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #33111
# Details
This PR updates the setup experience for MacOS to use a web view pointed
at the device's "Setting up your device" page rather than using native
MacOS UI elements, bringing it more in line with Linux and Windows setup
experiences.
This covers only the new web UI for the setup experience progress, _not_
the UI for the new case of blocking the device when a piece of software
fails to install. I'll add that in a separate PR.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
Added tests for the updates to the token rotation code.
- [X] QA'd all new/changed functionality manually
A new tool is provided to allow testing this code against a virtual
machine if a separate host that you can wipe and run setup on is not
available. See
https://github.com/fleetdm/fleet/blob/sgress454/new-setup-experience/tools/mdm/apple/setupexperience/README.md
for details.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- macOS setup experience moved to a new web-based UI.
- Automatic device token rotation during setup to keep sessions valid.
- Bug Fixes
- More reliable setup flow with improved dialog lifecycle and cleaner
handoff to web content.
- Dialog elements hidden/cleared appropriately when transitioning to the
browser.
- Documentation
- Added guide and tool to simulate the macOS setup experience on a VM,
with prerequisites and usage steps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#31897.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [ ] Added/updated automated tests
- [ ] QA'd all new/changed functionality manually
## New Fleet configuration settings
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- GitOps now supports software icons: generate and include icon
files/paths in specs for packages and App Store apps.
- CLI adds flags to control concurrent icon uploads/updates.
- Icons are uploaded, updated, or deleted automatically during GitOps
runs.
- UI YAML modal now includes icon_url and offers icon download.
- Improvements
- Robust path resolution for icon assets across specs.
- Non-YAML outputs handle both string and byte file contents.
- Bug Fixes
- Removes stale icons after App Store app re-association.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Scott Gress <scottmgress@gmail.com>
Co-authored-by: Scott Gress <scott@fleetdm.com>
Co-authored-by: Jahziel Villasana-Espinoza <jahziel@fleetdm.com>
Resolves#31890
This new approach allows up to 1000 consecutive failing requests per
minute.
If the threshold of 1000 consecutive failures is reached for an IP, then
we ban request (return 429) from such IP for a duration of 1 minute.
(Any successful request for an IP clears the count.)
This supports the scenario where all hosts are behind a NAT (same IP)
AND still provides protection against brute force attacks (attackers can
only probe 1k requests per minute).
This approach was discussed in Slack with @rfairburn:
https://fleetdm.slack.com/archives/C051QJU3D0V/p1755625131298319?thread_ts=1755101701.844249&cid=C051QJU3D0V.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Introduced IP-based rate limiting for Fleet Desktop endpoints to
better support many hosts behind a single public IP (NAT). Requests from
abusive IPs may be temporarily blocked, returning 429 Too Many Requests
with a retry-after hint.
- Documentation
- Added README for a new desktop rate-limit tester, describing usage and
expected behavior.
- Tests
- Added integration tests covering desktop endpoint rate limiting and
Redis-backed banning logic.
- Chores
- Added a command-line tool to stress-test desktop endpoints and verify
rate limiting behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This looks like a log but it's just moving the code that lived in
cmd/gm/ui.go into it's own pkg that makes it clearer where each part of
the UI lives.
I am also adding the functionality for bulk sprint demo to grab names
from github usernames.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33250
Waived most new failures. Planning to come back and fix some of them in
subsequent PRs.
What’s in this PR
1) Smarter default sorting for issues (used by the TUI)
New ghapi.SortIssuesForDisplay helper that orders issues by:
Priority label (P0 → P1 → P2 → none)
Presence of customer-* / prospect-* labels
Type labels (story → bug → ~sub-task → others)
Issue number (descending)
This is applied before filtering so views start in a meaningful order.
[GitHub](https://github.com/fleetdm/fleet/pull/32694/files)
Implementation lives in tools/github-manage/pkg/ghapi/sort.go.
Comprehensive tests cover all combinations, tie-breakers, and stability.
GitHub
+1
2) Estimates: show the sum for the current selection
The header now displays Σest sel=<sum> for the currently selected
issues, both in filtered and unfiltered views, making quick capacity
checks easier.
[GitHub](https://github.com/fleetdm/fleet/pull/32694/files)
3) Better progress UI for workflows
Task list is now windowed (last ~10 items) with auto-scroll to the
currently running or most recently finished task, plus “earlier/more
tasks” ellipses and a progress counter at the bottom. This keeps the
view focused during long runs.
[GitHub](https://github.com/fleetdm/fleet/pull/32694/files)
4) Project estimates fetch now includes total count
Switched from GetEstimatedTicketsForProject to
GetEstimatedTicketsForProjectWithTotal, so we can show totalAvailable
alongside rawFetched/limit.
[GitHub](https://github.com/fleetdm/fleet/pull/32694/files)
---------
Co-authored-by: Jordan Montgomery <elijah.jordan.montgomery@gmail.com>
Fixes#29909
- Do not update DB if rules haven't changed
- Cache Yara rules when retrieved by hosts. This should reduce DB
accesses with large number of hosts retrieving large numbers of rules
I manually QA'd using OpenTelemetry (APM would also work) and monitoring
the DB accesses when updating or retrieving yara rules.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Performance Improvements**
* Faster config saves when many YARA rules are present (incremental
updates, reduced work).
* Lower latency and load when many hosts fetch YARA rules (caching and
smarter retrieval).
* More efficient handling of unchanged, added, modified, and removed
YARA rules.
* **Documentation**
* Changelog entry noting YARA rules performance and fetch improvements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- **Adding sprint command to view items in current sprint and warning
when limit is too small**
- **Adding select / deselect all and demo output**
- **Added hotkey 's' to select all subissues along with the issue
selected**
Example of how I generated the demo docs for mdm this sprint
```
./gm sprint mdm --limit 200
'pressed l to select all issues'
'pressed w to select a workflow'
selected Bulk Demo workflow and hit enter
copy/paste features to features and bugs to bugs
modify gh usernames to @email and hit 'space' after every ) in the markdown to get it to trigger
hit tab for all issues listed under each user
...
profit
```
There are still some TODOs particularly within Gitops test code which
will be worked on in a followup PR
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [ ] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [x] Confirmed that the fix is not expected to adversely impact load
test results
- [x] Alerted the release DRI if additional load testing is needed
## Database migrations
- [x] Checked table schema to confirm autoupdate
- [x] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [x] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [x] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
## New Fleet configuration settings
- [ ] Setting(s) is/are explicitly excluded from GitOps
If you didn't check the box above, follow this checklist for
GitOps-enabled settings:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified the setting is documented in a separate PR to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [x] Verified that any relevant UI is disabled when GitOps mode is
enabled
---------
Co-authored-by: Gabriel Hernandez <ghernandez345@gmail.com>
Co-authored-by: Magnus Jensen <magnus@fleetdm.com>
Co-authored-by: Sarah Gillespie <73313222+gillespi314@users.noreply.github.com>
for #1817
# Details
This PR gives Fleet servers the ability to connect to RDS MySQL and
Elasticache Redis via AWS [Identity and Access Management
(IAM)](https://aws.amazon.com/iam/). It is based almost entirely on the
work of @titanous, branched from his [original pull
request](https://github.com/fleetdm/fleet/pull/31075). The main
differences between his branch and this are:
1. Removal of auto-detection of AWS region (and cache name for
Elasticache) in favor of specifying these values in configuration. The
auto-detection is admittedly handy but parsing AWS host URLs is not
considered a best practice.
2. Relying on the existence of these new configs to determine whether or
not to connect via IAM. This sidesteps a thorny issue of whether to try
an IAM-based Elasticache connection when a password is not supplied,
since this is technically a valid setup.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually - besides using
@titanous's excellent test tool, I verified the following end-to-end:
- [X] regular (non RDS) MySQL connection
- [X] RDS MySQL connection using username/password
- [X] RDS MySQL connection using IAM (no role)
- [X] RDS MySQL connection using IAM (assuming role)
- [X] regular (non Elasticache) Redis connection
- [X] Elasticache Redis connection using username/password
- [X] Elasticache Redis connection using NO password (without IAM)
- [X] Elasticache Redis connection using IAM (no role)
- [X] Elasticache Redis connection using IAM (assuming role)
---------
Co-authored-by: Jonathan Rudenberg <jonathan@titanous.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
For #30095.
#32482 is additional cleanup. Merging this to unblock orchestration
Linux setup experience work. Code has already been reviewed prior to
merging into the feature branch.
---------
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
Co-authored-by: Anthony Maxwell <133805840+Illbjorn@users.noreply.github.com>
Fixes#32060
This PR adds:
- new default_team_config_json table
- caching of config from that table, including deep copy methods -- all
of this is not absolutely needed for this change since we are only using
`webhook_settings.failing_policies_webhook` here but added for
completeness/future
- teams/0 API updates
- GitOps updates
- generate gitops updates
Future PRs will add:
- ticket automation
- primo mode migration
- frontend changes
- documentation
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## Database migrations
- [x] Checked table schema to confirm autoupdate
## New Fleet configuration settings
- [x] Verified that the setting is exported via `fleetctl
generate-gitops`
- [x] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Configure failing-policy webhooks for “No team” via GitOps
(no-team.yml) and API, including enable/disable, destination URL, policy
IDs, and batch size; settings clear when omitted.
- GitOps and CLI now read/apply the real “No team” settings with dry-run
support.
- Policy automation evaluates hosts without a team and triggers “No
team” webhooks when applicable.
- GET/PATCH team 0 returns/accepts a minimal, webhook-focused config.
- Chores
- Added persistence and caching for the default “No team” configuration.
- Introduced a database table to store the default configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
#31977
Tested with:
224 scripts, 102 Apple configuration profiles, 105 Apple declarations,
37 Windows configuration profiles. Scattered in 7 teams.
No variables on profiles/scripts:
- scan scripts: 22.079416ms
- scan apple profiles: 1.865083ms
- scan apple declarations: 683.75µs
- scan windows profiles: 531.167µs
A variable "X" on with profiles/scripts that have variable "Y":
- scan scripts: 26.305459ms
- scan apple profiles: 1.841708ms
- scan apple declarations: 756.917µs
- scan windows profiles: 571.875µs
---
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [X] Alerted the release DRI if additional load testing is needed
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Prevents deleting a secret that’s still in use. The system now checks
scripts, Apple/Windows MDM profiles, and Apple declarations across teams
before allowing deletion.
- The delete dialog now shows a clear message indicating which item (and
team) is using the secret, replacing the previous generic error.
- Deletion succeeds once all references are removed, ensuring safer
secret management.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#31781
Only test-related changes.
I found this useful for agentic AI workflows. For example, you have an
AI agent debugging/rerunning a test. Meanwhile, you can spin up another
`mysql_test` instance in another workarea and work there in parallel.
Ran
```
make update-go version=1.24.6
```
And then updated the `sha256`s manually in the Dockerfiles.
Fixes https://nvd.nist.gov/vuln/detail/CVE-2025-47907
```
Cancelling a query (e.g. by cancelling the context passed to one of the query methods) during a call
to the Scan method of the returned Rows can result in unexpected results if other queries are being
made in parallel. This can result in a race condition that may overwrite the expected results with those
of another query, causing the call to Scan to return either unexpected results from the other
query or an error.
```
Related to: #31753
Changes:
- Replaced the large GIFs used in the tools/github-manage folder's
README with compressed versions. (The uncompressed versions are possibly
exceeding Heroku's filesize limit for `git push`.)
If this script is added as the `post-checkout` git hook, if the branch
you're switching to has different migrations from the one you're coming
from, it will automatically dump the current db, and restore the dump
from the last time you visited this branch. If no dump exists, it just
leaves the DB as-is
