Resolves#37340.
These two issues are present on installations that used `fleetctl` (with
the `.sha512` caching optimization for `.tar.gz`) to generate the fleetd
installers.
I also recently hit this issue while releasing osqueryd to `edge` and
when releasing fleetd.
# Issue 1
First update of a `.tar.gz` component like Fleet Desktop on macOS/Linux
after installation doesn't work; second update after installation does
work:
1. Pushing a first update to TUF after the installation does the removal
of `.sha512` to `.tar.gz`, but contents are not extracted.
2. Pushing a second update to TUF after (1) does the `.tar.gz` update
and correctly updates.
How to reproduce locally:
```
# Create TUF repository
SYSTEMS="macos linux-arm64 windows-arm64" \
PKG_FLEET_URL=https://localhost:8080 \
PKG_TUF_URL=http://localhost:8081 \
DEB_FLEET_URL=https://host.docker.internal:8080 \
DEB_TUF_URL=http://host.docker.internal:8081 \
MSI_FLEET_URL=https://host.docker.internal:8080 \
MSI_TUF_URL=http://host.docker.internal:8081 \
GENERATE_PKG=1 \
GENERATE_DEB_ARM64=1 \
GENERATE_MSI_ARM64=1 \
ENROLL_SECRET=q6BjogOT6E04UmxrtZdXCE54fe89m35J \
FLEET_DESKTOP=1 \
USE_FLEET_SERVER_CERTIFICATE=1 \
DEBUG=1 \
./tools/tuf/test/main.sh
# Remove current installation in macOS.
sudo ./it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh remove
# Install the package
sudo installer -pkg fleet-osquery.pkg -target /
# Check version shown in Fleet Desktop icon (e.g. N)
# Update "Fleet Desktop" component to N+1.
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION
# Check version shown in Fleet Desktop icon, and it doesn't update (that's the bug).
# Update "Fleet Desktop" component to N+2.
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION
# Check version shown in Fleet Desktop icon, and now it updated to N+2.
```
# Issue 2
Installing on top of existing installation (re-install). Less likely to
happen but still an issue.
Re-installation of packages does not delete existing stuff at
`/opt/orbit/bin/`/`C:\Program Files\Orbit`.
So, e.g. `ls /opt/orbit/bin/desktop/macos/stable/` after a re-install
shows:
- desktop.app.tar.gz from before the installation.
- sha512 of the installed package.
- Fleet Desktop/ of the installed package..
It runs the version that came with the package, but not the updated
version.
This is fixed by a subsequent update after the re-install.
How to reproduce locally:
```
# Create TUF repository.
SYSTEMS="macos linux-arm64 windows-arm64" \
PKG_FLEET_URL=https://localhost:8080 \
PKG_TUF_URL=http://localhost:8081 \
DEB_FLEET_URL=https://host.docker.internal:8080 \
DEB_TUF_URL=http://host.docker.internal:8081 \
MSI_FLEET_URL=https://host.docker.internal:8080 \
MSI_TUF_URL=http://host.docker.internal:8081 \
GENERATE_PKG=1 \
GENERATE_DEB_ARM64=1 \
GENERATE_MSI_ARM64=1 \
ENROLL_SECRET=q6BjogOT6E04UmxrtZdXCE54fe89m35J \
FLEET_DESKTOP=1 \
USE_FLEET_SERVER_CERTIFICATE=1 \
DEBUG=1 \
./tools/tuf/test/main.sh
# Remove and install the package in macOS
sudo ./it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh remove
sudo installer -pkg fleet-osquery.pkg -target /
# Push a new update for "Fleet Desktop" (e.g. N+1).
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION
# Re-install the original installer
sudo installer -pkg fleet-osquery.pkg -target /
# Check version shown in Fleet Desktop icon, it says N instead of N+1 (that's the bug).
# A new push to TUF of N+2 fixes the issue.
```
# More info
Both issues happen also with `osqueryd` in macOS which comes bundled as
a `osqueryd.app.tar.gz`.
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [X] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [X] Verified that fleetd runs on macOS, Linux and Windows
- [X] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed auto-update mechanism for .tar.gz components to properly manage
cached hashes and ensure stale extracted contents are cleaned up during
re-downloads following hash mismatches.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35852
Tested by adding a small(100ms but even smaller should work) sleep in
the goroutine before calling renewReceiver.Run() which simulates the
active goroutine being preempted and the other running before it gets
scheduled again. When I did this it would hang and timeout every time
before the fix. AFter the fix I never saw a timeout over 500 runs with
the sleep added and without
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#35548
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually (QAed on macOS)
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows (Not
manually checked, but this change should not impact it)
Fixes#32239.
This changes tags to return a comma-delimited list on multiple tags, the
single tag when there's only one, and "is not set" (similar to other
values) when no tags are set.
Confirmed that this allows us to run `SELECT * FROM falconctl_options`
without issue on various configurations of Crowdstrike Falcon on Linux.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
Fixes#33967, #33193, #35149.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [ ] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux (skipped WIndows due to
runtime.GOOS gating)
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
**Related issue:** Fixes#32902
This changes the error message swift dialog popup for macos mdm
migration. it will not correctly show file protocol URLs and makes the
"Contact IT" the primary button action on this popup.
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
---------
Co-authored-by: Jordan Montgomery <elijah.jordan.montgomery@gmail.com>
**Related issue:** Resolves#34330
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually (so far just macOS)
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added a built-in mcp_listening_servers table to discover MCP servers
by inspecting listening ports and probing endpoints; returns process
info, server metadata, capabilities, tools, prompts, and resources
(supports macOS, Windows, Linux).
* **Tests**
* Added comprehensive unit tests covering detection, IPv6 handling, SSE
responses, and session lifecycle.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#34596
# Details
This PR adds back the "legacy" macOS setup experience, and a new
`CapabilityMacOSWebSetupExperience` capability check. The legacy
experience will be shown whenever the
`CapabilityMacOSWebSetupExperience` capability is _not_ detected in the
Fleet server response.
The majority of the code is just copying the `Run` and
`startSwiftDialog` from the [4.75.1 patch
release](https://github.com/fleetdm/fleet/blob/rc-patch-fleet-v4.75.1/orbit/pkg/setup_experience/setup_experience.go)
and adding them back as `RunLegacy` and `startSwiftDialogLegacy`, then
adding the little bit of plumbing to have `Run()` call `RunLegacy()`
when the new capability is not detected.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
## Testing
- [ ] Added/updated automated tests
- [ ] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [X] QA'd all new/changed functionality manually
Tested on a VM that the setup experience starts in "legacy" mode if
`CapabilityMacOSWebSetupExperience` is not set in the server, and that
it starts in the new web view if the capability _is_ set.
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
- [ ] Alerted the release DRI if additional load testing is needed
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
I have not re-verified this, but the changes will only run on macOS
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
This should not be affected.
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33173
**Related issue:** Resolves#33111
# Details
This is the remaining work to implement the "Stop the setup experience
when required software fails to install" feature. This didn't turn out
to be quite as straightforward as expected so I ended up doing a bit of
design-by-code and expect some feedback on the approach. I tried to make
it as low-touch as possible. The general design is:
1. In the `maybeUpdateSetupExperienceStatus` function which is called in
various places when a setup experience step is marked as completed, call
a new `maybeCancelPendingSetupExperienceSteps` function if the setup
step was marked as failed. Similarly call
`maybeCancelPendingSetupExperienceSteps` if a VPP app install fails to
enqueue.
2. In `maybeCancelPendingSetupExperienceSteps`, check whether the
specified host is MacOS and whether the "RequireAllSoftwareMacOS" flag
is set in the team (or global) config. If so, mark the remaining setup
experience items as canceled and cancel any upcoming activities related
to those steps.
3. On the front-end, if the `require_all_software_macos` is set and a
software step is marked as failed, show a new failure page indicating
that setup has failed and showing details of the failed software.
4. On the agent side, when checking setup experience status, send a
`reset_after_failure` flag _only the first time_. If this flag is set,
then the code in the `/orbit/setup_experience/status` handler will clear
and re-queue any failed setup experience steps (but leave successful
steps to avoid re-installing already-installed software). This
facilitates re-starting the setup experience when the host is rebooted.
I also updated the way that software (packages and VPP) is queued up for
the setup experience to be ordered alphabetically, to make it easier to
test _and_ because this is a desired outcome for a future story. Since
the order is not deterministic now, this update shouldn't cause any
problems (aside from a couple of test updates), but I'm ok taking it out
if desired.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [X] Added/updated automated tests
* Added a new integration test for software packages, testing that a
failed software package causes the rest of the setup experience to be
marked as failed when `require_all_software_macos` is set, and testing
that the "reset after failure" code works.
* Added a new integration test for VPP packages, testing that a failed
VPP enqueue causes the same halting of the setup experience.
I _don't_ have test for a failure _during_ a VPP install. It should go
through the same code path as the software package failure, so it's not
a huge gap.
- [ ] QA'd all new/changed functionality manually
Working on it
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [X] Verified that fleetd runs on macOS, Linux and Windows
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Configurable option to halt macOS device setup if any software install
fails.
- Device setup page now shows a clear “Device setup failed” state with
expandable error details when all software is required on macOS.
- Improvements
- Setup status now includes per-step error messages for better
troubleshooting.
- Pending setup steps are automatically canceled after a failure when
applicable, with support to reset and retry the setup flow as
configured.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #33111
Fixes a possible race condition introduced in
https://github.com/fleetdm/fleet/issues/33884 which saw a test failure
[here](https://github.com/fleetdm/fleet/actions/runs/18454713591/job/52574112376#step:14:3571).
# Checklist for submitter
## Testing
- [ ] Added/updated automated tests
Existing test is sufficient, will circle back if it fails again.
- [X] QA'd all new/changed functionality manually
Verified that Fleet Desktop still opens My Device page correctly and
token rotation logs are still seen.
For unreleased bug fixes in a release candidate, one of:
- [X] Confirmed that the fix is not expected to adversely impact load
test results
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** For #33111
# Details
This PR updates the setup experience for MacOS to use a web view pointed
at the device's "Setting up your device" page rather than using native
MacOS UI elements, bringing it more in line with Linux and Windows setup
experiences.
This covers only the new web UI for the setup experience progress, _not_
the UI for the new case of blocking the device when a piece of software
fails to install. I'll add that in a separate PR.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [X] Added/updated automated tests
Added tests for the updates to the token rotation code.
- [X] QA'd all new/changed functionality manually
A new tool is provided to allow testing this code against a virtual
machine if a separate host that you can wipe and run setup on is not
available. See
https://github.com/fleetdm/fleet/blob/sgress454/new-setup-experience/tools/mdm/apple/setupexperience/README.md
for details.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- macOS setup experience moved to a new web-based UI.
- Automatic device token rotation during setup to keep sessions valid.
- Bug Fixes
- More reliable setup flow with improved dialog lifecycle and cleaner
handoff to web content.
- Dialog elements hidden/cleared appropriately when transitioning to the
browser.
- Documentation
- Added guide and tool to simulate the macOS setup experience on a VM,
with prerequisites and usage steps.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- Add the related story/sub-task/bug number, like Resolves#123, or
remove if NA -->
**Related issue:** Resolves#33250
Waived most new failures. Planning to come back and fix some of them in
subsequent PRs.
Fixes#32580
- Added retry logic for software installs
- Added sending intermediate results to Fleet server
I QA'd this on Linux (see video below). For macOS and Windows, I QA'd it
by having the server force retries on normal (non-setup experience
software installs).
Demo video: https://youtu.be/dbu78G6bXf8
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Automatic retries for software installations, with exponential backoff
on transient/network errors.
- Intermediate failures are recorded without closing the original
request; subsequent attempts continue automatically.
- Activity feed entries are created for intermediate failures, including
install identifiers.
- Setup experience installs now retry automatically (up to 3 attempts).
- Tests
- Expanded test coverage for retry behavior, error classification, and
intermediate failure reporting.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #32788.
## Testing
- [X] Added/updated automated tests
- [X] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [X] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- New Features
- Enhanced Linux setup experience: persists status on disk, resumes
automatically, and completes when software/scripts finish.
- Opens the “My Device” page only when desktop is enabled, using a
user-aware launcher on Linux.
- Linux setup status now focuses on software progress for faster,
clearer feedback.
- Bug Fixes
- Corrected auth/MDM checks: macOS requires Apple MDM; Linux no longer
blocked by MDM configuration on shared endpoints.
- Improved reliability and logging around software installation and
temporary directory cleanup.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#31934
Manually QA'd using a Migration Assistant flow from one macOS VM to
another.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* macOS: Prevents duplicate hosts by persisting the hardware UUID and
triggering re-enrollment when it changes (e.g., after hardware migration
or system restore). Improves reliability across restarts and cleans up
legacy data during migration.
* **Documentation**
* Added changelog entry noting the macOS-specific fix for duplicate
enrolled hosts (UUID/SN).
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32280
- Removed osqueryd.tar.gz from macOS package and desktop.tar.gz from
macOS and Linux packages and replaced them with .sha512 hash caches.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Eliminated duplicate osqueryd and Fleet Desktop binaries in Linux and
macOS packages, preventing duplicate entries in .deb/.pkg and ensuring
cleaner installs.
* **Chores**
* Added packaging cleanup to remove leftover tar.gz artifacts, reducing
package size and avoiding accidental inclusion in builds.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
For #31675
For #32099
Adds a Github workflow to generate our packaged build of Swift
Dialog(following existing Nudge packager), updates the version to 2.5.6
and modifies the Migration dialog to render properly with the new Swift
Dialog version(it previously rendered it just didn't format as expected
due to changes in the markdown formatter)
Makefile changes are necessary not only because of the version bump but
because the latest package includes xattrs for some strange reason.
Extracting it verbatim on a system(at least with our Go implementation)
creates files that cause Gatekeeper to stop execution of swift dialog.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
for #32112
# Details
This PR reverts some earlier work using `runuser` and `runcon` intended
to allow Orbit to launch Fleet Desktop in the logged-in user's SELinux
context. This didn't work for out-of-the-box SELinux enforcement setups,
with a side-effect of each failed attempt to launch the desktop app
creating a new kernel keyring that doesn't get cleaned up until Orbit
quits (or GC runs, although I didn't see that happen in testing). While
using `runuser` has some possible benefits over using `sudo` to launch
processes, it also (when using `-l` to start a login shell) creates a
new keyring. This is not an issue if the command to launch the process
succeeds, but if it fails and retries over and over, we start amassing
keyrings.
It is the opinion of several other engineers (and now myself as well)
that the real solution to various Fleet Desktop launching issues is to
launch the desktop app as a user service using `systemctl --user` so
that it automatically inherits the user's environment, context, etc.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
# Testing
- [X] QA'd all new/changed functionality manually
- [ ] ~Ubuntu with SELinux on~ this is a very uncommon setup
- [x] Ubuntu with SELinux off
- [X] Fedora with SELinux on - double checked that `getenforce` returned
`Enforcing` this time
- [X] Fedora with SELinux off
- [ ] ~Debian with SELinux on~ this is a very uncommon setup
- [X] Debian with SELinux off
- [x] `runWithOutput` still works (tested with `go run
./tools/dialog/main.go --dialog=zenity` on all platforms)
- [ ] ~`runWithStdin` still works~ (this isn't currently used by Linux)
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [X] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
Fixes#31338, #31525.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [s] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [ ] Verified that fleetd runs on macOS, Linux and Windows
- [ ] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
For #30476
Contributor doc updates: https://github.com/fleetdm/fleet/pull/31371
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] Added/updated automated tests
- [x] Where appropriate, [automated tests simulate multiple hosts and
test for host
isolation](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/reference/patterns-backend.md#unit-testing)
(updates to one hosts's records do not affect another)
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Automated certificate renewal is now supported, including
proof-of-possession for enhanced security.
* Certificate renewal can be triggered when the existing certificate is
within 180 days of expiration.
* Dynamic configuration of certificate validity period via environment
variable.
* Improved TPM hardware integration for certificate management.
* **Bug Fixes**
* Enhanced error handling and logging for TPM device closure and
certificate operations.
* **Tests**
* Extended integration tests to cover certificate renewal flows, host
deletion, and TPM-based scenarios for improved reliability.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Admins _should_ be upgrading orbit/osqueryd/fleet-desktop using TUF, but
there's no official path for pushing updates to the orbit environment
file (`/etc/default/orbit`).
Knowing that this file is installed by fleet-osquery, I naively pushed a
new fleet-osquery package to a user's machine, thinking that would be
fine installing over the existing package. Instead it actually broke
orbit entirely on the host, rendering it unreachable until the user
manually reinstalled the package. This is because the pre-removal script
unconditionally stops orbit even if it's being upgraded, and when orbit
is stopped it terminates any processes underneath it, including the
ongoing package installation.
To workaround this, we replace the simple
`systemctl restart orbit.service` with a check for the `INSTALLER_PATH`
environment variable that orbit sets during software installations. If
the variable is present, `systemd-run` is used to schedule the service
restart 60 seconds in the future, which is assumed to be more than
enough time for the package manager to finish and exit. Unfortunately,
this bugfix cannot be made retroactive, because the broken version of
the prerm script is called before the new package starts to be
unpacked/installed.
Although there are other ways of doing anything an administrator might
be trying to accomplish by pushing a new fleet-osquery package, bricking
an endpoint simply by pushing this package through the Software page
seems like a pretty massive footgun that is easy to protect against.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for safe in-band upgrades of DEB and RPM packages
generated by the `fleetctl package` command via the Software page, after
an initial manual update.
* **Documentation**
* Added a note clarifying the new upgrade process and the need for a
one-time manual update before using in-band upgrades.
* **Bug Fixes**
* Improved upgrade scripts to prevent the Orbit service from stopping
unexpectedly during package upgrades, ensuring smoother and safer
updates.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Dan Fuhry <dan@fuhry.com>
For #29793
# Details
This PR changes the way that Orbit launches processes such as the
desktop app on Linux, in order to ensure that on SELinux-enabled systems
the correct user context is set when running the command.
Previously, `sudo -u` was used to launch commands on Linux. This PR
switches to use `runuser` instead, which is recommended in situations
where the root user wants to execute a command as a user with reduced
privileges (see [the blog post by one of the creators of
runuser](https://danwalsh.livejournal.com/55588.html)). This avoids
certain errors that can come from interacting with PAM modules as the
system user.
Additionally, if we detect that SELinux is set up on a system, we now
use `runcon` to force the command to run using the logged-in user's
SELinux context. It's possible that on some systems they may have
configuration where `sudo` will switch to the user's SELinux context
automatically, but this is not guaranteed. Using `runuser` + `runcon` is
our best bet for ensuring that the desktop app (and anything that it
spawns) runs under the correct context.
This PR also does some refactoring so that the three `run` methods for
Linux (`run`, `runWithOutput` and `runWithStdin`) all use the same base
code to create the command with the correct args and env vars, and
differ only in how they handle the i/o.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- For Orbit and Fleet Desktop changes:
- [x] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] ~Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.~ (n/a, code is linux only)
- [x] ~Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).~ n/a
# Testing
- [x] Ubuntu with SELinux on
- [x] Ubuntu with SELinux off
- [ ] Fedora with SELinux on
- [ ] Fedora with SELinux off
- [ ] Debian with SELinux on
- [x] Debian with SELinux off
- [x] `runWithOutput` still works (tested with `go run
./tools/dialog/main.go --dialog=zenity`)
- [ ] ~`runWithStdin` still works~ (this isn't currently used by Linux)
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved security and user context handling when launching the fleet
desktop application on Linux systems.
* **Refactor**
* Enhanced process launch mechanism to use proper SELinux context and
user session, ensuring processes start under the correct user and
security environment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This was required to test https://github.com/fleetdm/fleet/pull/30864 on
Apple Silicon.
I've created https://github.com/fleetdm/fleet/issues/31092 for tracking
purposes.
Fixes:
- Build univeral binary extension on macOS to test on VMs without
Rosetta.
- Add support for linux and Windows arm64. Which is also needed to test
Linux and Windows on UTM on Apple Silicon.
- Add Linux arm64 & Windows arm64 to the test scripts.
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
#30461
This PR contains the changes for the happy path.
On a separate PR we will be adding tests and further fixes for edge
cases.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* Added support for using a TPM-backed key and SCEP-issued certificate
to sign HTTP requests, enhancing security through hardware-based key
management.
* Introduced new CLI and environment flags to enable TPM-backed client
certificates for Linux packages and Orbit.
* Added a local HTTPS proxy that automatically signs requests using the
TPM-backed key.
* **Bug Fixes**
* Improved cleanup and restart behavior when authentication fails with a
host identity certificate.
* **Tests**
* Added comprehensive tests for SCEP client functionality and TPM
integration.
* **Chores**
* Updated scripts and documentation to support TPM-backed client
certificate packaging and configuration.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
for #29942
# Details
This PR addresses an issue on Linux where Orbit repeatedly attempts to
launch Fleet Desktop even though no GUI user is logged in. The fix is
similar to one implemented for MacOS, where we have Orbit check for the
presence of a real user (not a system user like `gdm` or `root`) before
trying to launch the desktop app.
Part of this work involved moving some functionality from the `execuser`
package to the `user` package, to avoid duplicating functionality.
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
The changed code is only executed on Linux, so I tested on Ubuntu,
Fedora and Debian. Also verified that it still works on MacOS and
Windows.
---------
Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
#28621
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [X] Added/updated automated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
Fixes#29581
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [ ] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [ ] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [ ] Added support on fleet's osquery simulator `cmd/osquery-perf` for
new osquery data ingestion features.
- [ ] If paths of existing endpoints are modified without backwards
compatibility, checked the frontend/CLI for any necessary changes
- [ ] If database migrations are included, checked table schema to
confirm autoupdate
- For new Fleet configuration settings
- [ ] Verified that the setting can be managed via GitOps, or confirmed
that the setting is explicitly being excluded from GitOps. If managing
via Gitops:
- [ ] Verified that the setting is exported via `fleetctl
generate-gitops`
- [ ] Added the setting to [the GitOps
documentation](https://github.com/fleetdm/fleet/blob/main/docs/Configuration/yaml-files.md#L485)
- [ ] Verified that the setting is cleared on the server if it is not
supplied in a YAML file (or that it is documented as being optional)
- [ ] Verified that any relevant UI is disabled when GitOps mode is
enabled
- For database migrations:
- [ ] Checked schema for all modified table for columns that will
auto-update timestamps during migration.
- [ ] Confirmed that updating the timestamps is acceptable, and will not
cause unwanted side effects.
- [ ] Ensured the correct collation is explicitly set for character
columns (`COLLATE utf8mb4_unicode_ci`).
- [ ] Added/updated automated tests
- [ ] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [ ] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md)).
- [ ] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
- [ ] For unreleased bug fixes in a release candidate, confirmed that
the fix is not expected to adversely impact load test results or alerted
the release DRI if additional load testing is needed.
