fleet/orbit/pkg/update
Lucas Manuel Rodriguez 61588a5ac1
Fix auto-update of .tar.gz components in orbit (#37741)
Resolves #37340.

These two issues are present on installations that used `fleetctl` (with
the `.sha512` caching optimization for `.tar.gz`) to generate the fleetd
installers.

I also recently hit this issue while releasing osqueryd to `edge` and
when releasing fleetd.

# Issue 1

First update of a `.tar.gz` component like Fleet Desktop on macOS/Linux
after installation doesn't work; second update after installation does
work:
1. Pushing a first update to TUF after the installation does the removal
of `.sha512` to `.tar.gz`, but contents are not extracted.
2. Pushing a second update to TUF after (1) does the `.tar.gz` update
and correctly updates.

How to reproduce locally:
```
# Create TUF repository
SYSTEMS="macos linux-arm64 windows-arm64" \
PKG_FLEET_URL=https://localhost:8080 \
PKG_TUF_URL=http://localhost:8081 \
DEB_FLEET_URL=https://host.docker.internal:8080 \
DEB_TUF_URL=http://host.docker.internal:8081 \
MSI_FLEET_URL=https://host.docker.internal:8080 \
MSI_TUF_URL=http://host.docker.internal:8081 \
GENERATE_PKG=1 \
GENERATE_DEB_ARM64=1 \
GENERATE_MSI_ARM64=1 \
ENROLL_SECRET=q6BjogOT6E04UmxrtZdXCE54fe89m35J \
FLEET_DESKTOP=1 \
USE_FLEET_SERVER_CERTIFICATE=1 \
DEBUG=1 \
./tools/tuf/test/main.sh

# Remove current installation in macOS.
sudo ./it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh remove

# Install the package
sudo installer -pkg fleet-osquery.pkg -target /

# Check version shown in Fleet Desktop icon (e.g. N)

# Update "Fleet Desktop" component to N+1.
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION

# Check version shown in Fleet Desktop icon, and it doesn't update (that's the bug).

# Update "Fleet Desktop" component to N+2.
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION

# Check version shown in Fleet Desktop icon, and now it updated to N+2.
```

# Issue 2

Installing on top of existing installation (re-install). Less likely to
happen but still an issue.
Re-installation of packages does not delete existing stuff at
`/opt/orbit/bin/`/`C:\Program Files\Orbit`.
So, e.g. `ls /opt/orbit/bin/desktop/macos/stable/` after a re-install
shows:
- desktop.app.tar.gz from before the installation.
- sha512 of the installed package.
- Fleet Desktop/ of the installed package..
It runs the version that came with the package, but not the updated
version.
This is fixed by a subsequent update after the re-install.

How to reproduce locally:

```
# Create TUF repository.
SYSTEMS="macos linux-arm64 windows-arm64" \
PKG_FLEET_URL=https://localhost:8080 \
PKG_TUF_URL=http://localhost:8081 \
DEB_FLEET_URL=https://host.docker.internal:8080 \
DEB_TUF_URL=http://host.docker.internal:8081 \
MSI_FLEET_URL=https://host.docker.internal:8080 \
MSI_TUF_URL=http://host.docker.internal:8081 \
GENERATE_PKG=1 \
GENERATE_DEB_ARM64=1 \
GENERATE_MSI_ARM64=1 \
ENROLL_SECRET=q6BjogOT6E04UmxrtZdXCE54fe89m35J \
FLEET_DESKTOP=1 \
USE_FLEET_SERVER_CERTIFICATE=1 \
DEBUG=1 \
./tools/tuf/test/main.sh

# Remove and install the package in macOS
sudo ./it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh remove
sudo installer -pkg fleet-osquery.pkg -target /

# Push a new update for "Fleet Desktop" (e.g. N+1).
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION

# Re-install the original installer
sudo installer -pkg fleet-osquery.pkg -target /

# Check version shown in Fleet Desktop icon, it says N instead of N+1 (that's the bug).

# A new push to TUF of N+2 fixes the issue.
```

# More info

Both issues happen also with `osqueryd` in macOS which comes bundled as
a `osqueryd.app.tar.gz`.

---

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [X] QA'd all new/changed functionality manually

## fleetd/orbit/Fleet Desktop

- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [X] Verified that fleetd runs on macOS, Linux and Windows
- [X] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed auto-update mechanism for .tar.gz components to properly manage
cached hashes and ensure stale extracted contents are cleaned up during
re-downloads following hash mismatches.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-12-30 11:17:32 -03:00
..
badgerstore Update go-tuf dependency (#3837) 2022-02-10 08:16:36 -08:00
filestore Update golangci-lint to v2.4.0 (#33251) 2025-09-22 13:17:11 -05:00
config_fetcher.go Enable installation and auto-updates of Nudge via Orbit (#9605) 2023-02-10 17:03:43 -03:00
disk_encryption.go fix issue with disk encryption banner (#21385) 2024-08-19 12:02:43 -03:00
escrow_buddy.go Revert "Always install Escrowbuddy and Swift Dialog (#28742)" (#29264) 2025-05-19 13:23:05 -04:00
escrow_buddy_test.go Revert "Always install Escrowbuddy and Swift Dialog (#28742)" (#29264) 2025-05-19 13:23:05 -04:00
execcmd.go Kickstart sofwareupdated periodically from fleetd/orbit to work around a macOS bug (#9465) 2023-01-24 10:14:17 -05:00
execcmd_darwin.go Fix bug where MDM migration fails when attempting to renew enrollment profiles on macOS Sonoma devices (#19726) 2024-06-13 14:13:43 -05:00
execcmd_stub.go don't automatically kickstart softwareupdated in Orbit (#12072) 2023-06-02 12:33:40 -03:00
execwinapi.go use OrbitNodeKey for windows mdm enrollment authentication instead of HostUUID (#13503) 2023-08-29 14:50:13 +01:00
execwinapi_stub.go Merging Bitlocker feature branch (#14350) 2023-10-06 19:04:33 -03:00
execwinapi_windows.go Update golangci-lint to v2.4.0 (#33251) 2025-09-22 13:17:11 -05:00
file.go Add 'orbit/' from commit 'ab3047bb39f1e2be331d1ff18b4eb768619033c4' 2021-08-04 16:58:25 -03:00
flag_runner.go Add arm64 support for fleetd extensions and fixes on test scripts (#31084) 2025-07-21 15:47:59 -03:00
flag_runner_test.go Update golangci-lint to v2.4.0 (#33251) 2025-09-22 13:17:11 -05:00
hash.go Fix auto-update of .tar.gz components in orbit (#37741) 2025-12-30 11:17:32 -03:00
hash_test.go chore: remove refs to deprecated io/ioutil (#14485) 2023-10-27 15:28:54 -03:00
notifications.go Fix references to enable-scripts flag in error messages and comments (#36340) 2025-11-26 13:53:57 -06:00
notifications_test.go Fix race condition in TestRenewEnrollmentProfilePrevented (#37576) 2025-12-19 22:09:26 -05:00
nudge.go Adding telemetry for specific Fleet Desktop errors (#23349) 2024-10-31 14:24:42 -05:00
nudge_test.go Add gosimple linter (#23250) 2024-10-29 14:17:51 -05:00
options.go Orbit for Windows ARM64 (#27882) 2025-04-11 10:18:28 -04:00
options_darwin.go Changes to migrate to new TUF repository (#23588) 2025-01-10 14:27:30 -03:00
options_linux_amd64.go Changes to migrate to new TUF repository (#23588) 2025-01-10 14:27:30 -03:00
options_linux_arm64.go Changes to migrate to new TUF repository (#23588) 2025-01-10 14:27:30 -03:00
options_windows_amd64.go Orbit for Windows ARM64 (#27882) 2025-04-11 10:18:28 -04:00
options_windows_arm64.go Orbit for Windows ARM64 (#27882) 2025-04-11 10:18:28 -04:00
runner.go Fleetd: Update the Registry DisplayVersion when fleetd auto-updates (#28183) 2025-04-17 11:04:02 -07:00
runner_test.go Fleetd: Update the Registry DisplayVersion when fleetd auto-updates (#28183) 2025-04-17 11:04:02 -07:00
swift_dialog.go Revert "Always install Escrowbuddy and Swift Dialog (#28742)" (#29264) 2025-05-19 13:23:05 -04:00
swift_dialog_test.go Orbit config receiver (#18518) 2024-05-09 15:22:56 -04:00
testing_utils.go use Escrow Buddy to rotate FileVault keys on macOS (#20842) 2024-07-31 16:59:30 -03:00
update.go Fix auto-update of .tar.gz components in orbit (#37741) 2025-12-30 11:17:32 -03:00
update_test.go Add support for Linux ARM64 (#19931) 2024-07-17 16:07:59 -04:00
windows_registry.go Fix Windows lint issues and enable linting on Windows (#28704) 2025-05-02 16:11:26 -04:00
windows_registry_stub.go Fleetd: Update the Registry DisplayVersion when fleetd auto-updates (#28183) 2025-04-17 11:04:02 -07:00