Resolves#37340.
These two issues are present on installations that used `fleetctl` (with
the `.sha512` caching optimization for `.tar.gz`) to generate the fleetd
installers.
I also recently hit this issue while releasing osqueryd to `edge` and
when releasing fleetd.
# Issue 1
First update of a `.tar.gz` component like Fleet Desktop on macOS/Linux
after installation doesn't work; second update after installation does
work:
1. Pushing a first update to TUF after the installation does the removal
of `.sha512` to `.tar.gz`, but contents are not extracted.
2. Pushing a second update to TUF after (1) does the `.tar.gz` update
and correctly updates.
How to reproduce locally:
```
# Create TUF repository
SYSTEMS="macos linux-arm64 windows-arm64" \
PKG_FLEET_URL=https://localhost:8080 \
PKG_TUF_URL=http://localhost:8081 \
DEB_FLEET_URL=https://host.docker.internal:8080 \
DEB_TUF_URL=http://host.docker.internal:8081 \
MSI_FLEET_URL=https://host.docker.internal:8080 \
MSI_TUF_URL=http://host.docker.internal:8081 \
GENERATE_PKG=1 \
GENERATE_DEB_ARM64=1 \
GENERATE_MSI_ARM64=1 \
ENROLL_SECRET=q6BjogOT6E04UmxrtZdXCE54fe89m35J \
FLEET_DESKTOP=1 \
USE_FLEET_SERVER_CERTIFICATE=1 \
DEBUG=1 \
./tools/tuf/test/main.sh
# Remove current installation in macOS.
sudo ./it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh remove
# Install the package
sudo installer -pkg fleet-osquery.pkg -target /
# Check version shown in Fleet Desktop icon (e.g. N)
# Update "Fleet Desktop" component to N+1.
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION
# Check version shown in Fleet Desktop icon, and it doesn't update (that's the bug).
# Update "Fleet Desktop" component to N+2.
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION
# Check version shown in Fleet Desktop icon, and now it updated to N+2.
```
# Issue 2
Installing on top of existing installation (re-install). Less likely to
happen but still an issue.
Re-installation of packages does not delete existing stuff at
`/opt/orbit/bin/`/`C:\Program Files\Orbit`.
So, e.g. `ls /opt/orbit/bin/desktop/macos/stable/` after a re-install
shows:
- desktop.app.tar.gz from before the installation.
- sha512 of the installed package.
- Fleet Desktop/ of the installed package..
It runs the version that came with the package, but not the updated
version.
This is fixed by a subsequent update after the re-install.
How to reproduce locally:
```
# Create TUF repository.
SYSTEMS="macos linux-arm64 windows-arm64" \
PKG_FLEET_URL=https://localhost:8080 \
PKG_TUF_URL=http://localhost:8081 \
DEB_FLEET_URL=https://host.docker.internal:8080 \
DEB_TUF_URL=http://host.docker.internal:8081 \
MSI_FLEET_URL=https://host.docker.internal:8080 \
MSI_TUF_URL=http://host.docker.internal:8081 \
GENERATE_PKG=1 \
GENERATE_DEB_ARM64=1 \
GENERATE_MSI_ARM64=1 \
ENROLL_SECRET=q6BjogOT6E04UmxrtZdXCE54fe89m35J \
FLEET_DESKTOP=1 \
USE_FLEET_SERVER_CERTIFICATE=1 \
DEBUG=1 \
./tools/tuf/test/main.sh
# Remove and install the package in macOS
sudo ./it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh remove
sudo installer -pkg fleet-osquery.pkg -target /
# Push a new update for "Fleet Desktop" (e.g. N+1).
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION
# Re-install the original installer
sudo installer -pkg fleet-osquery.pkg -target /
# Check version shown in Fleet Desktop icon, it says N instead of N+1 (that's the bug).
# A new push to TUF of N+2 fixes the issue.
```
# More info
Both issues happen also with `osqueryd` in macOS which comes bundled as
a `osqueryd.app.tar.gz`.
---
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
## Testing
- [X] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [X] Verified that fleetd runs on macOS, Linux and Windows
- [X] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Fixed auto-update mechanism for .tar.gz components to properly manage
cached hashes and ensure stale extracted contents are cleaned up during
re-downloads following hash mismatches.
<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Fixes#32280
- Removed osqueryd.tar.gz from macOS package and desktop.tar.gz from
macOS and Linux packages and replaced them with .sha512 hash caches.
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
## Testing
- [x] QA'd all new/changed functionality manually
## fleetd/orbit/Fleet Desktop
- [x] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [x] Verified that fleetd runs on macOS, Linux and Windows
- [x] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Eliminated duplicate osqueryd and Fleet Desktop binaries in Linux and
macOS packages, preventing duplicate entries in .deb/.pkg and ensuring
cleaner installs.
* **Chores**
* Added packaging cleanup to remove leftover tar.gz artifacts, reducing
package size and avoiding accidental inclusion in builds.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
