fleet/orbit/pkg/update/hash.go
Lucas Manuel Rodriguez 61588a5ac1
Fix auto-update of .tar.gz components in orbit (#37741)
Resolves #37340.

These two issues are present on installations that used `fleetctl` (with
the `.sha512` caching optimization for `.tar.gz`) to generate the fleetd
installers.

I also recently hit this issue while releasing osqueryd to `edge` and
when releasing fleetd.

# Issue 1

First update of a `.tar.gz` component like Fleet Desktop on macOS/Linux
after installation doesn't work; second update after installation does
work:
1. Pushing a first update to TUF after the installation does the removal
of `.sha512` to `.tar.gz`, but contents are not extracted.
2. Pushing a second update to TUF after (1) does the `.tar.gz` update
and correctly updates.

How to reproduce locally:
```
# Create TUF repository
SYSTEMS="macos linux-arm64 windows-arm64" \
PKG_FLEET_URL=https://localhost:8080 \
PKG_TUF_URL=http://localhost:8081 \
DEB_FLEET_URL=https://host.docker.internal:8080 \
DEB_TUF_URL=http://host.docker.internal:8081 \
MSI_FLEET_URL=https://host.docker.internal:8080 \
MSI_TUF_URL=http://host.docker.internal:8081 \
GENERATE_PKG=1 \
GENERATE_DEB_ARM64=1 \
GENERATE_MSI_ARM64=1 \
ENROLL_SECRET=q6BjogOT6E04UmxrtZdXCE54fe89m35J \
FLEET_DESKTOP=1 \
USE_FLEET_SERVER_CERTIFICATE=1 \
DEBUG=1 \
./tools/tuf/test/main.sh

# Remove current installation in macOS.
sudo ./it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh remove

# Install the package
sudo installer -pkg fleet-osquery.pkg -target /

# Check version shown in Fleet Desktop icon (e.g. N)

# Update "Fleet Desktop" component to N+1.
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION

# Check version shown in Fleet Desktop icon, and it doesn't update (that's the bug).

# Update "Fleet Desktop" component to N+2.
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION

# Check version shown in Fleet Desktop icon, and now it updated to N+2.
```

# Issue 2

Installing on top of existing installation (re-install). Less likely to
happen but still an issue.
Re-installation of packages does not delete existing stuff at
`/opt/orbit/bin/`/`C:\Program Files\Orbit`.
So, e.g. `ls /opt/orbit/bin/desktop/macos/stable/` after a re-install
shows:
- desktop.app.tar.gz from before the installation.
- sha512 of the installed package.
- Fleet Desktop/ of the installed package..
It runs the version that came with the package, but not the updated
version.
This is fixed by a subsequent update after the re-install.

How to reproduce locally:

```
# Create TUF repository.
SYSTEMS="macos linux-arm64 windows-arm64" \
PKG_FLEET_URL=https://localhost:8080 \
PKG_TUF_URL=http://localhost:8081 \
DEB_FLEET_URL=https://host.docker.internal:8080 \
DEB_TUF_URL=http://host.docker.internal:8081 \
MSI_FLEET_URL=https://host.docker.internal:8080 \
MSI_TUF_URL=http://host.docker.internal:8081 \
GENERATE_PKG=1 \
GENERATE_DEB_ARM64=1 \
GENERATE_MSI_ARM64=1 \
ENROLL_SECRET=q6BjogOT6E04UmxrtZdXCE54fe89m35J \
FLEET_DESKTOP=1 \
USE_FLEET_SERVER_CERTIFICATE=1 \
DEBUG=1 \
./tools/tuf/test/main.sh

# Remove and install the package in macOS
sudo ./it-and-security/lib/macos/scripts/uninstall-fleetd-macos.sh remove
sudo installer -pkg fleet-osquery.pkg -target /

# Push a new update for "Fleet Desktop" (e.g. N+1).
source ./tools/tuf/test/load_orbit_version_vars.sh
echo $ORBIT_VERSION
FLEET_DESKTOP_VERSION=$ORBIT_VERSION make desktop-app-tar-gz
./tools/tuf/test/push_target.sh macos desktop desktop.app.tar.gz $ORBIT_VERSION

# Re-install the original installer
sudo installer -pkg fleet-osquery.pkg -target /

# Check version shown in Fleet Desktop icon, it says N instead of N+1 (that's the bug).

# A new push to TUF of N+2 fixes the issue.
```

# More info

Both issues happen also with `osqueryd` in macOS which comes bundled as
a `osqueryd.app.tar.gz`.

---

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.

## Testing

- [X] QA'd all new/changed functionality manually

## fleetd/orbit/Fleet Desktop

- [X] Verified compatibility with the latest released version of Fleet
(see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/workflows/fleetd-development-and-release-strategy.md))
- [X] If the change applies to only one platform, confirmed that
`runtime.GOOS` is used as needed to isolate changes
- [X] Verified that fleetd runs on macOS, Linux and Windows
- [X] Verified auto-update works from the released version of component
to the new version (see [tools/tuf/test](../tools/tuf/test/README.md))


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed auto-update mechanism for .tar.gz components to properly manage
cached hashes and ensure stale extracted contents are cleaned up during
re-downloads following hash mismatches.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-12-30 11:17:32 -03:00

98 lines
2.7 KiB
Go

package update
import (
"bytes"
"crypto/sha256"
"crypto/sha512"
"encoding/hex"
"fmt"
"hash"
"io"
"os"
"strings"
"github.com/rs/zerolog/log"
"github.com/theupdateframework/go-tuf/data"
)
// checkFileHash checks the file at the local path against the provided hash functions.
func checkFileHash(meta *data.TargetFileMeta, localPath string) error {
metaHash, localHash, err := fileHashes(meta, localPath)
if err != nil {
return fmt.Errorf("failed to calculate local file hash: %s", err)
}
if !bytes.Equal(localHash, metaHash) {
return fmt.Errorf("hash %x does not match expected: %x", localHash, metaHash)
}
return nil
}
func fileHashes(meta *data.TargetFileMeta, localPath string) (metaHash []byte, localHash []byte, err error) {
hashFn, metaHash, err := selectHashFunction(meta)
if err != nil {
return nil, nil, err
}
// For .tar.gz components, try cached hash file first.
if strings.HasSuffix(localPath, ".tar.gz") {
cachedHash, err := readCachedHash(localPath, meta)
switch {
case err == nil:
return metaHash, cachedHash, nil
case os.IsNotExist(err):
// OK
default:
log.Info().Err(err).Msg("failed to read cached hash file")
}
}
f, err := os.Open(localPath)
if err != nil {
return nil, nil, fmt.Errorf("open file for hash: %w", err)
}
defer f.Close()
if _, err := io.Copy(hashFn, f); err != nil {
return nil, nil, fmt.Errorf("read file for hash: %w", err)
}
return metaHash, hashFn.Sum(nil), nil
}
// selectHashFunction returns the first matching hash function and expected
// hash, otherwise returning an error if no matching hash can be found.
//
// SHA512 is preferred, and SHA256 is returned if 512 is not available.
func selectHashFunction(meta *data.TargetFileMeta) (hash.Hash, []byte, error) {
for hashName, hashVal := range meta.Hashes {
if hashName == "sha512" {
return sha512.New(), hashVal, nil
}
}
for hashName, hashVal := range meta.Hashes {
if hashName == "sha256" {
return sha256.New(), hashVal, nil
}
}
return nil, nil, fmt.Errorf("no matching hash function found: %v", meta.HashAlgorithms())
}
// readCachedHash reads a cached hash from a .sha512 file
// created during packaging when the tar.gz was removed to save space.
func readCachedHash(tarGzPath string, meta *data.TargetFileMeta) ([]byte, error) {
// Check if TUF metadata has SHA512 (currently the only hash file used)
for hashName := range meta.Hashes {
if hashName == "sha512" {
hashPath := tarGzPath + ".sha512"
var hashHex []byte
var err error
if hashHex, err = os.ReadFile(hashPath); err != nil {
return nil, err
}
return hex.DecodeString(strings.TrimSpace(string(hashHex)))
}
}
return nil, fmt.Errorf("no cached hash file found for %s", tarGzPath)
}