## Summary
The nightly OSV artifact generation in `fleetdm/vulnerabilities` failed
over the weekend with:
```
fatal: error processing shallow info: 4
```
at `cmd/osv-processor/sync-and-detect-changes.sh` during:
```bash
git fetch --shallow-since="3 days ago" origin main
```
Root cause: `git fetch --shallow-since` errors out when the upstream
(`canonical/ubuntu-security-notices`) has zero commits newer than the
cutoff. Canonical didn't push anything over the weekend, so the 3-day
window returned empty and upload-pack produced an unusable shallow
response.
Fix:
- Fall back to `git fetch --depth=3` if `--shallow-since` still returns
empty, so the initial clone always succeeds.
Subsequent runs reuse the existing clone and take the other branch of
the script (plain `git fetch origin main`), which doesn't have this
failure mode.
Failing run:
https://github.com/fleetdm/vulnerabilities/actions/runs/24330589309/job/71035337352
## Test plan
- [x] Re-run the Ubuntu OSV artifact generation workflow; initial clone
succeeds regardless of upstream push frequency.
- [x] Manually exercise the cold-cache path locally: `rm -rf
ubuntu-security-notices &&
./cmd/osv-processor/sync-and-detect-changes.sh` — completes without
error.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Improved initial repository sync: if the primary shallow fetch returns
no commits, the process now falls back to a limited-depth fetch, warns
the user, and shows recent commit history before continuing. Downstream
change detection and existing behavior for already-cloned repos remain
unchanged.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
---------
Co-authored-by: Konstantin Sykulev <konst@sykulev.com>
**Related issue:** Resolves#40057
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/guides/committing-changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements), JS
inline code is prevented especially for url redirects, and untrusted
data interpolated into shell scripts/commands is validated against shell
metacharacters.
## Testing
- [x] Added/updated automated tests
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **New Features**
* OSV (Open Source Vulnerabilities) added as an optional Ubuntu
vulnerability data source and enabled by default.
* **Features**
* Integrated OSV into the vulnerability scanning pipeline, artifact
sync/refresh, detection, and cleanup flows.
* Improved Ubuntu package/kernel version matching for more accurate OSV
detections.
* **Chores**
* Added configuration flag and updated expected config fixtures.
* **Tests**
* Added extensive tests for OSV sync, artifact handling, analyzer logic,
and cleanup behaviors.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
**Related issue:** Resolves#41571
Changing the way delta artifacts are generated.
`changed_files_today.txt` and `changed_files_yesterday.txt` will always
look back into git history for any commits added yesterday and today.
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
## Release Notes
* **Bug Fixes**
* Improved repository synchronization to ensure local state accurately
reflects the remote main branch.
* Enhanced file change detection logic for more reliable identification
of modified files.
* Refined output file handling to maintain consistency across
synchronization cycles.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Related issue: Resolves https://github.com/fleetdm/fleet/issues/41571
It appears that there is some sort of error with shallow cloning.
```
Run echo "=== Generating OSV Artifacts for Ubuntu ==="
=== Generating OSV Artifacts for Ubuntu ===
=== OSV Repository Sync ===
Repository exists, updating with rolling window...
fatal: error processing shallow info: 4
Error: Process completed with exit code 128.
```
Since we are only keeping a limited history of the repository via cache
before re-clone, fall back to doing a regular `git pull`. This avoids
the complicated shallow cloning / Git having to reconcile the
overlapping but different shallow boundaries, which can cause "error
processing shallow info: 4".
## Testing
- [x] QA'd all new/changed functionality manually
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Refactor**
* Modified repository synchronization to use full fetches instead of
rolling-window shallow fetches.
* Updated sync status messaging for clarity.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
