Commit graph

634 commits

Author SHA1 Message Date
Lucas Manuel Rodriguez
5da0008872
Fix code scanning alert for not pinned actions (#28305)
Fixes https://github.com/fleetdm/fleet/security/code-scanning/1381.
2025-04-16 16:35:13 -05:00
Lucas Manuel Rodriguez
243ca206bf
Fix tagging of fleetdm/fleet and fleetdm/fleetctl when releasing (#28251)
For #26988.
2025-04-16 15:48:47 -03:00
Lucas Manuel Rodriguez
68ee136e3b
Release osqueryd 5.17.0 (#28294) 2025-04-16 15:25:26 -03:00
Lucas Manuel Rodriguez
3c86055139
Add wait until fleet is responding (#28241)
For #26349.
2025-04-16 13:11:03 -03:00
Lucas Manuel Rodriguez
895194d63b
Add scanning to released images and process to track vulnerabilities (#28087)
For #25902.

---------

Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
2025-04-16 11:50:10 -03:00
Victor Lyuboslavsky
229b51ff5b
Report aggregate results immediately after test-go matrix runs. (#28123)
Report aggregate results immediately after test-go matrix runs
2025-04-11 14:05:19 -05:00
Dante Catalfamo
94f6127edc
Orbit for Windows ARM64 (#27882)
#27275 and #27274

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [x] Make sure fleetd is compatible with the latest released version of
Fleet (see [Must
rule](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/fleetd-development-and-release-strategy.md)).
- [x] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [x] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

---------

Co-authored-by: Lucas Rodriguez <lucas@fleetdm.com>
2025-04-11 10:18:28 -04:00
Victor Lyuboslavsky
5a549a3d03
Mark test-go failed if one test failed. (#28052) 2025-04-09 17:39:47 -05:00
Allen Houchins
00480f8fb0
Renaming teams to support Android devices (#27991)
Related to fleetdm/confidential#9192
2025-04-08 17:45:14 -05:00
Ian Littman
008eff27b0
Add MySQL 8.0.32 to test matrix (#27985)
Adding to validate Aurora 3.05.2 compat per a customer request.
2025-04-08 12:51:36 -05:00
Scott Gress
c45e0df72b
Update permissions in workflows to be more restrictive (#27959) 2025-04-07 16:16:16 -05:00
Victor Lyuboslavsky
7558d5afa1
Improvements to test-go CI job (#27916)
This PR makes several improvements to test-go CI job
- remove ZSH dependency from all test suites except for a new `scripts`
suite
- add a `fast` suite that does not have out-of-process dependencies,
which completes in ~1.5 minutes
- contributors can add their fast tests to this suite so they can see
the results in CI faster
- Rename `core` to `main` test suite to be consistent with Makefile. It
is the default bucket for tests.
- Cleaned up Makefile so that it is more straightforward to add new test
suites or move Go packages between suites
- Do not stop the test suites on a fail.
- We do not want to be blocked by a test fail that another product team
introduced
- Sometimes, we want to see all test failures so we can fix them all at
once.
- Removed `test-schema` and `mock` prerequisites for `test-go` since
they are not needed and just take up time.
- But also added `test-schema` run to one of the test suites just in
case.

Unfortunately, `fleetctl` is still the bottleneck and needs to be
refactored. New issue filed:
https://github.com/fleetdm/fleet/issues/27927

## Before
<img width="248" alt="image"
src="https://github.com/user-attachments/assets/110ffc1d-f090-4d3e-be77-0419b9577d20"
/>

## After
<img width="320" alt="image"
src="https://github.com/user-attachments/assets/8d01ea11-408f-4eb6-81d8-9c25410b8830"
/>
2025-04-07 12:52:26 -05:00
Lucas Manuel Rodriguez
8003227554
Use go 1.24 new tool feature (#27765)
Release notes: https://tip.golang.org/doc/go1.24

> Go modules can now track executable dependencies using tool directives
in go.mod. This removes the need for the previous workaround of adding
tools as blank imports to a file conventionally named “tools.go”. The go
tool command can now run these tools in addition to tools shipped with
the Go distribution. For more information see [the
documentation](https://tip.golang.org/doc/modules/managing-dependencies#tools).
The new -tool flag for go get causes a tool directive to be added to the
current module for named packages in addition to adding require
directives.

I ran:
```
go get -tool github.com/fleetdm/fleet/v4/server/goose
go get -tool github.com/kevinburke/go-bindata
go get -tool github.com/quasilyte/go-ruleguard/dsl
go rm tools.go
go mod tidy
```

`make deps-go` was failing in CI because of the removal of `tools.go`
(my guess is that `go get .` was a nop because there was nothing in `.`
to download).
So, taking the chance of removing `deps-go` because `go` will download
packages during the build process. AFAICS there's no need to download
everything beforehand.
2025-04-07 11:12:05 -03:00
Scott Gress
caf5e6f3b0
Allow patch release of fleetd via releaser script (#27448)
For #21396

# Details

This PR updates the automated release cycle for Orbit desktop, so that
it triggers based on a pushed _tag_ rather than a pushed PR. This has
the following benefits:

* The release can be based off of any branch, rather than always using
`main` as the base, so we can safely do patch release of desktop without
including in-progress code from main
* It brings the desktop release process more in line with the main Orbit
release process -- both are now triggered by a tag push.

We still create a PR for the release, to include a changelog.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [ ] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [ ] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).

## Testing

To do -- will discuss with @lucasmrod

---------

Co-authored-by: Luke Heath <luke@fleetdm.com>
2025-04-04 09:47:16 -05:00
Lucas Manuel Rodriguez
e2e8c32797
Increase sleep time in verify-fleetd-base.yml (#27763)
https://github.com/fleetdm/fleet/actions/runs/14211929711
2025-04-02 14:30:04 -03:00
Ian Littman
6be5cf55cb
Fix path for go-version-file in FMA ingest job (#27744)
When Go version switched from being hardcoded to being based off of the
deps file, Fleet being checked out into a subdir wasn't taken into
account, so FMA ingest jobs started failing. This adds the (hopefully)
correct dir to fix the issue and get FMA ingest working again.
2025-04-01 16:19:21 -05:00
Ian Littman
0293d99800
Remove default EXE install/uninstall scripts, require entering install/uninstall scripts on EXE upload (#27268)
For #27267.

Below is what's shown immediately after selecting an EXE:

<img width="1254" alt="image"
src="https://github.com/user-attachments/assets/a28d8565-de88-448a-bdbc-92aefc34ad55"
/>


TODO:

* Tests
* GitOps requirements changes
* Disabling add button/adding errors when required scripts aren't
specified

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Input data is properly validated, `SELECT *` is avoided, SQL
injection is prevented (using placeholders for values in statements)
- [x] Added/updated automated tests
- [x] A detailed QA plan exists on the associated ticket (if it isn't
there, work with the product group's QA engineer to add it)
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Luke Heath <luke@fleetdm.com>
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
Co-authored-by: RachelElysia <rachel@fleetdm.com>
2025-03-31 13:52:06 -05:00
Scott Gress
59f96651b6
Update to Go 1.24.1 (#27506)
For #26713 

# Details

This PR updates Fleet and its related tools and binaries to use Go
version 1.24.1.

Scanning through the changelog, I didn't see anything relevant to Fleet
that requires action. The only possible breaking change I spotted was:

> As [announced](https://tip.golang.org/doc/go1.23#linux) in the Go 1.23
release notes, Go 1.24 requires Linux kernel version 3.2 or later.

Linux kernel 3.2 was released in January of 2012, so I think we can
commit to dropping support for earlier kernel versions.

The new [tools directive](https://tip.golang.org/doc/go1.24#tools) is
interesting as it means we can move away from using `tools.go` files,
but it's not a required update.

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
- [x] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Make sure fleetd is compatible with the latest released version of
Fleet
   - [x] Orbit runs on macOS  , Linux   and Windows. 
- [x] Manual QA must be performed in the three main OSs, macOS ,
Windows and Linux .
2025-03-31 11:14:09 -05:00
Noah Talerman
e93750edab
Cleanup scripts (#27307)
- Move duplicate scripts out of `scripts/mdm/` and into
`it-and-security/` so we have one version that we can continue to
iterate and improve.
- Remove no longer used scripts out of `scripts/mdm/`

---------

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
2025-03-27 16:43:53 -05:00
Lucas Manuel Rodriguez
60f829f6bd
Fix code scanning alerts (#27074)
These are fixes to some of the alerts in:
https://github.com/fleetdm/fleet/security/code-scanning
2025-03-27 10:01:20 -03:00
Victor Lyuboslavsky
527428f0b4
Speeding up Go tests in CI. (#27434)
Using `go get .` (~3s) is faster than `go mod download` (~13s) in CI.
2025-03-25 12:03:24 -05:00
Scott Gress
bbe3f18b88
Dismiss old FMA PRs after creating a new one (#27359)
We have a scheduled action to create an "Update Fleet-maintained apps"
PR with the latest updates for FMAs. The bot creates a new PR each time,
even if the previous PRs haven't been merged. Since the latest PR will
always have all the latest updates, it's unnecessary to keep the old
ones around. We should have the bot close the old PRs once the new one
is opened. Additionally it was recommended that assignees be added to
the PR.

This PR updates the action to:

1. Find existing "Update Fleet-maintained apps" PRs created by the bot
2. Get the list of Github IDs of the devs on the software product team
3. Creates a new PR as usual, adding the devs as assignees
4. If the new PR is created successfully, closes the old PRs with
comments linking to the new PR

See a successful run
[here](https://github.com/fleetdm/fleet/actions/runs/13977643317/job/39135240445),
+ the resulting PR [here](https://github.com/fleetdm/fleet/pull/27357)

---------

Co-authored-by: Tim Lee <timlee@fleetdm.com>
2025-03-21 09:32:02 -05:00
Luke Heath
0c81e7ba04
Address code scanning permissions warnings (#27250) 2025-03-20 12:07:41 -05:00
Allen Houchins
30012688d1
Added policy automation and new Windows installer (#27244)
- Added script and workflow to keep 1Password policy up to date
- Changed to using .msi instead of .exe for Windows installer
2025-03-19 13:54:25 -05:00
Victor Lyuboslavsky
eae1fd30a1
Add stale issues workflow (#27047)
Add stale issues workflow which:
- marks `~engineering-initiated` issues with `stale` label and comment
if they had no activity in the last 365 days (we have ~74 such issues
right now)
- closes `stale` issues after 14 days with no activity

Examples of "recent" stale issues:
- https://github.com/fleetdm/fleet/issues/13984
- https://github.com/fleetdm/fleet/issues/15571
- https://github.com/fleetdm/fleet/issues/16117
- https://github.com/fleetdm/fleet/issues/16352

Here's an example issue with a `stale` label. I ran the workflow and
marked 27 issues as `stale` that had no activity for 1000 days.
- https://github.com/fleetdm/fleet/issues/4631
2025-03-14 16:11:43 -05:00
Lucas Manuel Rodriguez
5a84fa0d11
Release fleetd 1.40.1 (#27157) 2025-03-14 17:10:15 -03:00
Lucas Manuel Rodriguez
b30a008aac
Simplify DB test/upgrade tool (#27141)
This PR simplifies the `test/upgrade` tool the QA team uses to test DB
upgrades.

- Removes "online migration" approach because we currently don't support
it (so it removes nginx as dependency).
- Adds a workflow to manually run this on Github actions (in case dev/QA
folks have issues with Docker on macOS, which is a common thing...)
- Adds logging to the output to ease troubleshoot (previous versions was
too quiet making it impossible to troubleshoot).
2025-03-14 17:07:41 -03:00
Lucas Manuel Rodriguez
30f1c329b4
Build orbit on ubuntu-20.04 (#27156)
For #27155.

Reverting to use ubuntu-20.04 to unblock the release of fleetd 1.40.1.
We'll need a proper solution for 1.41.0 given that ubuntu-20.04 will be
removed on April 1st 2025.

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
2025-03-14 16:17:38 -03:00
Tim Lee
baf220a779
Fix: Update FMA Action (#27128) 2025-03-13 16:54:04 -06:00
Jahziel Villasana-Espinoza
4e325bb131
FMAv2 ingestion functionality (#27018)
> For #26083

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality

---------

Co-authored-by: Ian Littman <iansltx@gmail.com>
2025-03-12 17:51:14 -04:00
Lucas Manuel Rodriguez
a0be5164e3
Release fleetd 1.40.0 (#27053) 2025-03-12 13:15:38 -03:00
Robert Fairburn
f32f80261a
Add cloudfront to dogfood (#26962) 2025-03-12 10:59:39 -05:00
Scott Gress
caa91dbc04
Update goreleaser-fleet workflow to use ubuntu-22.04-4-cores runner (#26930)
For #26927 

See
[actions/runner-images#11101](https://github.com/actions/runner-images/issues/11101)
and https://github.com/fleetdm/fleet/pull/26466/files. Updating
explicitly to `ubuntu-22.04-4-cores` to not have it float with `latest`.
[Other](6e097988d5/.github/workflows/goreleaser-orbit.yaml (L81))
[goreleaser](6e097988d5/.github/workflows/goreleaser-snapshot-fleet.yaml (L40))
workflows use 22.04.
2025-03-10 10:01:36 -05:00
Ian Littman
e7d04f2ff0
Clean up Python 3.12 vulns test expectations, fix continue-on-error behavior for integration and vulns suites (#26903)
From what I can tell, continue-on-error has been false for the
integration suites since the suites were renamed to `integration-*`, so
this fixes that issue in addition to continuing to run test suites when
the vulns suite fails (which may be due to vulns feed updates).

This also makes the vulns test more resilient to new CVEs being reported
on Python 3.12.0, which is rather likely to collect new CVEs.

# Checklist for submitter

- [x] Added/updated automated tests
2025-03-06 08:46:07 -06:00
Victor Lyuboslavsky
46719c9fb2
Auto-generate and check Android schema.sql (#26720)
For #26218 

This is an infrastructure change. No functional feature changes.
2025-02-28 16:30:40 -06:00
Luke Heath
658330d4f4
Enable Android in dogfood (#26735) 2025-02-28 16:26:11 -06:00
Scott Gress
f7e4a39915
Add help system to Makefile + FDM command (#25028) 2025-02-28 07:42:32 -06:00
Lucas Manuel Rodriguez
74f7548fd1
Add workflow to update timestamp on new repository (#26635)
For #26482.
2025-02-27 18:02:42 -03:00
Lucas Manuel Rodriguez
5e453b7ca6
Check for timestamps on the new TUF repository (#26638)
For #26483.
2025-02-27 17:59:11 -03:00
Lucas Manuel Rodriguez
6e097988d5
Update ubuntu-20.04 to ubuntu-22.04 on CI actions due to deprecation (#26466)
> We will soon start the deprecation process for Ubuntu 20.04. While the
image is being deprecated, you may experience longer queue times during
peak usage hours. Deprecation will begin on 2025-02-01 and the image
will be fully unsupported by 2025-04-01.

From https://github.com/actions/runner-images/issues/11101.

![Screenshot 2025-02-19 at 4 39
10 PM](https://github.com/user-attachments/assets/803e8ed3-31b0-4221-9d29-446a0d305567)
2025-02-21 13:11:50 -03:00
Allen Houchins
0a94f81431
Switched from metadata_url to metadata for end user authentication (#26042)
Switched from metadata_url to metadata for end user authentication.

---------

Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2025-02-20 10:02:24 -06:00
Lucas Manuel Rodriguez
e5fd094c68
Release osqueryd 5.16.0 (#26394)
For #26324.
2025-02-18 16:59:09 -03:00
Tim Lee
c582f74fdd
Release fleetd 1.39.1 (#26314) 2025-02-13 16:26:18 -07:00
Martin Angers
240f55b9e8
Add script execs and software installs stats to osquery-perf (#26239) 2025-02-11 12:46:53 -05:00
Lucas Manuel Rodriguez
a216c9d9e5
Release fleetd 1.39.0 (#26185) 2025-02-07 18:52:06 -03:00
Lucas Manuel Rodriguez
9114d0405b
Update TUF status generation to use new TUF repository (#26099)
For #25853.
2025-02-07 08:30:07 -03:00
Scott Gress
5c0b2dc6b0
Use tag rather than commit sha when attesting images (#26032) 2025-02-04 16:11:02 -06:00
StepSecurity Bot
d109e72cbe
[StepSecurity] ci: Harden GitHub Actions (#25985) 2025-02-03 12:25:18 -06:00
Luke Heath
e9b55e6ef9
Revert goreleaser test changes (#25949) 2025-01-31 15:03:56 -06:00
Allen Houchins
99ce080e3e
Implement policy checks and automated updates (#25829)
This is the initial pull request to implement keeping policy logic up to
date automatically. For example, when a new version of macOS releases,
admins don't need to manually update the policy logic for checking
version numbers.

This is currently blocked by this issue: fleetdm/confidential#9470
This is also to support the following issue and demonstrate to customers
a fully automated patch management strategy:
https://github.com/fleetdm/confidential/issues/8825

This current iteration contains a script/workflow that runs every 6
hours to check if a new version of macOS has been released and compares
the version string to what is currently defined in our policy. If it
detects a change, it will automatically create a new branch with the
updated version string and create a pull request to be reviewed before
merging.
2025-01-30 20:55:22 -06:00