Commit graph

575 commits

Author SHA1 Message Date
Gabriel Hernandez
22baa5af94
bump action/cache to version 4.2.0 (#25508)
For #25507

A bump to the latest version to the github `cache` action to 4.2.0. our
current version (v2) was deprecated. more info for the deprecation can
be found here https://github.com/actions/cache/discussions/1510

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
2025-01-17 15:01:27 +00:00
Noah Talerman
d5626acc6c
Dogfood hiding secrets added via YAML (#25384) 2025-01-15 13:40:39 -06:00
Victor Lyuboslavsky
d15d2e324e
Speculative fix for flaky TestVPPApps. (#25385)
For #25086 
Speculative fix for flaky TestVPPApps (disable worker jobs). Ran 50
tests with randokiller and did not see a fail.
2025-01-13 16:28:48 -06:00
Dante Catalfamo
8c338a1d77
Try splitting up integration tests (#25312)
Follow up to #25271 and #21774

Integration test failures will happen much faster of they occur, but now
the bottleneck is the `fleetctl` test suite.

It's trivial to continue splitting tests up now. We should look into
creating an action that checks that mock generation is up-to-date, run
it before all the tests, and then remove the mock generation step from
each test step. That would save about a minute and a half of runtime
from each test and help offset the cost of splitting the tests up.

![ci runtime
breakdown](https://github.com/user-attachments/assets/057b8ee1-782c-4e1f-9486-42c7d1169c81)
![ci runtime
max](https://github.com/user-attachments/assets/3a26995f-d9cb-490b-84d9-1a7fbb3cd6b3)

![image](https://github.com/user-attachments/assets/b4c888c8-867f-4bdd-9b69-0dc20d0d202a)
2025-01-10 12:52:13 -05:00
Lucas Manuel Rodriguez
7e419f97cb
Fix missing docs and yaml (#25333) 2025-01-10 11:42:55 -06:00
Scott Gress
147c5542e8
fix path to artifacts json (#25331) 2025-01-10 11:40:46 -06:00
Lucas Manuel Rodriguez
009f54bdda
Changes to migrate to new TUF repository (#23588)
# Changes

- orbit >= 1.38.0, when configured to connect to
https://tuf.fleetctl.com (existing fleetd deployments) will now connect
to https://updates.fleetdm.com and start using the metadata in path
`/opt/orbit/updates-metadata.json`.
- orbit >= 1.38.0, when configured to connect to some custom TUF (not
Fleet's TUFs) will copy `/opt/orbit/tuf-metadata.json` to
`/opt/orbit/updates-metadata.json` (if it doesn't exist) and start using
the latter.
- fleetctl `4.63.0` will now generate artifacts using
https://updates.fleetdm.com by default (or a custom TUF if
`--update-url` is set) and generate two (same file) metadata files
`/opt/orbit/updates-metadata.json` and the legacy one to support
downgrades `/opt/orbit/tuf-metadata.json`.
- fleetctl `4.62.0` when configured to use custom TUF (not Fleet's TUF)
will generate just the legacy metadata file
`/opt/orbit/tuf-metadata.json`.

## User stories

See "User stories" in
https://github.com/fleetdm/confidential/issues/8488.

- [x] Update `update.defaultRootMetadata` and `update.DefaultURL` when
the new repository is ready.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
2025-01-10 14:27:30 -03:00
Victor Lyuboslavsky
dfa9a90775
Updating deprecated actions/download-artifact (#25296) 2025-01-09 12:58:59 -06:00
Dante Catalfamo
e6fb647d95
Run CI tests in parallel (#25271)
#21774

Improves run time by about 30%.

Things have been arranged in such a way that splitting modules out
further will be trivial in the future, such as breaking the different
integration test suited into their own units.


![image](https://github.com/user-attachments/assets/ead46e4c-6f14-406d-a29b-b25abc79c384)


![image](https://github.com/user-attachments/assets/3f7fd7f3-d7a8-4ff8-a184-646a72f1d015)
2025-01-09 13:38:24 -05:00
Victor Lyuboslavsky
b1891b8838
Update deprecated actions/upload-artifact to v4.5.0 (#25295) 2025-01-09 12:08:02 -06:00
Robert Fairburn
8449879c81
Changes needed for new dogfood monitoring (#25147) 2025-01-06 10:34:36 -06:00
Scott Gress
9181ba7053
Update Dogfood monitoring to v1.5 (#24425) 2025-01-03 16:24:01 -06:00
Victor Lyuboslavsky
0b9f36ad03
Enable MySQL 9.1 Go tests (#25055)
Enable MySQL 9 tests in the nightly Go test run.

The tests passed in my run:
https://github.com/fleetdm/fleet/actions/runs/12552738253/job/34999129651
2025-01-02 15:36:10 -06:00
jacobshandling
120f01ad1e
Fix verify fleetd-base files > verify-fleetd-base-msi powershell script (#25064)
See failed workflow run
[here](https://github.com/fleetdm/fleet/actions/runs/12555703803)

- Fix the powershell script that was broken by `.yml` auto-format
- Exclude github workflow `.yml` files from prettier autoformating,
since they often contain non-yaml code as part of job definitions

- [ ] Manual QA for all new/changed functionality

---------

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2025-01-02 10:24:25 -08:00
Lucas Manuel Rodriguez
b73876878a
Release osqueryd 5.15.0 (#25087) 2025-01-02 11:41:03 -03:00
jacobshandling
ca37183e5c
generate Slack notfication if any of the 3 verify fleetd-base steps fail (#25049)
## #24531 

These changes were approved by @lucasmrod
[here](https://github.com/fleetdm/fleet/pull/25019)

"<test - ignore me>" will not be present in real notifications 
<img width="652" alt="398807048-d208c9f8-999e-4c0a-a818-5e72570481ab"
src="https://github.com/user-attachments/assets/a06f63a8-e5b7-4b5a-881d-606b3e66c6d9"
/>

- [x] Manual QA for all new/changed functionality

Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
2024-12-30 12:46:50 -08:00
Allen Houchins
c50a096527
Add new "📱🔐 Personally-owned iPhones" team (#24952) 2024-12-20 14:24:59 -06:00
Ian Littman
11c316e653
Pin Python version in GitHub Actions for osquery version updater, use Python HTTP client directly to avoid needing to figure out how to pin requests lib (#24861)
For #24274. Skipping changes file since this is an internal tool.

# Checklist for submitter
- [x] Manual QA for all new/changed functionality
2024-12-18 14:04:39 -06:00
Scott Gress
0e5541979a
Update attestation implementation (#24837)
for #23825 

This PR fixes the previous implementation for attesting
fleet/fleetctl/orbit binaries, and adds attestation to the fleet desktop
and osqueryd artifacts.

* correct permissions are added to all jobs
* tag removed from `subject-name` when attesting docker image
* using `artifacts.json` rather than the `artifacts` step output from
goreleaser to determine image digest

I'd like to add a separate job verifying the attestations, working on
that now but since all attestation steps are marked as
`continue-on-error` it can be a follow-on if we don't get it in with
this PR.
2024-12-17 15:26:59 -06:00
Lucas Manuel Rodriguez
7d0609341e
Release fleetd 1.37.0 (#24752) 2024-12-13 17:08:36 -03:00
Scott Gress
1c3487ad86
Create RandoKiller™ to help diagnose and fix flaky tests (#24696)
This PR adds a new workflow called "Stress Test Go Test" (aka the
RandoKiller) that allows running one or more tests repeatedly up to a
set number of times, or until a test fails. This is useful for:

* Trying to diagnose and debug a flaky test
* Verifying that a proposed fix for a flaky test actually works.

To use:

1. Create a branch whose name ends with "-randokiller"
2. Modify the .github/workflows/config/randokiller.json file to your
specifications (choosing the packages and tests to run, the mysql
matrix, and the number of runs to do)
3. Push up the branch

Since the stress test is intended to run a branch that you'll never
merge, you should feel free to add whatever logs to your tests or code
that will help diagnose failures.

I used this to diagnose and fix
https://github.com/fleetdm/fleet/pull/24697!
2024-12-12 12:02:21 -06:00
Robert Fairburn
e361073fca
Update terraform version used in tfvalidate (#24699) 2024-12-12 11:25:26 -06:00
Scott Gress
149e5b129f
23285 add attestation (#24604) 2024-12-10 16:02:08 -06:00
Scott Gress
4a623812e8
Add notification for dogfood GitOps failures (#24402)
for #19106 

This PR adds a Slack notification when the GitOps run fails in the
dogfood-gitops workflow. Whenever the actual GitOps action fails, it
should notify #help-dogfooding with a link to the failed action. Note
that this will alert on both merges to main and scheduled runs, which I
think we want. Also note that this is [currently failing on
main](https://github.com/fleetdm/fleet/actions/runs/12154006118) so this
alert will start going off daily until the issue is fixed 😶

### > Note: this will need a new Slack incoming webhook for sending
messages to #help-dogfooding, and a new
`SLACK_G_HELP_DOGFOODING_WEBHOOK_URL` repo secret with the webhook URL.

I tested this on a personal private repo just to make sure I got all the
syntax right:

<img width="422" alt="image"
src="https://github.com/user-attachments/assets/74d188eb-5c03-471b-a5db-9f578a56e2ab">
2024-12-10 13:39:32 -06:00
Victor Lyuboslavsky
f2fc2bc0ea
Add retry to fleetd base pkg build. (#24489) 2024-12-09 13:24:38 -06:00
Tim Lee
7547dcb74e
bugfix: orbit linux zenity progress windows (#24280) 2024-12-05 08:02:03 -07:00
Scott Gress
04010ba5f3
Use goreleaser v2 in CI (#23748) 2024-12-03 16:15:31 -06:00
Lucas Manuel Rodriguez
8dbfbad167
Add github action to automate timestamp update (#24074)
#23042
2024-11-27 16:13:54 -03:00
Lucas Manuel Rodriguez
7d04119245
Release fleetd 1.36.0 (#24136) 2024-11-25 16:34:09 -03:00
Marko Lisica
04d1ded81a
Remove step to apply env variables to profiles (fleetctl gitops is doing this) (#24078)
I beleive we don't need this step anymore, since `fleetctl gitops` will
replace it with real value and send to the server. This should be done
in #17309.
2024-11-22 19:30:40 +01:00
Lucas Manuel Rodriguez
a8ba004c61
Add expiration checks for targets and snapshot roles (#24081)
[#8489](https://github.com/fleetdm/confidential/issues/8489)

We had the timestamp check.
Robert added the root check recently.
Am now duplicating the check for `snapshot` and `targets` metadata
files.

PS: Please review with whitespace changes disabled.
2024-11-22 14:46:03 -03:00
Victor Lyuboslavsky
e4df954b0f
Update nanomdm dependency with latest bug fixes and improvements. (#23906)
#23905 

- Update with upstream nanomdm changes up to
825f2979a2
- Removed PostgeSQL folder from our nanomdm
- Added nanomdm MySQL test job to our CI

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-11-20 11:47:11 -06:00
Lucas Manuel Rodriguez
c93a28b00f
Fix build-fleetd_tables.yml workflow (#23875)
This fixes the workflow, successful run:
https://github.com/fleetdm/fleet/actions/runs/11864218412
2024-11-15 19:41:19 -03:00
Lucas Manuel Rodriguez
c993e5a485
Add workflow to build fleetd_tables (#23873)
Workflow to build `fleetd_tables` in Github.
2024-11-15 19:05:07 -03:00
Robert Fairburn
e20820dfe5
Deploy to free.fleetdm.com before dogfood (#23762) 2024-11-13 11:17:23 -06:00
StepSecurity Bot
552e76b68e
[StepSecurity] ci: Harden GitHub Actions (#23765) 2024-11-13 10:43:13 -06:00
Lucas Manuel Rodriguez
299a7999b2
Fix trivy fleetctl workflow (#23643)
Exactly same fix as #23634.

PS: Thanks @sgress454!
2024-11-12 14:58:41 -03:00
Scott Gress
1551157c23
Fix rate limiting issue in Trivy workflow scan (#23634) 2024-11-07 15:06:17 -06:00
Robert Fairburn
e525eed606
Include root.json in check-tuf-timestamps.yml (#23608) 2024-11-07 11:32:01 -06:00
Lucas Manuel Rodriguez
c09360aabb
Add workflow to ease QA of ADE workflows (#23470)
From discussions with @jahzielv.

QAing ADE flows:

1. New version of fleetd is pushed to `edge`
2. QA folks can trigger this new workflow and download the generated
`fleetd-base.pkg` and `fleetd-base-manifest.plist`.
3. Host the downloaded files (in `foobar/`) in their ngroks URLs (using
e.g. `go tools ./tools/file-server 8085 foobar/`)
4. Use Fleet's `FLEET_DEV_DOWNLOAD_FLEETDM_URL` to point the Fleet
server to their ngrok URL.
2024-11-05 05:28:50 -03:00
Lucas Manuel Rodriguez
3cdafcee5f
Release fleetd 1.35.0 (#23467) 2024-11-04 11:39:46 -03:00
RachelElysia
951542dacf
Fleet UI: Automatically create a PR if Osquery version is not up to date in UI (#23290) 2024-10-30 16:35:44 -04:00
Victor Lyuboslavsky
d0ed03f3c8
Update Go CI tests: MySQL 8.4.2 -> 8.4.3 (#23255)
Make sure Fleet works with the latest MySQL 8.4
2024-10-25 15:58:10 -05:00
Victor Lyuboslavsky
40f7cdf547
Added Render deploy workflow for fleet-gitops CI. (#23190)
#21715 

# Checklist for submitter
- [x] Manual QA for all new/changed functionality
2024-10-25 15:55:42 -05:00
Ian Littman
921d8c8afd
Check for TUF expirations 2x daily, and warn 4 days in advance rather than 2 (#23039) 2024-10-23 10:06:03 -05:00
Lucas Manuel Rodriguez
103727b5f4
Use ubuntu 22.04 for trivy ci scheduled check (#23088)
Fixing these failures:
https://github.com/fleetdm/fleet/actions/runs/11454239765/job/31881900492

Following this advice
https://github.com/orgs/community/discussions/139074#discussioncomment-10935090.
It seems it works.
2024-10-22 15:41:32 -05:00
Lucas Manuel Rodriguez
2c891447d4
Release osqueryd 5.14.1 (#23045) 2024-10-22 12:27:00 -05:00
Victor Lyuboslavsky
f85b6f776f
Updating golangci-lint to 1.61.0 (#22973) 2024-10-18 12:38:26 -05:00
Eric
3dda9d0252
Delete deploy-bulk-operations-dashboard workflow (#22940)
Closes: https://github.com/fleetdm/confidential/issues/8351


Changes:
- Deleted the "Deploy app to bulk operations dashboard pipeline on
Heroku" workflow. This dashboard is now hosted in Render, and deploys
are triggered manually via the Render dashboard.
2024-10-18 12:02:21 -05:00
Eric
cb67c32afc
Change Ubuntu version in Heroku deploy workflows (#22939)
Closes: #22931

Changes:
- Updated the deploy workflows for the Fleet website and the
vulnerability dashboard to run on Ubuntu 22.04 to prevent issues we've
been seeing with the Heroku deploy action and the latest version of
Ubuntu.
2024-10-15 16:20:12 -05:00