Commit graph

518 commits

Author SHA1 Message Date
Ian Littman
59160c2f30
Exclude migration tests from migration timestamp/ordering check (#22496)
This fix applies to cases (e.g.
00ec402f18) where order of files being
added is e.g.:

1. Migration A
2. Migration B
3. Test for migration A

This also reorders workflow steps so the ones that don't require setting
up Go + compiling happen first, so if we have a migration issue it gets
reported sooner.

# Checklist for submitter

- [x] Manual QA for all new/changed functionality
2024-09-30 09:53:19 -05:00
Lucas Manuel Rodriguez
e861ae7319
Release fleetd 1.33.0 (#22283) 2024-09-23 06:59:04 -03:00
Lucas Manuel Rodriguez
c49bed104d
Add CI check to detect issues with pushed fleetdm/fleetctl docker image (#22020)
Adds detection of issues like #21992.
2024-09-16 13:05:28 -03:00
Ian Littman
ab996dc57d
Note that minimum MySQL version is 8.0.36 everywhere (not just 8.0) because that's what we test with (#22072)
Also mention that we test with 8.4.2 in a few more places.

Note that while I'm editing release articles, this isn't retconning
minimum requirements; we mention in 4.55.0 release notes further down
that we expect 8.0.36.
2024-09-13 10:59:38 -05:00
Roberto Dip
078c0ac3b7
document and use MDM SSO settings via gitops (#21869)
for #21313
2024-09-09 17:03:20 -03:00
Roberto Dip
70923b8352
add wait until mysql is ready (#21883) 2024-09-06 18:17:22 -03:00
Lucas Manuel Rodriguez
6a5c515dc4
Attempt to use go.mod version instead of hidden Github var (#21768)
Done as part of oncall improvements.

`vars.GO_VERSION` can only be changed by admins and it's not public
(Fleet devs don't know the current value of the variable), this approach
uses the version specified in our `go.mod` file.
2024-09-03 20:49:50 -03:00
Victor Lyuboslavsky
1b06b050d7
Fix issues with coverage uploads (#21736)
#21707
2024-09-03 09:07:16 -05:00
Lucas Manuel Rodriguez
ea7d08fc88
Release fleetd 1.32.0 (#21658)
Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
2024-08-29 18:51:10 -03:00
Lucas Manuel Rodriguez
8d4b7ad526
Remove unused workflow and dockerfile (#21551)
- `.github/workflows/push-osquery-perf-to-ecr.yml` has 0 workflow runs
(added but never used)
- `Dockerfile.osquery-perf` is only used by
`.github/workflows/push-osquery-perf-to-ecr.yml`.
2024-08-26 13:28:25 -03:00
Victor Lyuboslavsky
fdfc12982b
Improvements to go tests in CI (#21545)
#21546 
Some improvements to overall go test CI run time.
2024-08-26 08:55:53 -05:00
Eric
debb2d1790
Add app to manage scripts and profiles. (#21450)
Related to: #20296 

Changes:
- Added `ee/bulk-operations-dashboard`, a Sails.js app that lets users
manage configuration profiles and scripts across multiple teams on a
Fleet instance.
- Added a Github workflow to deploy the app to Heroku
- Added a Github workflow to test changes to the bulk operations
dashboard.
2024-08-22 14:59:15 -06:00
Lucas Manuel Rodriguez
ede0897acd
Pin trivy action (#21425)
Fixing code scanning warnings
https://github.com/fleetdm/fleet/security/code-scanning for the recently
added workflow.
2024-08-20 15:27:41 -03:00
Lucas Manuel Rodriguez
18f010f228
Update fleetdm/fleetctl, fleetdm/wix and fleetdm/bomutils docker images (#21063)
#20571

## Summary of changes

We have a few moving parts in fleetctl land (`fleetdm/wix` is used to
build `msi`s and `fleetdm/bomutils` is used to build `pkg`s, and
`fleetdm/fleetctl` can be used to build packages using docker, no need
for fleetctl executable):
```mermaid
graph LR

fleetctl_exec[fleetctl<br>executable];
wix_image[fleetdm/wix<br>docker image];
bomutils_image[fleetdm/bomutils<br>docker image];
fleetctl_image[fleetdm/fleetctl<br>docker image];

fleetctl_exec -- uses --> wix_image;

fleetctl_image -- COPY dependencies<br>FROM --> wix_image;

fleetctl_exec -- uses --> bomutils_image;

fleetctl_image -- COPY dependencies<br>FROM --> bomutils_image;
```
So, we'll need to update the three images: `fleetdm/bomutils`,
`fleetdm/wix` & `fleetdm/fleetctl`.

- `tools/bomutils-docker/Dockerfile`, `tools/wix-docker/Dockerfile` and
`tools/fleetctl-docker/Dockerfile`: Updating the base image to fix the
CRITICAL vulnerabilities.
- Modified existing+unused
`.github/workflows/build-and-check-fleetctl-docker-and-deps.yml` to run
every day to check for CRITICAL vulnerabilities in `fleetdm/wix`,
`fleetdm/bomutils` and `fleetdm/fleetctl`.
- `.github/workflows/goreleaser-fleetctl-docker-deps.yaml`:
`fleetdm/bomutils` and `fleetdm/wix` were pushed manually a few years
ago (most likely by Zach), so I've added a new action to release them
when we have changes to release (like now). It will basically release
`fleetctl/bomutils` and `fleetdm/wix` when pushing a tag of the form
`fleetctl-docker-deps-*` (we'll need to protect such tag prefix).
- Changes in `.github/workflows/test-native-tooling-packaging.yml` to
build `fleetdm/bomutils` and `fleetdm/wix` for `fleetdm/fleetctl` to use
them instead of the ones in docker hub.

--

Build before upgrading `debian:stable-slim`:

https://github.com/fleetdm/fleet/actions/runs/10255391418/job/28372231837
![Screenshot 2024-08-05 at 5 24
25 PM](https://github.com/user-attachments/assets/8a7d3576-3eb6-474f-989a-079873fca4fa)

Build after upgrading `debian:stable-slim`:
https://github.com/fleetdm/fleet/actions/runs/10255550034

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-08-20 14:07:59 -03:00
Roberto Dip
f4820e2af9
Release fleetd 1.31.0 (#21391)
- **Release fleetd 1.30.0**
- **fix issue with disk encryption banner (#21385)**
- **Release fleetd 1.31.0**
2024-08-19 16:53:14 -03:00
Roberto Dip
936cc4a6d7
notify #help-engineering when a dogfood deploy is in progress (#21347)
This ensures `#help-engineering` is notified when a dogfood deploy is in
progress. It helps set people's expectations about what's going on while
the server is temporarily down.
2024-08-19 15:33:00 -03:00
Lucas Manuel Rodriguez
cdfa31ada5
Release osqueryd 5.13.1 (#21329) 2024-08-19 13:52:30 -03:00
Victor Lyuboslavsky
4eb72535dc
Support for MySQL 8.4.2 (#21364)
#21270

The main change for MySQL 8.4.2 is that foreign key constraints are
stricter:
https://dev.mysql.com/doc/refman/8.4/en/server-system-variables.html#sysvar_restrict_fk_on_non_standard_key

Also, most replica-related commands have been renamed.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
2024-08-16 16:32:38 +02:00
Dante Catalfamo
9a5e5372eb
SSVPP Backend and CLI (#21132)
#19882 and #20532
2024-08-14 10:25:32 -04:00
Luke Heath
34901a9dc7
Update goreleaser runner (#21226) 2024-08-09 10:50:43 -07:00
Roberto Dip
4853ecbf4e
Release fleetd 1.30.0 (#21071) 2024-08-06 07:32:24 -03:00
Lucas Manuel Rodriguez
ab7df5155d
Use docker compose on CI instead of docker-compose (#21017)
After this is merged I'll cherry pick to `minor-fleet-4.55.0`.
2024-08-02 18:12:36 -03:00
Lucas Manuel Rodriguez
2f479b3ba9
Release osqueryd 5.13.0 (#20949) 2024-08-02 14:57:50 -03:00
Dante Catalfamo
0a15647e10
Host software deleted at remigration (#20996)
# Recreate out of order migration, replace `docker-compose` with `docker compose` in db test runner
2024-08-02 10:47:40 -04:00
Lucas Manuel Rodriguez
6d87091a89
Release fleetd 1.29.0 (#20700) 2024-07-26 15:04:03 -03:00
Dante Catalfamo
5e1a3d03ae
MySQL 8.0 Migration (#20225)
#17249
2024-07-22 16:27:36 -04:00
Victor Lyuboslavsky
8c1c016b54
Don't stop unit tests if Go integration tests fail (#20628) 2024-07-20 19:22:07 +02:00
Lucas Manuel Rodriguez
84a81bafde
Release fleetd 1.28.0 (#20581)
Co-authored-by: Luke Heath <luke@fleetdm.com>
2024-07-19 12:47:00 -03:00
Dante Catalfamo
ecf2346ace
Add support for Linux ARM64 (#19931)
#1845

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-07-17 16:07:59 -04:00
Luke Heath
d20ccd65d7
Run workflow on push to branches beginning the minor- (#20549) 2024-07-17 12:31:12 -07:00
Noah Talerman
02f4bfb794
Dogfood: best practice teams (#20359) 2024-07-15 16:22:15 -07:00
Lucas Manuel Rodriguez
2875a9dbb8
Fixes to fleetctl debug connection and TLS certs documentation (#20166)
#6085

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-09 14:04:23 -03:00
Victor Lyuboslavsky
4a09cd1ce8
Validate base-fleetd daily (#20196)
#19126
Validate base-fleetd daily

Workflow run: https://github.com/fleetdm/fleet/actions/runs/9781375393
2024-07-09 09:54:01 -05:00
Roberto Dip
01030cbde6
test approach for running integration suite in parallel (#20085)
for #18297

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-07-01 11:37:12 -03:00
Victor Lyuboslavsky
6015717838
Releasing new fleetd-base flow (#20093)
#19182
Releasing new fleetd-base flow
- The flow has been QA'd
- The flow puts the generated files into new directories (`stable` and
`archive`), so risk is low
2024-06-28 12:57:14 -05:00
Dante Catalfamo
188b52c979
Generate a summary on any go-test failure, add context after panic (#20066)
Stops developers from having to manually grep through the logs to find
the failing test
2024-06-27 15:39:23 -04:00
Lucas Manuel Rodriguez
3c02ac1278
Add github action to check the auto-generated documentation is up-to-date (#20025)
Oncall project to keep auto-generated documentation up-to-date.
- Auto-generated documentation from Golang source code.
- Auto-generated table JSON schema from yaml files in `schema/tables/`
(we currently remind the developer to run the sails.js commands in the
PR or ask Eric to do it).

Sample of the failure if the developer forgot to run `make generate-doc`
(similar to `make test-db-schema`):
![Screenshot 2024-06-26 at 2 41
23 PM](https://github.com/fleetdm/fleet/assets/2073526/9bbfee24-f5cc-4ce5-bc90-5eb94231c24c)
2024-06-27 06:35:09 -03:00
Lucas Manuel Rodriguez
3a64c83145
Release fleetd 1.27.0 (#19933)
- **Release fleetd 1.27.0**

---------

Co-authored-by: George Karr <georgekarrv@users.noreply.github.com>
2024-06-21 16:00:13 -03:00
Lucas Manuel Rodriguez
addb665c8b
Dogfood iPhones team GitOps (#19897)
Changes to dogfood GitOps for #18866.
2024-06-20 17:13:24 -03:00
Luke Heath
0002b178ff
Schedule nightly gitops run in dogfood (#19816) 2024-06-17 14:12:17 -07:00
Victor Lyuboslavsky
d3b9bade74
Keep all fleetd-base and fleetd-chrome artifacts. (#19749)
#19182 and #19111

- Upload and keep all fleetd-base and fleetd-chrome artifacts
- Code sign fleetd-base.msi
- Verify checksums and try installing fleetd-base packages

These changes will apply the fleet-base workflow to
download-testing.fleetdm.com, and another PR will change to the
production endpoint (download.fleetdm.com) after QA.

## fleetd-base
Successful fleetd-base workflow run:
https://github.com/fleetdm/fleet/actions/runs/9522282299

New meta files will be in the `stable` directory:
- https://download-testing.fleetdm.com/stable/meta.json
- https://download-testing.fleetdm.com/stable/tuf-meta.json

The files in the root directory will no longer be updated for backward
compatibility.

## fleetd-chrome
Successful fleetd-chrome beta run:
https://github.com/fleetdm/fleet/actions/runs/9552391075/job/26328861033
2024-06-17 15:49:06 -05:00
Robert Fairburn
dcd551f671
initial osquery docker sidecar and osquery local builds (#19641) 2024-06-12 13:25:07 -05:00
Lucas Manuel Rodriguez
606635b131
Release fleetd 1.26.0 (#19673)
- **Release fleetd 1.26.0**

---------

Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
2024-06-11 19:51:57 -03:00
Roberto Dip
a24e665c13
Lock goreleaser version in CI to ~> 1 (#19529)
A few days ago, a new major version of goreleaser was published, which
is currently breaking our workflows:

```
⨯ command failed                                   error=unknown flag: --rm-dist
```

This locks the version to a max satisfying semver under 1 until we have
time to update to the new major.
2024-06-05 13:35:28 -03:00
Victor Lyuboslavsky
f761827850
Move CalculateAggregatedPerfStatsPercentiles reads to the replica (#19206)
Move CalculateAggregatedPerfStatsPercentiles reads to the replica
#18838

I manually tested the aggregated query stats change by using a read
replica.
https://github.com/fleetdm/fleet/blob/main/tools/mysql-replica-testing/README.md
- But set `master_delay=0` due to issue
https://github.com/fleetdm/fleet/issues/19272
- Run a saved query as a live query, and see that its stats in
`aggregated_stats` table are updated.


# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-31 07:08:31 -05:00
JD
94be5c0a9c
Adds VM team to gitops (#19316)
Adds Virtual machines team to gitops. fleetdm/confidential#6762

---------

Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
2024-05-29 11:19:24 -07:00
Lucas Manuel Rodriguez
a10befe657
Release fleetd 1.25.0 (#19203) 2024-05-22 14:38:24 -03:00
Victor Lyuboslavsky
d1b1c181f1
Refactoring fleetd-chrome workflows to use R2 upload action (#19157)
Refactoring fleetd-chrome workflows to use the reusable R2 upload
action.
2024-05-21 09:23:03 -05:00
Victor Lyuboslavsky
550f08d62d
Generate plist for fleetd-base pkg. (#19112)
#19111 
Generate plist for fleetd-base pkg.

Currently configured for TESTING. Uploaded file:
https://download-testing.fleetdm.com/fleetd-base-manifest.plist
2024-05-17 13:48:53 -05:00
Victor Lyuboslavsky
4d671e63d4
fleetd-chrome v1.3.1 release (#19087)
#18811

* Fixed bug where fleetd-chrome sent multiple read requests to Fleet
server at the same time.

* Improved console log output messages when Fleet server is down.
2024-05-16 16:39:36 -05:00