Commit graph

693 commits

Author SHA1 Message Date
Victor Lyuboslavsky
1b06b050d7
Fix issues with coverage uploads (#21736)
#21707
2024-09-03 09:07:16 -05:00
Lucas Manuel Rodriguez
ea7d08fc88
Release fleetd 1.32.0 (#21658)
Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
2024-08-29 18:51:10 -03:00
Noah Talerman
c29323c021
Update story template (#21670)
- "CLI changes" are for new fleetctl commands, options etc. fleetctl
wireframes go in Figma.
- "YAML changes" are for changes to Fleet's YAML used in GitOps
workflows. YAML wireframes are made as a draft PR to the YAML reference
[here](https://fleetdm.com/docs/configuration/yaml-files).
2024-08-29 14:18:32 -05:00
Lucas Manuel Rodriguez
8d4b7ad526
Remove unused workflow and dockerfile (#21551)
- `.github/workflows/push-osquery-perf-to-ecr.yml` has 0 workflow runs
(added but never used)
- `Dockerfile.osquery-perf` is only used by
`.github/workflows/push-osquery-perf-to-ecr.yml`.
2024-08-26 13:28:25 -03:00
Victor Lyuboslavsky
fdfc12982b
Improvements to go tests in CI (#21545)
#21546 
Some improvements to overall go test CI run time.
2024-08-26 08:55:53 -05:00
Eric
debb2d1790
Add app to manage scripts and profiles. (#21450)
Related to: #20296 

Changes:
- Added `ee/bulk-operations-dashboard`, a Sails.js app that lets users
manage configuration profiles and scripts across multiple teams on a
Fleet instance.
- Added a Github workflow to deploy the app to Heroku
- Added a Github workflow to test changes to the bulk operations
dashboard.
2024-08-22 14:59:15 -06:00
Lucas Manuel Rodriguez
ede0897acd
Pin trivy action (#21425)
Fixing code scanning warnings
https://github.com/fleetdm/fleet/security/code-scanning for the recently
added workflow.
2024-08-20 15:27:41 -03:00
Lucas Manuel Rodriguez
18f010f228
Update fleetdm/fleetctl, fleetdm/wix and fleetdm/bomutils docker images (#21063)
#20571

## Summary of changes

We have a few moving parts in fleetctl land (`fleetdm/wix` is used to
build `msi`s and `fleetdm/bomutils` is used to build `pkg`s, and
`fleetdm/fleetctl` can be used to build packages using docker, no need
for fleetctl executable):
```mermaid
graph LR

fleetctl_exec[fleetctl<br>executable];
wix_image[fleetdm/wix<br>docker image];
bomutils_image[fleetdm/bomutils<br>docker image];
fleetctl_image[fleetdm/fleetctl<br>docker image];

fleetctl_exec -- uses --> wix_image;

fleetctl_image -- COPY dependencies<br>FROM --> wix_image;

fleetctl_exec -- uses --> bomutils_image;

fleetctl_image -- COPY dependencies<br>FROM --> bomutils_image;
```
So, we'll need to update the three images: `fleetdm/bomutils`,
`fleetdm/wix` & `fleetdm/fleetctl`.

- `tools/bomutils-docker/Dockerfile`, `tools/wix-docker/Dockerfile` and
`tools/fleetctl-docker/Dockerfile`: Updating the base image to fix the
CRITICAL vulnerabilities.
- Modified existing+unused
`.github/workflows/build-and-check-fleetctl-docker-and-deps.yml` to run
every day to check for CRITICAL vulnerabilities in `fleetdm/wix`,
`fleetdm/bomutils` and `fleetdm/fleetctl`.
- `.github/workflows/goreleaser-fleetctl-docker-deps.yaml`:
`fleetdm/bomutils` and `fleetdm/wix` were pushed manually a few years
ago (most likely by Zach), so I've added a new action to release them
when we have changes to release (like now). It will basically release
`fleetctl/bomutils` and `fleetdm/wix` when pushing a tag of the form
`fleetctl-docker-deps-*` (we'll need to protect such tag prefix).
- Changes in `.github/workflows/test-native-tooling-packaging.yml` to
build `fleetdm/bomutils` and `fleetdm/wix` for `fleetdm/fleetctl` to use
them instead of the ones in docker hub.

--

Build before upgrading `debian:stable-slim`:

https://github.com/fleetdm/fleet/actions/runs/10255391418/job/28372231837
![Screenshot 2024-08-05 at 5 24
25 PM](https://github.com/user-attachments/assets/8a7d3576-3eb6-474f-989a-079873fca4fa)

Build after upgrading `debian:stable-slim`:
https://github.com/fleetdm/fleet/actions/runs/10255550034

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
2024-08-20 14:07:59 -03:00
Roberto Dip
f4820e2af9
Release fleetd 1.31.0 (#21391)
- **Release fleetd 1.30.0**
- **fix issue with disk encryption banner (#21385)**
- **Release fleetd 1.31.0**
2024-08-19 16:53:14 -03:00
Roberto Dip
936cc4a6d7
notify #help-engineering when a dogfood deploy is in progress (#21347)
This ensures `#help-engineering` is notified when a dogfood deploy is in
progress. It helps set people's expectations about what's going on while
the server is temporarily down.
2024-08-19 15:33:00 -03:00
Lucas Manuel Rodriguez
cdfa31ada5
Release osqueryd 5.13.1 (#21329) 2024-08-19 13:52:30 -03:00
Victor Lyuboslavsky
4eb72535dc
Support for MySQL 8.4.2 (#21364)
#21270

The main change for MySQL 8.4.2 is that foreign key constraints are
stricter:
https://dev.mysql.com/doc/refman/8.4/en/server-system-variables.html#sysvar_restrict_fk_on_non_standard_key

Also, most replica-related commands have been renamed.

# Checklist for submitter

- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] If database migrations are included, checked table schema to
confirm autoupdate
- [x] Manual QA for all new/changed functionality
2024-08-16 16:32:38 +02:00
Martin Angers
6220b2ed0e
Add checkbox to change frontend/CLI if API paths are modified (#21316) 2024-08-14 14:28:48 -04:00
Dante Catalfamo
9a5e5372eb
SSVPP Backend and CLI (#21132)
#19882 and #20532
2024-08-14 10:25:32 -04:00
George Karr
38edb2dbf7
Update release-qa.md (#21257)
Adding path to remember to verify Software install / download.
2024-08-13 23:37:27 -05:00
Luke Heath
34901a9dc7
Update goreleaser runner (#21226) 2024-08-09 10:50:43 -07:00
Roberto Dip
4853ecbf4e
Release fleetd 1.30.0 (#21071) 2024-08-06 07:32:24 -03:00
Lucas Manuel Rodriguez
ab7df5155d
Use docker compose on CI instead of docker-compose (#21017)
After this is merged I'll cherry pick to `minor-fleet-4.55.0`.
2024-08-02 18:12:36 -03:00
Lucas Manuel Rodriguez
2f479b3ba9
Release osqueryd 5.13.0 (#20949) 2024-08-02 14:57:50 -03:00
Dante Catalfamo
0a15647e10
Host software deleted at remigration (#20996)
# Recreate out of order migration, replace `docker-compose` with `docker compose` in db test runner
2024-08-02 10:47:40 -04:00
Lucas Manuel Rodriguez
6d87091a89
Release fleetd 1.29.0 (#20700) 2024-07-26 15:04:03 -03:00
Roberto Dip
a33a110f70
fix link in PR template to documentation about changes file (#20752) 2024-07-26 10:00:53 -03:00
Dante Catalfamo
5e1a3d03ae
MySQL 8.0 Migration (#20225)
#17249
2024-07-22 16:27:36 -04:00
Victor Lyuboslavsky
8c1c016b54
Don't stop unit tests if Go integration tests fail (#20628) 2024-07-20 19:22:07 +02:00
Lucas Manuel Rodriguez
84a81bafde
Release fleetd 1.28.0 (#20581)
Co-authored-by: Luke Heath <luke@fleetdm.com>
2024-07-19 12:47:00 -03:00
Luke Heath
9c701a59d1
Move reference docs responsibility back to product check list (#20551) 2024-07-17 13:16:47 -07:00
Dante Catalfamo
ecf2346ace
Add support for Linux ARM64 (#19931)
#1845

Co-authored-by: Lucas Manuel Rodriguez <lucas@fleetdm.com>
Co-authored-by: Roberto Dip <rroperzh@gmail.com>
2024-07-17 16:07:59 -04:00
Luke Heath
d20ccd65d7
Run workflow on push to branches beginning the minor- (#20549) 2024-07-17 12:31:12 -07:00
Noah Talerman
02f4bfb794
Dogfood: best practice teams (#20359) 2024-07-15 16:22:15 -07:00
Lucas Manuel Rodriguez
73a8936568
Add orbit checklist to pull request template (#20417)
To help prevent issues like #20168.
2024-07-12 14:42:17 -03:00
Lucas Manuel Rodriguez
2875a9dbb8
Fixes to fleetctl debug connection and TLS certs documentation (#20166)
#6085

- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-07-09 14:04:23 -03:00
Victor Lyuboslavsky
4a09cd1ce8
Validate base-fleetd daily (#20196)
#19126
Validate base-fleetd daily

Workflow run: https://github.com/fleetdm/fleet/actions/runs/9781375393
2024-07-09 09:54:01 -05:00
Luke Heath
9d9975136b
Move user story documentation responsibilities to engineering (#20207) 2024-07-03 14:28:15 -07:00
Zay Hanlon
dbe846a612
Update feature-request.md (#20158)
Updated this template to try and get you more information and
fully flushed out FR's
2024-07-03 16:11:35 -05:00
Roberto Dip
01030cbde6
test approach for running integration suite in parallel (#20085)
for #18297

# Checklist for submitter

If some of the following don't apply, delete the relevant line.

<!-- Note that API documentation changes are now addressed by the
product design team. -->

- [x] Manual QA for all new/changed functionality
2024-07-01 11:37:12 -03:00
Victor Lyuboslavsky
6015717838
Releasing new fleetd-base flow (#20093)
#19182
Releasing new fleetd-base flow
- The flow has been QA'd
- The flow puts the generated files into new directories (`stable` and
`archive`), so risk is low
2024-06-28 12:57:14 -05:00
Dante Catalfamo
188b52c979
Generate a summary on any go-test failure, add context after panic (#20066)
Stops developers from having to manually grep through the logs to find
the failing test
2024-06-27 15:39:23 -04:00
Lucas Manuel Rodriguez
3c02ac1278
Add github action to check the auto-generated documentation is up-to-date (#20025)
Oncall project to keep auto-generated documentation up-to-date.
- Auto-generated documentation from Golang source code.
- Auto-generated table JSON schema from yaml files in `schema/tables/`
(we currently remind the developer to run the sails.js commands in the
PR or ask Eric to do it).

Sample of the failure if the developer forgot to run `make generate-doc`
(similar to `make test-db-schema`):
![Screenshot 2024-06-26 at 2 41
23 PM](https://github.com/fleetdm/fleet/assets/2073526/9bbfee24-f5cc-4ce5-bc90-5eb94231c24c)
2024-06-27 06:35:09 -03:00
Gabe Lopez
caec222fe9
Update release-qa.md (#19993)
added MDM tests for APNs and ABM cert and token renewal workflows.
Replaced old QA Engineer with PezHub. Also updated a few typos.
2024-06-25 15:43:52 -05:00
Rachael Shaw
1604010c66
Update story issue template (#19953)
Add note about manually requesting reviews on API design draft PRs,
because reviews aren't automatically requested until PRs are marked
ready for review.

cc @marko-lisica @noahtalerman
2024-06-24 10:23:59 -05:00
Lucas Manuel Rodriguez
3a64c83145
Release fleetd 1.27.0 (#19933)
- **Release fleetd 1.27.0**

---------

Co-authored-by: George Karr <georgekarrv@users.noreply.github.com>
2024-06-21 16:00:13 -03:00
Lucas Manuel Rodriguez
addb665c8b
Dogfood iPhones team GitOps (#19897)
Changes to dogfood GitOps for #18866.
2024-06-20 17:13:24 -03:00
Luke Heath
0002b178ff
Schedule nightly gitops run in dogfood (#19816) 2024-06-17 14:12:17 -07:00
Victor Lyuboslavsky
d3b9bade74
Keep all fleetd-base and fleetd-chrome artifacts. (#19749)
#19182 and #19111

- Upload and keep all fleetd-base and fleetd-chrome artifacts
- Code sign fleetd-base.msi
- Verify checksums and try installing fleetd-base packages

These changes will apply the fleet-base workflow to
download-testing.fleetdm.com, and another PR will change to the
production endpoint (download.fleetdm.com) after QA.

## fleetd-base
Successful fleetd-base workflow run:
https://github.com/fleetdm/fleet/actions/runs/9522282299

New meta files will be in the `stable` directory:
- https://download-testing.fleetdm.com/stable/meta.json
- https://download-testing.fleetdm.com/stable/tuf-meta.json

The files in the root directory will no longer be updated for backward
compatibility.

## fleetd-chrome
Successful fleetd-chrome beta run:
https://github.com/fleetdm/fleet/actions/runs/9552391075/job/26328861033
2024-06-17 15:49:06 -05:00
Noah Talerman
443564b553
Update story issue template (#19735)
- Add "Fleet's agent (fleetd) changes" section. Response to the
following bug:
  - https://github.com/fleetdm/fleet/issues/19736

More info in Slack
[here](https://fleetdm.slack.com/archives/C03C41L5YEL/p1718299265992889?thread_ts=1718298355.630389&cid=C03C41L5YEL)
(internal).
2024-06-13 14:24:09 -05:00
Robert Fairburn
dcd551f671
initial osquery docker sidecar and osquery local builds (#19641) 2024-06-12 13:25:07 -05:00
Lucas Manuel Rodriguez
606635b131
Release fleetd 1.26.0 (#19673)
- **Release fleetd 1.26.0**

---------

Co-authored-by: Sharon Katz <121527325+sharon-fdm@users.noreply.github.com>
2024-06-11 19:51:57 -03:00
Noah Talerman
436aaf1eaa
Release QA issue template (#19631)
- Test the migration workflow each release
2024-06-11 18:14:43 -04:00
Roberto Dip
a24e665c13
Lock goreleaser version in CI to ~> 1 (#19529)
A few days ago, a new major version of goreleaser was published, which
is currently breaking our workflows:

```
⨯ command failed                                   error=unknown flag: --rm-dist
```

This locks the version to a max satisfying semver under 1 until we have
time to update to the new major.
2024-06-05 13:35:28 -03:00
Victor Lyuboslavsky
f761827850
Move CalculateAggregatedPerfStatsPercentiles reads to the replica (#19206)
Move CalculateAggregatedPerfStatsPercentiles reads to the replica
#18838

I manually tested the aggregated query stats change by using a read
replica.
https://github.com/fleetdm/fleet/blob/main/tools/mysql-replica-testing/README.md
- But set `master_delay=0` due to issue
https://github.com/fleetdm/fleet/issues/19272
- Run a saved query as a live query, and see that its stats in
`aggregated_stats` table are updated.


# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
2024-05-31 07:08:31 -05:00