> For #26083
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Added/updated automated tests
- [x] Manual QA for all new/changed functionality
---------
Co-authored-by: Ian Littman <iansltx@gmail.com>
From what I can tell, continue-on-error has been false for the
integration suites since the suites were renamed to `integration-*`, so
this fixes that issue in addition to continuing to run test suites when
the vulns suite fails (which may be due to vulns feed updates).
This also makes the vulns test more resilient to new CVEs being reported
on Python 3.12.0, which is rather likely to collect new CVEs.
# Checklist for submitter
- [x] Added/updated automated tests
Switched from metadata_url to metadata for end user authentication.
---------
Co-authored-by: Noah Talerman <47070608+noahtalerman@users.noreply.github.com>
This is the initial pull request to implement keeping policy logic up to
date automatically. For example, when a new version of macOS releases,
admins don't need to manually update the policy logic for checking
version numbers.
This is currently blocked by this issue: fleetdm/confidential#9470
This is also to support the following issue and demonstrate to customers
a fully automated patch management strategy:
https://github.com/fleetdm/confidential/issues/8825
This current iteration contains a script/workflow that runs every 6
hours to check if a new version of macOS has been released and compares
the version string to what is currently defined in our policy. If it
detects a change, it will automatically create a new branch with the
updated version string and create a pull request to be reviewed before
merging.
For #25349
This updates storybook and its addons to 8.4.7. This is done to remove
the transitive dependency on path-to-regexp,
which is no longer used in this version of storybook.
This will fix the original vulnerability issue for `path-to-regexp`
For #25507
A bump to the latest version to the github `cache` action to 4.2.0. our
current version (v2) was deprecated. more info for the deprecation can
be found here https://github.com/actions/cache/discussions/1510
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
# Changes
- orbit >= 1.38.0, when configured to connect to
https://tuf.fleetctl.com (existing fleetd deployments) will now connect
to https://updates.fleetdm.com and start using the metadata in path
`/opt/orbit/updates-metadata.json`.
- orbit >= 1.38.0, when configured to connect to some custom TUF (not
Fleet's TUFs) will copy `/opt/orbit/tuf-metadata.json` to
`/opt/orbit/updates-metadata.json` (if it doesn't exist) and start using
the latter.
- fleetctl `4.63.0` will now generate artifacts using
https://updates.fleetdm.com by default (or a custom TUF if
`--update-url` is set) and generate two (same file) metadata files
`/opt/orbit/updates-metadata.json` and the legacy one to support
downgrades `/opt/orbit/tuf-metadata.json`.
- fleetctl `4.62.0` when configured to use custom TUF (not Fleet's TUF)
will generate just the legacy metadata file
`/opt/orbit/tuf-metadata.json`.
## User stories
See "User stories" in
https://github.com/fleetdm/confidential/issues/8488.
- [x] Update `update.defaultRootMetadata` and `update.DefaultURL` when
the new repository is ready.
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [X] Added/updated tests
- [X] Manual QA for all new/changed functionality
- For Orbit and Fleet Desktop changes:
- [X] Orbit runs on macOS, Linux and Windows. Check if the orbit
feature/bugfix should only apply to one platform (`runtime.GOOS`).
- [X] Manual QA must be performed in the three main OSs, macOS, Windows
and Linux.
- [X] Auto-update manual QA, from released version of component to new
version (see [tools/tuf/test](../tools/tuf/test/README.md)).
See failed workflow run
[here](https://github.com/fleetdm/fleet/actions/runs/12555703803)
- Fix the powershell script that was broken by `.yml` auto-format
- Exclude github workflow `.yml` files from prettier autoformating,
since they often contain non-yaml code as part of job definitions
- [ ] Manual QA for all new/changed functionality
---------
Co-authored-by: Jacob Shandling <jacob@fleetdm.com>
for #23825
This PR fixes the previous implementation for attesting
fleet/fleetctl/orbit binaries, and adds attestation to the fleet desktop
and osqueryd artifacts.
* correct permissions are added to all jobs
* tag removed from `subject-name` when attesting docker image
* using `artifacts.json` rather than the `artifacts` step output from
goreleaser to determine image digest
I'd like to add a separate job verifying the attestations, working on
that now but since all attestation steps are marked as
`continue-on-error` it can be a follow-on if we don't get it in with
this PR.
This PR adds a new workflow called "Stress Test Go Test" (aka the
RandoKiller) that allows running one or more tests repeatedly up to a
set number of times, or until a test fails. This is useful for:
* Trying to diagnose and debug a flaky test
* Verifying that a proposed fix for a flaky test actually works.
To use:
1. Create a branch whose name ends with "-randokiller"
2. Modify the .github/workflows/config/randokiller.json file to your
specifications (choosing the packages and tests to run, the mysql
matrix, and the number of runs to do)
3. Push up the branch
Since the stress test is intended to run a branch that you'll never
merge, you should feel free to add whatever logs to your tests or code
that will help diagnose failures.
I used this to diagnose and fix
https://github.com/fleetdm/fleet/pull/24697!
for #19106
This PR adds a Slack notification when the GitOps run fails in the
dogfood-gitops workflow. Whenever the actual GitOps action fails, it
should notify #help-dogfooding with a link to the failed action. Note
that this will alert on both merges to main and scheduled runs, which I
think we want. Also note that this is [currently failing on
main](https://github.com/fleetdm/fleet/actions/runs/12154006118) so this
alert will start going off daily until the issue is fixed 😶
### > Note: this will need a new Slack incoming webhook for sending
messages to #help-dogfooding, and a new
`SLACK_G_HELP_DOGFOODING_WEBHOOK_URL` repo secret with the webhook URL.
I tested this on a personal private repo just to make sure I got all the
syntax right:
<img width="422" alt="image"
src="https://github.com/user-attachments/assets/74d188eb-5c03-471b-a5db-9f578a56e2ab">
I beleive we don't need this step anymore, since `fleetctl gitops` will
replace it with real value and send to the server. This should be done
in #17309.
[#8489](https://github.com/fleetdm/confidential/issues/8489)
We had the timestamp check.
Robert added the root check recently.
Am now duplicating the check for `snapshot` and `targets` metadata
files.
PS: Please review with whitespace changes disabled.
#23905
- Update with upstream nanomdm changes up to
825f2979a2
- Removed PostgeSQL folder from our nanomdm
- Added nanomdm MySQL test job to our CI
# Checklist for submitter
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Added/updated tests
- [x] Manual QA for all new/changed functionality
From discussions with @jahzielv.
QAing ADE flows:
1. New version of fleetd is pushed to `edge`
2. QA folks can trigger this new workflow and download the generated
`fleetd-base.pkg` and `fleetd-base-manifest.plist`.
3. Host the downloaded files (in `foobar/`) in their ngroks URLs (using
e.g. `go tools ./tools/file-server 8085 foobar/`)
4. Use Fleet's `FLEET_DEV_DOWNLOAD_FLEETDM_URL` to point the Fleet
server to their ngrok URL.
Closes: https://github.com/fleetdm/confidential/issues/8351
Changes:
- Deleted the "Deploy app to bulk operations dashboard pipeline on
Heroku" workflow. This dashboard is now hosted in Render, and deploys
are triggered manually via the Render dashboard.
Closes: #22931
Changes:
- Updated the deploy workflows for the Fleet website and the
vulnerability dashboard to run on Ubuntu 22.04 to prevent issues we've
been seeing with the Heroku deploy action and the latest version of
Ubuntu.
#22206
This was discussed in the backend weekly.
Currently the test-packaging.yml is extremely unreliable (it has more
failures than successes), because of issues with Docker and colima on
Github macOS runners (we tried docker then colima but both have issues,
timeouts, etc.).
This only removes testing of MSI package generation from macOS. IMO this
is low risk as almost all Fleet devs generate MSI packages from their
macOS workstations.
This fix applies to cases (e.g.
00ec402f18) where order of files being
added is e.g.:
1. Migration A
2. Migration B
3. Test for migration A
This also reorders workflow steps so the ones that don't require setting
up Go + compiling happen first, so if we have a migration issue it gets
reported sooner.
# Checklist for submitter
- [x] Manual QA for all new/changed functionality
Also mention that we test with 8.4.2 in a few more places.
Note that while I'm editing release articles, this isn't retconning
minimum requirements; we mention in 4.55.0 release notes further down
that we expect 8.0.36.
Done as part of oncall improvements.
`vars.GO_VERSION` can only be changed by admins and it's not public
(Fleet devs don't know the current value of the variable), this approach
uses the version specified in our `go.mod` file.
- `.github/workflows/push-osquery-perf-to-ecr.yml` has 0 workflow runs
(added but never used)
- `Dockerfile.osquery-perf` is only used by
`.github/workflows/push-osquery-perf-to-ecr.yml`.
Related to: #20296
Changes:
- Added `ee/bulk-operations-dashboard`, a Sails.js app that lets users
manage configuration profiles and scripts across multiple teams on a
Fleet instance.
- Added a Github workflow to deploy the app to Heroku
- Added a Github workflow to test changes to the bulk operations
dashboard.
#20571
## Summary of changes
We have a few moving parts in fleetctl land (`fleetdm/wix` is used to
build `msi`s and `fleetdm/bomutils` is used to build `pkg`s, and
`fleetdm/fleetctl` can be used to build packages using docker, no need
for fleetctl executable):
```mermaid
graph LR
fleetctl_exec[fleetctl<br>executable];
wix_image[fleetdm/wix<br>docker image];
bomutils_image[fleetdm/bomutils<br>docker image];
fleetctl_image[fleetdm/fleetctl<br>docker image];
fleetctl_exec -- uses --> wix_image;
fleetctl_image -- COPY dependencies<br>FROM --> wix_image;
fleetctl_exec -- uses --> bomutils_image;
fleetctl_image -- COPY dependencies<br>FROM --> bomutils_image;
```
So, we'll need to update the three images: `fleetdm/bomutils`,
`fleetdm/wix` & `fleetdm/fleetctl`.
- `tools/bomutils-docker/Dockerfile`, `tools/wix-docker/Dockerfile` and
`tools/fleetctl-docker/Dockerfile`: Updating the base image to fix the
CRITICAL vulnerabilities.
- Modified existing+unused
`.github/workflows/build-and-check-fleetctl-docker-and-deps.yml` to run
every day to check for CRITICAL vulnerabilities in `fleetdm/wix`,
`fleetdm/bomutils` and `fleetdm/fleetctl`.
- `.github/workflows/goreleaser-fleetctl-docker-deps.yaml`:
`fleetdm/bomutils` and `fleetdm/wix` were pushed manually a few years
ago (most likely by Zach), so I've added a new action to release them
when we have changes to release (like now). It will basically release
`fleetctl/bomutils` and `fleetdm/wix` when pushing a tag of the form
`fleetctl-docker-deps-*` (we'll need to protect such tag prefix).
- Changes in `.github/workflows/test-native-tooling-packaging.yml` to
build `fleetdm/bomutils` and `fleetdm/wix` for `fleetdm/fleetctl` to use
them instead of the ones in docker hub.
--
Build before upgrading `debian:stable-slim`:
https://github.com/fleetdm/fleet/actions/runs/10255391418/job/28372231837
Build after upgrading `debian:stable-slim`:
https://github.com/fleetdm/fleet/actions/runs/10255550034
- [x] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://github.com/fleetdm/fleet/blob/main/docs/Contributing/Committing-Changes.md#changes-files)
for more information.
- [x] Manual QA for all new/changed functionality
This ensures `#help-engineering` is notified when a dogfood deploy is in
progress. It helps set people's expectations about what's going on while
the server is temporarily down.
#6085
- [X] Changes file added for user-visible changes in `changes/`,
`orbit/changes/` or `ee/fleetd-chrome/changes`.
See [Changes
files](https://fleetdm.com/docs/contributing/committing-changes#changes-files)
for more information.
- [X] Added/updated tests
- [x] Manual QA for all new/changed functionality
for #18297
# Checklist for submitter
If some of the following don't apply, delete the relevant line.
<!-- Note that API documentation changes are now addressed by the
product design team. -->
- [x] Manual QA for all new/changed functionality
#19182
Releasing new fleetd-base flow
- The flow has been QA'd
- The flow puts the generated files into new directories (`stable` and
`archive`), so risk is low
Oncall project to keep auto-generated documentation up-to-date.
- Auto-generated documentation from Golang source code.
- Auto-generated table JSON schema from yaml files in `schema/tables/`
(we currently remind the developer to run the sails.js commands in the
PR or ask Eric to do it).
Sample of the failure if the developer forgot to run `make generate-doc`
(similar to `make test-db-schema`):
A few days ago, a new major version of goreleaser was published, which
is currently breaking our workflows:
```
⨯ command failed error=unknown flag: --rm-dist
```
This locks the version to a max satisfying semver under 1 until we have
time to update to the new major.
#18811
* Fixed bug where fleetd-chrome sent multiple read requests to Fleet
server at the same time.
* Improved console log output messages when Fleet server is down.
The `macos-latest` runner is using `macos-14` + ARM now, which was
causing the Docker install to fail.
I switched to `macos-13` since seems to be a cheap x86_64 alternative
and figured what was the problem with Colima so we don't have to deal
with Docker anymore.
