fleet/.github/workflows
Victor Lyuboslavsky 0d095b3778
Add workflows to collect engineering metrics. (#30540)
Fixes #29140 

This is an engineering initiated story that does not impact product.
This code has been running and manually tested in my own repo:
https://github.com/getvictor/eng-metrics

See
[README.md](https://github.com/fleetdm/fleet/blob/victor/29140-eng-metrics/.github/actions/eng-metrics/README.md)
in this branch for details.

The metrics can be viewed on
https://fleeteng.grafana.net/d/b97a629f-3626-4a28-9781-0fa3c8427897/engineering-metrics
(credentials in 1Password)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Introduced an engineering metrics collection tool that gathers GitHub
metrics (e.g., Time to First Review, Time to Merge) and uploads them to
BigQuery.
* Added support for user group management and product group mapping via
markdown parsing.
* Enabled print-only mode for testing metrics output without uploading
to BigQuery.
* Added automatic handling of bot filtering, weekend-aware time
calculations, and differential syncing of user groups.
* Implemented robust GitHub username validation and retry logic for API
rate limits.

* **Documentation**
* Added comprehensive usage and configuration documentation for the
engineering metrics tool.

* **Chores**
* Added configuration, environment example, and workflow files for
automated metrics collection and testing.
* Specified Node.js version and set up project dependencies and scripts.

* **Tests**
* Added extensive unit and end-to-end test suites to ensure reliability
of metrics collection, configuration, and integrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2025-07-03 16:59:25 -05:00
..
config Speculative fix for flaky TestVPPApps. (#25385) 2025-01-13 16:28:48 -06:00
build-and-check-fleetctl-docker-and-deps.yml Fix detected CVEs and docker scout exit code to fail the Github Action (#28836) 2025-05-06 13:35:27 -03:00
build-binaries.yaml Use go 1.24 new tool feature (#27765) 2025-04-07 11:12:05 -03:00
build-fleetd-base-msi.yml Address code scanning permissions warnings (#27250) 2025-03-20 12:07:41 -05:00
build-fleetd-base-pkg.yml Update deprecated actions/upload-artifact to v4.5.0 (#25295) 2025-01-09 12:08:02 -06:00
build-fleetd_tables.yaml Use go 1.24 new tool feature (#27765) 2025-04-07 11:12:05 -03:00
build-orbit.yaml Update deprecated actions/upload-artifact missed in previous PR. (#25724) 2025-01-23 10:50:55 -06:00
check-automated-doc.yml Add help system to Makefile + FDM command (#25028) 2025-02-28 07:42:32 -06:00
check-tuf-timestamps.yml Check for timestamps on the new TUF repository (#26638) 2025-02-27 17:59:11 -03:00
check-updates-timestamps.yml Check for timestamps on the new TUF repository (#26638) 2025-02-27 17:59:11 -03:00
check-vulnerabilities-in-released-docker-images.yml Add scanning to released images and process to track vulnerabilities (#28087) 2025-04-16 11:50:10 -03:00
close-stale-eng-initiated-issues.yml Add stale issues workflow (#27047) 2025-03-14 16:11:43 -05:00
code-sign-windows.yml Windows orbit.exe and fleet-desktop.exe are now signed. (#18201) 2024-04-26 12:46:23 -05:00
codeql-analysis.yml Attempt to use go.mod version instead of hidden Github var (#21768) 2024-09-03 20:49:50 -03:00
collect-eng-metrics-test.yml Add workflows to collect engineering metrics. (#30540) 2025-07-03 16:59:25 -05:00
collect-eng-metrics.yml Add workflows to collect engineering metrics. (#30540) 2025-07-03 16:59:25 -05:00
db-upgrade-test.yml Simplify DB test/upgrade tool (#27141) 2025-03-14 17:07:41 -03:00
dependency-review.yml Update dependency-review-action (#29910) 2025-06-11 11:15:48 -03:00
deploy-fleet-website.yml Update node version used in website workflows. (#25605) 2025-01-20 14:33:43 -06:00
deploy-vulnerability-dashboard.yml Change Ubuntu version in Heroku deploy workflows (#22939) 2024-10-15 16:20:12 -05:00
docs.yml Fail CI if Markdown files have "here" or "click here" as link anchors (#30027) 2025-06-19 10:12:31 -05:00
dogfood-automated-policy-updates.yml Added policy automation and new Windows installer (#27244) 2025-03-19 13:54:25 -05:00
dogfood-deploy.yml added env vars for webhook osquery results logging destination (#29809) 2025-06-16 10:22:31 -05:00
dogfood-gitops.yml Update SSO settings for dogfood (#28435) 2025-04-22 10:40:34 -05:00
fleet-and-orbit.yml Pin version of cloudflared to 2025.5.0 (#30179) 2025-06-20 13:20:19 -03:00
fleetctl-preview-latest.yml Prevent starting browser in fleetctl preview (#30424) 2025-06-30 10:41:42 -04:00
fleetctl-preview.yml Prevent starting browser in fleetctl preview (#30424) 2025-06-30 10:41:42 -04:00
fleetd-tuf.yml Update TUF status generation to use new TUF repository (#26099) 2025-02-07 08:30:07 -03:00
generate-desktop-targets.yml Orbit for Windows ARM64 (#27882) 2025-04-11 10:18:28 -04:00
generate-nudge-targets.yml Update deprecated actions/upload-artifact to v4.5.0 (#25295) 2025-01-09 12:08:02 -06:00
generate-osqueryd-targets.yml Release osqueryd 5.18.1 (#30348) 2025-06-26 18:35:50 -03:00
golangci-lint.yml Fix Windows lint issues and enable linting on Windows (#28704) 2025-05-02 16:11:26 -04:00
goreleaser-fleet.yaml Move GitHub token to correct step (#30022) (#30023) 2025-06-14 14:36:03 -05:00
goreleaser-orbit.yaml Update changelog for fleetd 1.42.0 release (#29186) 2025-05-19 08:22:30 -03:00
goreleaser-snapshot-fleet.yaml Fix detected CVEs and docker scout exit code to fail the Github Action (#28836) 2025-05-06 13:35:27 -03:00
ingest-maintained-apps.yml Drop assignees from FMA automated PRs (#29049) 2025-05-12 12:54:38 -05:00
integration.yml Pin version of cloudflared to 2025.5.0 (#30179) 2025-06-20 13:20:19 -03:00
pr-helm.yaml Update ubuntu-20.04 to ubuntu-22.04 on CI actions due to deprecation (#26466) 2025-02-21 13:11:50 -03:00
randokiller-go.yml Use go 1.24 new tool feature (#27765) 2025-04-07 11:12:05 -03:00
README.md add concurrency to ci (#8271) 2022-10-24 14:01:00 -06:00
release-fleetctl-docker-deps.yaml Use go 1.24 new tool feature (#27765) 2025-04-07 11:12:05 -03:00
release-fleetd-base.yml Add retry to fleetd base pkg build. (#24489) 2024-12-09 13:24:38 -06:00
release-fleetd-chrome-beta.yml Keep all fleetd-base and fleetd-chrome artifacts. (#19749) 2024-06-17 15:49:06 -05:00
release-fleetd-chrome.yml Keep all fleetd-base and fleetd-chrome artifacts. (#19749) 2024-06-17 15:49:06 -05:00
release-helm.yaml Update ubuntu-20.04 to ubuntu-22.04 on CI actions due to deprecation (#26466) 2025-02-21 13:11:50 -03:00
render-deploy.yml Added Render deploy workflow for fleet-gitops CI. (#23190) 2024-10-25 15:55:42 -05:00
scorecards-analysis.yml Update deprecated actions/upload-artifact to v4.5.0 (#25295) 2025-01-09 12:08:02 -06:00
secrets-to-confidential.yml Initial action to synchronize signing secrets to confidential repo (#30561) 2025-07-03 16:45:39 -05:00
test-bulk-operations-dashboard-changes.yml Add app to manage scripts and profiles. (#21450) 2024-08-22 14:59:15 -06:00
test-db-changes.yml add a test that checks collation on new migrations (#29309) 2025-05-29 17:00:30 -04:00
test-fleetd-chrome.yml bump action/cache to version 4.2.0 (#25508) 2025-01-17 15:01:27 +00:00
test-go.yaml Remove unneeded exposed ports on osquery-in-a-box minio to avoid host-port conflicts (#30416) 2025-06-29 12:40:17 -05:00
test-js.yml update storybook to 8.4.7 (#25451) 2025-01-20 16:17:33 +00:00
test-native-tooling-packaging.yml Use go 1.24 new tool feature (#27765) 2025-04-07 11:12:05 -03:00
test-packaging-build-docker-deps.yml Use go 1.24 new tool feature (#27765) 2025-04-07 11:12:05 -03:00
test-packaging.yml Use go 1.24 new tool feature (#27765) 2025-04-07 11:12:05 -03:00
test-puppet.yml [StepSecurity] ci: Harden GitHub Actions (#17780) 2024-03-22 15:32:23 -05:00
test-vulnerability-dashboard-changes.yml [StepSecurity] ci: Harden GitHub Actions (#17780) 2024-03-22 15:32:23 -05:00
test-website.yml Update node version used in website workflows. (#25605) 2025-01-20 14:33:43 -06:00
test-yml-specs.yml Attempt to use go.mod version instead of hidden Github var (#21768) 2024-09-03 20:49:50 -03:00
tfvalidate.yml Update terraform version used in tfvalidate (#24699) 2024-12-12 11:25:26 -06:00
trivy-scan.yml Fix detected CVEs and docker scout exit code to fail the Github Action (#28836) 2025-05-06 13:35:27 -03:00
update-certs.yml Add reviewers to automated PRs (#18390) 2024-04-18 10:51:07 -03:00
update-old-tuf-timestamp-signature.yaml Add workflow to update timestamp on new repository (#26635) 2025-02-27 18:02:42 -03:00
update-osquery-versions.yml [StepSecurity] ci: Harden GitHub Actions (#25985) 2025-02-03 12:25:18 -06:00
update-tuf-timestamp-signature.yaml Fix code scanning alerts (#27074) 2025-03-27 10:01:20 -03:00
verify-fleetd-base.yml Increase sleep time in verify-fleetd-base.yml (#27763) 2025-04-02 14:30:04 -03:00

Github Actions

Fleet uses Github Actions for continuous integration (CI). This document describes best practices and at patterns for writing and maintaining Fleet's Github Actions workflows.

Bash

By default, Github Actions sets the shell to bash -e for linux and MacOS runners. To help write safer bash scripts in run jobs and avoid common issues, override the default by adding the following to the workflow file

defaults:
  run:
    # fail-fast using bash -eo pipefail. See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#exit-codes-and-error-action-preference
    shell: bash

By specifying the default shell to bash, some extra flags are set. The option pipefail changes the behaviour when using the pipe | operator such that if any command in a pipeline fails, that commands return code will be used a the return code for the whole pipeline. Consider the following example in test-go.yaml

    - name: Run Go Tests
      run: |
        # omitted ...
          make test-go 2>&1 | tee /tmp/gotest.log

If the pipefail option was not set, this job would always succeed because tee would always return success. This is not the intended behavior. Instead, we want the job to fail if make test-go fails.

Concurrency

Github Action runners are limited. If a lot of workflows are queued, they will wait in pending until a runner becomes available. This has caused issue in the past where workflows take an excessively long time to start. To help with this issue, use the following in workflows

# This allows a subsequently queued workflow run to interrupt previous runs
concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.run_id}}
  cancel-in-progress: true

When a workflow is triggered via a pull request, it will cancel previous running workflows for that pull request. This is especially useful when changes are pushed to a pull request frequently. Manually triggered workflows, workflows that run on a schedule, and workflows triggered by pushes to main are unaffected.