mirror of
https://github.com/fleetdm/fleet
synced 2026-05-23 17:08:53 +00:00
Add workflows to collect engineering metrics. (#30540)
Fixes #29140 This is an engineering initiated story that does not impact product. This code has been running and manually tested in my own repo: https://github.com/getvictor/eng-metrics See [README.md](https://github.com/fleetdm/fleet/blob/victor/29140-eng-metrics/.github/actions/eng-metrics/README.md) in this branch for details. The metrics can be viewed on https://fleeteng.grafana.net/d/b97a629f-3626-4a28-9781-0fa3c8427897/engineering-metrics (credentials in 1Password) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Introduced an engineering metrics collection tool that gathers GitHub metrics (e.g., Time to First Review, Time to Merge) and uploads them to BigQuery. * Added support for user group management and product group mapping via markdown parsing. * Enabled print-only mode for testing metrics output without uploading to BigQuery. * Added automatic handling of bot filtering, weekend-aware time calculations, and differential syncing of user groups. * Implemented robust GitHub username validation and retry logic for API rate limits. * **Documentation** * Added comprehensive usage and configuration documentation for the engineering metrics tool. * **Chores** * Added configuration, environment example, and workflow files for automated metrics collection and testing. * Specified Node.js version and set up project dependencies and scripts. * **Tests** * Added extensive unit and end-to-end test suites to ensure reliability of metrics collection, configuration, and integrations. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
parent
dce722cc07
commit
0d095b3778
30 changed files with 12885 additions and 0 deletions
17
.github/actions/eng-metrics/.env.example
vendored
Normal file
17
.github/actions/eng-metrics/.env.example
vendored
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
# GitHub API token with repo scope
|
||||
GITHUB_TOKEN=your_github_token_here
|
||||
|
||||
# Repositories to track (comma-separated list)
|
||||
# REPOSITORIES=octocat/Hello-World,example-org/example-repo
|
||||
|
||||
# BigQuery dataset and table IDs (optional, defaults in config.json)
|
||||
# BIGQUERY_DATASET_ID=github_metrics
|
||||
|
||||
# Path to the service account key file (optional, default in config.json)
|
||||
# SERVICE_ACCOUNT_KEY_PATH=./service-account-key.json
|
||||
|
||||
# Target branch to track PRs for (optional, default: main)
|
||||
# TARGET_BRANCH=main
|
||||
|
||||
# Print-only mode (set to 'true' to print metrics to console instead of uploading to BigQuery)
|
||||
# PRINT_ONLY=true
|
||||
1
.github/actions/eng-metrics/.nvmrc
vendored
Normal file
1
.github/actions/eng-metrics/.nvmrc
vendored
Normal file
|
|
@ -0,0 +1 @@
|
|||
20.18.1
|
||||
308
.github/actions/eng-metrics/README.md
vendored
Normal file
308
.github/actions/eng-metrics/README.md
vendored
Normal file
|
|
@ -0,0 +1,308 @@
|
|||
# Engineering metrics collector
|
||||
|
||||
A comprehensive tool to collect GitHub engineering metrics and upload them to BigQuery. This tool can be used as a standalone application or as a GitHub Action to track various development workflow metrics.
|
||||
|
||||
## Supported Metrics
|
||||
|
||||
```mermaid
|
||||
---
|
||||
title: Software development life cycle (SDLC)
|
||||
---
|
||||
graph TD
|
||||
requirements[Understand the why]
|
||||
design[Design solution]
|
||||
implement[Implement code]
|
||||
|
||||
subgraph Time_to_Merge["⏱️ Time to Merge"]
|
||||
subgraph Time_to_First_Review["⏱️ Time to First Review"]
|
||||
pr_ready_for_review[PR ready for review]
|
||||
first_review[Receive first review]
|
||||
end
|
||||
first_review --> rework["Changes requested &<br/>re-review (optional)"]
|
||||
rework --> code_approved[Code approved]
|
||||
code_approved --> pr_merged[Merge pull request]
|
||||
end
|
||||
|
||||
testing[QA]
|
||||
ready_for_release[Ready for release]
|
||||
release[Release]
|
||||
monitoring[Monitoring & feedback]
|
||||
|
||||
requirements --> design
|
||||
design --> implement
|
||||
implement --> pr_ready_for_review
|
||||
pr_ready_for_review --> first_review
|
||||
pr_merged --> testing
|
||||
testing --> ready_for_release
|
||||
ready_for_release --> release
|
||||
release --> monitoring
|
||||
```
|
||||
|
||||
This tool collects the following engineering metrics:
|
||||
|
||||
### 1. Time to First Review
|
||||
Previously known as "PR Pickup Time" - measures the time between when a PR is marked as "Ready for Review" and when a reviewer has submitted a review.
|
||||
|
||||
- **Start Time**: When a PR is marked as "Ready for Review" - this can be:
|
||||
- When a PR is created as a non-draft PR
|
||||
- When a draft PR is converted to ready for review
|
||||
- If multiple ready_for_review events exist, the tool uses the most recent one that occurred before the first review
|
||||
- **End Time**: When the first review submission occurs (comment, approval, or changes requested)
|
||||
- **Metric**: The time difference between these two events, excluding weekends
|
||||
|
||||
### 2. Time to Merge
|
||||
Measures the time from when a PR is marked as "Ready for Review" to when it is merged.
|
||||
|
||||
- **Start Time**: When a PR is marked as "Ready for Review"
|
||||
- **End Time**: When the PR is merged into the target branch
|
||||
- **Metric**: The time difference between these two events, excluding weekends
|
||||
|
||||
### 3. User Group Management
|
||||
Extracts GitHub usernames from a product groups markdown file and syncs them to BigQuery for team analytics.
|
||||
|
||||
- **Source**: Markdown file containing product group tables with GitHub usernames
|
||||
- **Processing**: Validates GitHub usernames and creates dual group membership (specific group + engineering)
|
||||
- **Storage**: Syncs to BigQuery `user_group` table with differential updates
|
||||
- **Groups**: Supports MDM, Orchestration, and Software groups with automatic engineering group inclusion
|
||||
|
||||
### 4. Time to QA Ready (Planned)
|
||||
Measures the time from issue reaching "In Progress" status to when it reaches "Awaiting QA" status in GitHub Projects.
|
||||
|
||||
- **Start Time**: When an issue is moved to "In Progress"
|
||||
- **End Time**: When the issue status changes to "Awaiting QA" or "Ready for release" in GitHub Projects
|
||||
- **Metric**: The time difference between these two events, excluding weekends
|
||||
|
||||
### 5. Time to Production Ready (Planned)
|
||||
Measures the time from issue reaching "In Progress" status to when it reaches "Ready for Release" status in GitHub Projects.
|
||||
|
||||
- **Start Time**: When an issue is moved to "In Progress"
|
||||
- **End Time**: When the issue status changes to "Ready for Release" in GitHub Projects
|
||||
- **Metric**: The time difference between these two events, excluding weekends
|
||||
|
||||
## Current Implementation Status
|
||||
|
||||
- ✅ **Time to First Review**: Fully implemented and active
|
||||
- ✅ **Time to Merge**: Fully implemented and active
|
||||
- ✅ **User Group Management**: Fully implemented and active
|
||||
- 🚧 **Time to QA Ready**: Planned for future implementation
|
||||
- 🚧 **Time to Production Ready**: Planned for future implementation
|
||||
|
||||
## Features
|
||||
|
||||
- Collects engineering metrics from GitHub repositories
|
||||
- Uploads metrics to Google BigQuery for analysis
|
||||
- Configurable via JSON file and environment variables
|
||||
- Can run as a standalone application or as a GitHub Action
|
||||
- Supports multiple repositories (not tested)
|
||||
- Only tracks PRs targeting the main branch
|
||||
- Excludes weekends from time calculations
|
||||
- Supports print-only mode for testing without BigQuery
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 20 or higher
|
||||
- A GitHub token with `public_repo` scope or `repo` scope (for private repos)
|
||||
- A Google Cloud project with BigQuery enabled
|
||||
- A Google Cloud service account with BigQuery permissions
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Clone the repo and go here
|
||||
git clone <repo.git>
|
||||
cd <repo_dir>/.github/actions/eng-metrics
|
||||
|
||||
# Install dependencies
|
||||
npm install
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Configuration File
|
||||
|
||||
Create a `config.json` file with the following structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"repositories": [
|
||||
"owner/repo1",
|
||||
"owner/repo2"
|
||||
],
|
||||
"targetBranch": "main",
|
||||
"bigQueryDatasetId": "github_metrics",
|
||||
"lookbackDays": 5,
|
||||
"serviceAccountKeyPath": "./service-account-key.json",
|
||||
"printOnly": false,
|
||||
"userGroupEnabled": true,
|
||||
"userGroupFilepath": "../../../handbook/company/product-groups.md",
|
||||
"metrics": {
|
||||
"timeToFirstReview": {
|
||||
"enabled": true,
|
||||
"tableName": "pr_first_review"
|
||||
},
|
||||
"timeToMerge": {
|
||||
"enabled": true,
|
||||
"tableName": "pr_merge"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
You can also configure the tool using environment variables:
|
||||
|
||||
- `GITHUB_TOKEN`: GitHub token with repo scope
|
||||
- `REPOSITORIES`: Comma-separated list of repositories to track (optional, overrides config.json)
|
||||
- `BIGQUERY_DATASET_ID`: BigQuery dataset ID (optional, defaults to config.json)
|
||||
- `SERVICE_ACCOUNT_KEY_PATH`: Path to the service account key file (optional, overrides config.json)
|
||||
- `TARGET_BRANCH`: Target branch to track PRs for (optional, default: main)
|
||||
- `PRINT_ONLY`: Set to 'true' to print metrics to console instead of uploading to BigQuery
|
||||
- `ENABLED_METRICS`: Comma-separated list of metrics to collect (e.g., "time_to_first_review,time_to_merge")
|
||||
- `TIME_TO_FIRST_REVIEW_TABLE`: Override table name for Time to First Review metrics (optional, defaults to "pr_first_review")
|
||||
- `TIME_TO_MERGE_TABLE`: Override table name for Time to Merge metrics (optional, defaults to "pr_merge")
|
||||
- `USER_GROUP_ENABLED`: Set to 'true' to enable user group processing (optional, defaults to false)
|
||||
- `USER_GROUP_FILEPATH`: Path to the product groups markdown file (optional, defaults to "../../../handbook/company/product-groups.md")
|
||||
|
||||
Create a `.env` file based on the provided `.env.example` to set these variables.
|
||||
|
||||
### Configuration Priority
|
||||
|
||||
The tool uses the following configuration priority order (highest priority overrides lower priority):
|
||||
|
||||
1. **Environment Variables** (highest priority) - Values from `.env` file or system environment
|
||||
2. **JSON Configuration File** (medium priority) - Values from `config.json` or specified config file
|
||||
3. **Default Configuration** (lowest priority) - Built-in default values
|
||||
|
||||
This means that environment variables will always override values in the JSON configuration file, and both will override any default values. For example, if you have `"printOnly": false` in your `config.json` file but set `PRINT_ONLY=true` in your `.env` file, the tool will run in print-only mode.
|
||||
|
||||
## Usage
|
||||
|
||||
### As a Standalone Application
|
||||
|
||||
```bash
|
||||
# Run with default config.json
|
||||
npm start
|
||||
|
||||
# Run with a custom config file
|
||||
npm start -- path/to/config.json
|
||||
|
||||
# Run in print-only mode (no BigQuery upload)
|
||||
npm start -- --print-only
|
||||
|
||||
# Run with a custom config file in print-only mode
|
||||
npm start -- path/to/config.json --print-only
|
||||
```
|
||||
|
||||
### As a GitHub Action
|
||||
|
||||
See example in .github/workflows/collect-eng-metrics.yml
|
||||
|
||||
#### How the GitHub Action Works
|
||||
|
||||
1. **Service Account Key Handling**:
|
||||
- The workflow writes the `ENG_METRICS_GCP_SERVICE_ACCOUNT_KEY` secret directly to a file
|
||||
- It verifies that the file contains valid JSON using `jq`
|
||||
- It then sets the `SERVICE_ACCOUNT_KEY_PATH` environment variable to point to this file
|
||||
- The application uses this environment variable to locate the service account key file
|
||||
|
||||
**Important**: The service account key should be stored as a JSON string in the GitHub secret. Copy the entire contents of your service account key JSON file to the secret value.
|
||||
|
||||
2. **Configuration**:
|
||||
- The workflow passes configuration values as environment variables
|
||||
- Environment variables are used directly by the application
|
||||
|
||||
Make sure to set the following secrets in your repository:
|
||||
|
||||
- `GITHUB_TOKEN`: GitHub token with repo scope (automatically provided by GitHub Actions)
|
||||
- `ENG_METRICS_GCP_SERVICE_ACCOUNT_KEY`: JSON service account key as a string
|
||||
|
||||
## BigQuery Schema
|
||||
|
||||
### Multi-Table Architecture
|
||||
|
||||
The tool uses separate BigQuery tables for different metric types to optimize performance and enable independent analysis:
|
||||
|
||||
#### Table 1: `pr_first_review` (Time to First Review)
|
||||
|
||||
| Field | Type | Description |
|
||||
|---------------------|-----------|------------------------------------------------------------------------------|
|
||||
| review_date | DATE | Date when the reviewer started looking at the PR |
|
||||
| pr_creator | STRING | GitHub username of the PR creator (cluster key) |
|
||||
| pr_url | STRING | HTTP link to the PR |
|
||||
| pickup_time_seconds | INTEGER | Time in seconds from "Ready for Review" to first review (excluding weekends) |
|
||||
| repository | STRING | Repository name (owner/repo) |
|
||||
| pr_number | INTEGER | PR number (cluster key) |
|
||||
| target_branch | STRING | Branch the PR is targeting (always "main") |
|
||||
| ready_time | TIMESTAMP | Timestamp when PR was marked ready for review |
|
||||
| first_review_time | TIMESTAMP | Timestamp of first review activity (partition key) |
|
||||
|
||||
#### Table 2: `pr_merge` (Time to Merge)
|
||||
|
||||
| Field | Type | Description |
|
||||
|--------------------|-----------|-----------------------------------------------------------------------|
|
||||
| merge_date | DATE | Date when the PR was merged |
|
||||
| pr_creator | STRING | GitHub username of the PR creator (cluster key) |
|
||||
| pr_url | STRING | HTTP link to the PR |
|
||||
| merge_time_seconds | INTEGER | Time in seconds from "Ready for Review" to merge (excluding weekends) |
|
||||
| repository | STRING | Repository name (owner/repo) |
|
||||
| pr_number | INTEGER | PR number (cluster key) |
|
||||
| target_branch | STRING | Branch the PR is targeting (always "main") |
|
||||
| ready_time | TIMESTAMP | Timestamp when PR was marked ready for review |
|
||||
| merge_time | TIMESTAMP | Timestamp when PR was merged (partition key) |
|
||||
|
||||
#### Table 3: `user_group` (User Group Management)
|
||||
|
||||
| Field | Type | Description |
|
||||
|----------|--------|----------------------------------------------------------------------|
|
||||
| group | STRING | Group name (mdm, orchestration, software, engineering) - cluster key |
|
||||
| username | STRING | GitHub username |
|
||||
|
||||
**Table Optimizations:**
|
||||
- `user_group` table is clustered by `group` for efficient group-based queries
|
||||
- Supports dual group membership (specific group + engineering group for all users)
|
||||
- Uses differential sync to minimize BigQuery operations
|
||||
- Records are managed through intelligent insert/delete operations
|
||||
- Due to BigQuery buffering, records cannot be updated/deleted within a 90-minute timeframe. This is OK as this update should run once per day.
|
||||
|
||||
**Multi-Table Optimizations:**
|
||||
- `pr_first_review` table is partitioned by `DATE(first_review_time)` for efficient date-range queries
|
||||
- `pr_merge` table is partitioned by `DATE(merge_time)` for efficient date-range queries
|
||||
- `pr_first_review` and `pr_merge` tables are clustered by `pr_creator` and `pr_number` for efficient user-based analysis
|
||||
- Each table uses `pr_number` as unique identifier (enforced at application level)
|
||||
- Records are insert-only (no updates) to preserve historical data integrity
|
||||
|
||||
## Print-Only Mode
|
||||
|
||||
The tool supports a print-only mode that prints metrics to the console instead of uploading them to BigQuery. This is useful for testing without setting up BigQuery.
|
||||
|
||||
To enable print-only mode:
|
||||
|
||||
1. Set `printOnly: true` in your config.json file, OR
|
||||
2. Set the `PRINT_ONLY=true` environment variable, OR
|
||||
3. Use the `--print-only` command line flag
|
||||
|
||||
When running in print-only mode, you don't need to provide BigQuery credentials or configuration.
|
||||
|
||||
## Development
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
npm test
|
||||
```
|
||||
|
||||
### Linting
|
||||
|
||||
```bash
|
||||
npm run lint
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
When contributing to this project, please:
|
||||
|
||||
1. Update tests for any new metrics or functionality
|
||||
2. Update documentation to reflect changes
|
||||
3. Follow the existing code style and patterns
|
||||
4. Consider backward compatibility for BigQuery schema changes
|
||||
10
.github/actions/eng-metrics/action.yml
vendored
Normal file
10
.github/actions/eng-metrics/action.yml
vendored
Normal file
|
|
@ -0,0 +1,10 @@
|
|||
name: 'Engineering metrics collector'
|
||||
description: 'Collects comprehensive GitHub engineering metrics including time to first review, time to merge, and GitHub Projects workflow metrics, then uploads them to BigQuery for analysis'
|
||||
|
||||
runs:
|
||||
using: 'node20'
|
||||
main: 'src/index.js'
|
||||
|
||||
branding:
|
||||
icon: 'bar-chart-2'
|
||||
color: 'blue'
|
||||
22
.github/actions/eng-metrics/config.json
vendored
Normal file
22
.github/actions/eng-metrics/config.json
vendored
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
{
|
||||
"repositories": [
|
||||
"fleetdm/fleet"
|
||||
],
|
||||
"targetBranch": "main",
|
||||
"bigQueryDatasetId": "github_metrics",
|
||||
"lookbackDays": 5,
|
||||
"serviceAccountKeyPath": "./service-account-key.json",
|
||||
"printOnly": false,
|
||||
"userGroupEnabled": true,
|
||||
"userGroupFilepath": "../../../handbook/company/product-groups.md",
|
||||
"metrics": {
|
||||
"timeToFirstReview": {
|
||||
"enabled": true,
|
||||
"tableName": "pr_first_review"
|
||||
},
|
||||
"timeToMerge": {
|
||||
"enabled": true,
|
||||
"tableName": "pr_merge"
|
||||
}
|
||||
}
|
||||
}
|
||||
53
.github/actions/eng-metrics/eslint.config.js
vendored
Normal file
53
.github/actions/eng-metrics/eslint.config.js
vendored
Normal file
|
|
@ -0,0 +1,53 @@
|
|||
import js from '@eslint/js';
|
||||
|
||||
export default [
|
||||
js.configs.recommended,
|
||||
{
|
||||
languageOptions: {
|
||||
ecmaVersion: 2022,
|
||||
sourceType: 'module',
|
||||
globals: {
|
||||
console: 'readonly',
|
||||
process: 'readonly',
|
||||
Buffer: 'readonly',
|
||||
__dirname: 'readonly',
|
||||
__filename: 'readonly',
|
||||
global: 'readonly',
|
||||
setTimeout: 'readonly',
|
||||
clearTimeout: 'readonly',
|
||||
setInterval: 'readonly',
|
||||
clearInterval: 'readonly'
|
||||
}
|
||||
},
|
||||
rules: {
|
||||
'indent': ['error', 2],
|
||||
'linebreak-style': ['error', 'unix'],
|
||||
'quotes': ['error', 'single'],
|
||||
'semi': ['error', 'always'],
|
||||
'no-unused-vars': ['error', { 'argsIgnorePattern': '^_' }],
|
||||
'no-console': 'off',
|
||||
'prefer-const': 'error',
|
||||
'no-var': 'error',
|
||||
'object-shorthand': 'error',
|
||||
'prefer-arrow-callback': 'error'
|
||||
}
|
||||
},
|
||||
{
|
||||
files: ['test/**/*.js'],
|
||||
languageOptions: {
|
||||
globals: {
|
||||
describe: 'readonly',
|
||||
test: 'readonly',
|
||||
expect: 'readonly',
|
||||
beforeEach: 'readonly',
|
||||
afterEach: 'readonly',
|
||||
beforeAll: 'readonly',
|
||||
afterAll: 'readonly',
|
||||
jest: 'readonly'
|
||||
}
|
||||
},
|
||||
rules: {
|
||||
'no-unused-vars': ['error', { 'argsIgnorePattern': '^_', 'varsIgnorePattern': '^_' }]
|
||||
}
|
||||
}
|
||||
];
|
||||
41
.github/actions/eng-metrics/jest.config.js
vendored
Normal file
41
.github/actions/eng-metrics/jest.config.js
vendored
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
/**
|
||||
* Jest configuration for engineering metrics tests
|
||||
*/
|
||||
|
||||
export default {
|
||||
// Test environment
|
||||
testEnvironment: "node",
|
||||
|
||||
// Transform configuration for ES modules
|
||||
transform: {},
|
||||
|
||||
// File extensions to consider
|
||||
moduleFileExtensions: ["js", "json"],
|
||||
|
||||
// Test file patterns
|
||||
testMatch: ["**/test/**/*.test.js"],
|
||||
|
||||
// Coverage configuration
|
||||
collectCoverage: false, // Disable for now to focus on functionality
|
||||
coverageDirectory: "coverage",
|
||||
coverageReporters: ["text", "lcov", "html"],
|
||||
|
||||
// Files to collect coverage from
|
||||
collectCoverageFrom: [
|
||||
"src/**/*.js",
|
||||
"!src/index.js", // Exclude main entry point
|
||||
"!src/logger.js", // Exclude logger (simple utility)
|
||||
],
|
||||
|
||||
// Setup files
|
||||
setupFilesAfterEnv: ["<rootDir>/test/setup.js"],
|
||||
|
||||
// Clear mocks between tests
|
||||
clearMocks: true,
|
||||
|
||||
// Restore mocks after each test
|
||||
restoreMocks: true,
|
||||
|
||||
// Verbose output
|
||||
verbose: true,
|
||||
};
|
||||
6571
.github/actions/eng-metrics/package-lock.json
generated
vendored
Normal file
6571
.github/actions/eng-metrics/package-lock.json
generated
vendored
Normal file
File diff suppressed because it is too large
Load diff
44
.github/actions/eng-metrics/package.json
vendored
Normal file
44
.github/actions/eng-metrics/package.json
vendored
Normal file
|
|
@ -0,0 +1,44 @@
|
|||
{
|
||||
"name": "engineering-metrics-collector",
|
||||
"version": "1.0.0",
|
||||
"description": "Comprehensive GitHub engineering metrics collector - tracks time to first review, time to merge, and GitHub Projects workflow metrics",
|
||||
"type": "module",
|
||||
"main": "src/index.js",
|
||||
"directories": {
|
||||
"test": "test"
|
||||
},
|
||||
"scripts": {
|
||||
"start": "node src/index.js",
|
||||
"start:print": "node src/index.js --print-only",
|
||||
"test": "node --experimental-vm-modules ./node_modules/jest/bin/jest.js",
|
||||
"test:watch": "node --experimental-vm-modules ./node_modules/jest/bin/jest.js --watch",
|
||||
"test:coverage": "node --experimental-vm-modules ./node_modules/jest/bin/jest.js --coverage",
|
||||
"test:ci": "node --experimental-vm-modules ./node_modules/jest/bin/jest.js --ci --coverage --watchAll=false",
|
||||
"lint": "./node_modules/.bin/eslint src/**/*.js",
|
||||
"lint:fix": "./node_modules/.bin/eslint src/**/*.js --fix"
|
||||
},
|
||||
"keywords": [
|
||||
"github",
|
||||
"engineering-metrics",
|
||||
"developer-productivity",
|
||||
"time-to-first-review",
|
||||
"time-to-merge",
|
||||
"github-projects",
|
||||
"workflow-metrics",
|
||||
"bigquery",
|
||||
"analytics"
|
||||
],
|
||||
"dependencies": {
|
||||
"@google-cloud/bigquery": "^8.1.0",
|
||||
"dotenv": "^17.0.0",
|
||||
"octokit": "^5.0.3"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^9.30.0",
|
||||
"eslint": "^9.30.0",
|
||||
"jest": "^30.0.3"
|
||||
},
|
||||
"engines": {
|
||||
"node": "20.18.1"
|
||||
}
|
||||
}
|
||||
400
.github/actions/eng-metrics/src/bigquery-client.js
vendored
Normal file
400
.github/actions/eng-metrics/src/bigquery-client.js
vendored
Normal file
|
|
@ -0,0 +1,400 @@
|
|||
/**
|
||||
* BigQuery client module for PR pickup time metrics collector
|
||||
* Handles authentication and data upload to BigQuery
|
||||
*/
|
||||
|
||||
import { BigQuery } from '@google-cloud/bigquery';
|
||||
import fs from 'fs';
|
||||
import logger from './logger.js';
|
||||
|
||||
/**
|
||||
* BigQuery client class
|
||||
*/
|
||||
export class BigQueryClient {
|
||||
/**
|
||||
* Creates a new BigQuery client
|
||||
* @param {string} keyFilePath - Path to the service account key file
|
||||
*/
|
||||
constructor(keyFilePath) {
|
||||
this.bigquery = null;
|
||||
this.projectId = null;
|
||||
this.initialize(keyFilePath);
|
||||
}
|
||||
|
||||
/**
|
||||
* Initializes the BigQuery client
|
||||
* @param {string} keyFilePath - Path to the service account key file
|
||||
*/
|
||||
initialize(keyFilePath) {
|
||||
// Check if the key file exists
|
||||
if (!fs.existsSync(keyFilePath)) {
|
||||
const err = new Error(
|
||||
`Service account key file not found at ${keyFilePath}`
|
||||
);
|
||||
logger.error('Failed to initialize BigQuery client', {}, err);
|
||||
throw err;
|
||||
}
|
||||
|
||||
try {
|
||||
// Read and parse the service account key to extract project_id
|
||||
const keyFileContent = fs.readFileSync(keyFilePath, 'utf8');
|
||||
const serviceAccountKey = JSON.parse(keyFileContent);
|
||||
|
||||
if (!serviceAccountKey.project_id) {
|
||||
throw new Error(
|
||||
'Service account key file must contain a project_id field'
|
||||
);
|
||||
}
|
||||
|
||||
this.projectId = serviceAccountKey.project_id;
|
||||
logger.info(
|
||||
`Extracted project ID from service account key: ${this.projectId}`
|
||||
);
|
||||
|
||||
this.bigquery = new BigQuery({
|
||||
keyFilename: keyFilePath,
|
||||
projectId: this.projectId,
|
||||
});
|
||||
|
||||
logger.info('BigQuery client initialized');
|
||||
} catch (err) {
|
||||
logger.error('Failed to initialize BigQuery client', {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the project ID extracted from the service account key
|
||||
* @returns {string} Project ID
|
||||
*/
|
||||
getProjectId() {
|
||||
return this.projectId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates a table if it doesn't exist with table-specific configuration
|
||||
* @param {string} datasetId - BigQuery dataset ID
|
||||
* @param {string} tableId - BigQuery table ID
|
||||
* @param {Object} schema - BigQuery table schema
|
||||
* @param {string} metricType - Type of metric (e.g., 'time_to_first_review', 'time_to_merge')
|
||||
*/
|
||||
async createTableIfNotExists(datasetId, tableId, schema, metricType) {
|
||||
try {
|
||||
// Get a reference to the dataset
|
||||
const dataset = this.bigquery.dataset(datasetId);
|
||||
|
||||
// Check if the dataset exists, create it if it doesn't
|
||||
const [datasetExists] = await dataset.exists();
|
||||
|
||||
if (!datasetExists) {
|
||||
logger.info(`Dataset ${datasetId} does not exist, creating it`);
|
||||
await dataset.create();
|
||||
logger.info(`Dataset ${datasetId} created`);
|
||||
}
|
||||
|
||||
// Get a reference to the table
|
||||
const table = dataset.table(tableId);
|
||||
|
||||
// Check if the table exists, create it if it doesn't
|
||||
const [tableExists] = await table.exists();
|
||||
|
||||
if (!tableExists) {
|
||||
logger.info(`Table ${tableId} does not exist, creating it`);
|
||||
|
||||
// Get table-specific configuration based on metric type
|
||||
const tableConfig = this.getConfigurationForMetricType(metricType);
|
||||
|
||||
const options = {
|
||||
schema,
|
||||
timePartitioning: tableConfig.timePartitioning,
|
||||
clustering: tableConfig.clustering,
|
||||
};
|
||||
|
||||
await table.create(options);
|
||||
logger.info(`Table ${tableId} created`);
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error(`Error creating table ${datasetId}.${tableId}`, {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets table-specific configuration for partitioning and clustering based on metric type
|
||||
* @param {string} metricType - Type of metric (e.g., 'time_to_first_review', 'time_to_merge')
|
||||
* @returns {Object} Table configuration
|
||||
*/
|
||||
getConfigurationForMetricType(metricType) {
|
||||
switch (metricType) {
|
||||
case 'time_to_first_review':
|
||||
return {
|
||||
timePartitioning: {
|
||||
type: 'DAY',
|
||||
field: 'first_review_time',
|
||||
},
|
||||
clustering: {
|
||||
fields: ['pr_creator', 'pr_number'],
|
||||
},
|
||||
};
|
||||
|
||||
case 'time_to_merge':
|
||||
return {
|
||||
timePartitioning: {
|
||||
type: 'DAY',
|
||||
field: 'merge_time',
|
||||
},
|
||||
clustering: {
|
||||
fields: ['pr_creator', 'pr_number'],
|
||||
},
|
||||
};
|
||||
|
||||
default:
|
||||
throw new Error(
|
||||
`Unknown metric type for table configuration: ${metricType}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the BigQuery table schema for a specific metric type
|
||||
* @param {string} metricType - Type of metric (e.g., 'time_to_first_review', 'time_to_merge')
|
||||
* @returns {Object} BigQuery table schema
|
||||
*/
|
||||
getSchemaForMetricType(metricType) {
|
||||
switch (metricType) {
|
||||
case 'time_to_first_review':
|
||||
return {
|
||||
fields: [
|
||||
{ name: 'review_date', type: 'DATE', mode: 'REQUIRED' },
|
||||
{ name: 'pr_creator', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'pr_url', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'pickup_time_seconds', type: 'INTEGER', mode: 'REQUIRED' },
|
||||
{ name: 'repository', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'pr_number', type: 'INTEGER', mode: 'REQUIRED' },
|
||||
{ name: 'target_branch', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'ready_time', type: 'TIMESTAMP', mode: 'REQUIRED' },
|
||||
{ name: 'first_review_time', type: 'TIMESTAMP', mode: 'REQUIRED' },
|
||||
],
|
||||
};
|
||||
|
||||
case 'time_to_merge':
|
||||
return {
|
||||
fields: [
|
||||
{ name: 'merge_date', type: 'DATE', mode: 'REQUIRED' },
|
||||
{ name: 'pr_creator', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'pr_url', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'merge_time_seconds', type: 'INTEGER', mode: 'REQUIRED' },
|
||||
{ name: 'repository', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'pr_number', type: 'INTEGER', mode: 'REQUIRED' },
|
||||
{ name: 'target_branch', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'ready_time', type: 'TIMESTAMP', mode: 'REQUIRED' },
|
||||
{ name: 'merge_time', type: 'TIMESTAMP', mode: 'REQUIRED' },
|
||||
],
|
||||
};
|
||||
|
||||
default:
|
||||
throw new Error(`Unknown metric type: ${metricType}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Transforms metrics to BigQuery row format based on metric type
|
||||
* @param {Object} metrics - Metrics object
|
||||
* @returns {Object} BigQuery row
|
||||
*/
|
||||
transformMetricsToRow(metrics) {
|
||||
switch (metrics.metricType) {
|
||||
case 'time_to_first_review':
|
||||
return {
|
||||
review_date: metrics.reviewDate,
|
||||
pr_creator: metrics.prCreator,
|
||||
pr_url: metrics.prUrl,
|
||||
pickup_time_seconds: metrics.pickupTimeSeconds,
|
||||
repository: metrics.repository,
|
||||
pr_number: metrics.prNumber,
|
||||
target_branch: metrics.targetBranch,
|
||||
ready_time: metrics.readyTime.toISOString(),
|
||||
first_review_time: metrics.firstReviewTime.toISOString(),
|
||||
};
|
||||
|
||||
case 'time_to_merge':
|
||||
return {
|
||||
merge_date: metrics.mergeDate,
|
||||
pr_creator: metrics.prCreator,
|
||||
pr_url: metrics.prUrl,
|
||||
merge_time_seconds: metrics.mergeTimeSeconds,
|
||||
repository: metrics.repository,
|
||||
pr_number: metrics.prNumber,
|
||||
target_branch: metrics.targetBranch,
|
||||
ready_time: metrics.readyTime.toISOString(),
|
||||
merge_time: metrics.mergeTime.toISOString(),
|
||||
};
|
||||
|
||||
default:
|
||||
throw new Error(`Unknown metric type: ${metrics.metricType}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Checks if metrics already exist in BigQuery for the given PR numbers
|
||||
* @param {string} datasetId - BigQuery dataset ID
|
||||
* @param {string} tableId - BigQuery table ID
|
||||
* @param {Array} prNumbers - Array of PR numbers to check
|
||||
* @returns {Object} Object with PR numbers as keys and boolean values indicating if they exist
|
||||
*/
|
||||
async checkExistingMetrics(datasetId, tableId, prNumbers) {
|
||||
try {
|
||||
// Get a reference to the table
|
||||
const table = this.bigquery.dataset(datasetId).table(tableId);
|
||||
|
||||
// Check if the table exists
|
||||
const [tableExists] = await table.exists();
|
||||
if (!tableExists) {
|
||||
// If the table doesn't exist, no metrics exist
|
||||
return prNumbers.reduce((acc, prNumber) => {
|
||||
acc[prNumber] = false;
|
||||
return acc;
|
||||
}, {});
|
||||
}
|
||||
|
||||
// Create a parameterized query to check for existing PR numbers
|
||||
const query = `
|
||||
SELECT pr_number
|
||||
FROM \`${this.projectId}.${datasetId}.${tableId}\`
|
||||
WHERE pr_number IN UNNEST(@prNumbers)
|
||||
`;
|
||||
|
||||
const options = {
|
||||
query,
|
||||
params: {
|
||||
prNumbers,
|
||||
},
|
||||
};
|
||||
|
||||
// Run the query
|
||||
const [rows] = await this.bigquery.query(options);
|
||||
|
||||
// Create a map of existing PR numbers
|
||||
const existingPRs = rows.reduce((acc, row) => {
|
||||
acc[row.pr_number] = true;
|
||||
return acc;
|
||||
}, {});
|
||||
|
||||
// Return a map of all PR numbers with their existence status
|
||||
return prNumbers.reduce((acc, prNumber) => {
|
||||
acc[prNumber] = !!existingPRs[prNumber];
|
||||
return acc;
|
||||
}, {});
|
||||
} catch (err) {
|
||||
logger.error(
|
||||
`Error checking existing metrics in BigQuery ${datasetId}.${tableId}`,
|
||||
{},
|
||||
err
|
||||
);
|
||||
// If there's an error, assume no metrics exist
|
||||
return prNumbers.reduce((acc, prNumber) => {
|
||||
acc[prNumber] = false;
|
||||
return acc;
|
||||
}, {});
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Uploads metrics to BigQuery
|
||||
* @param {string} datasetId - BigQuery dataset ID
|
||||
* @param {string} tableId - BigQuery table ID
|
||||
* @param {Array} metrics - Array of metrics
|
||||
*/
|
||||
async uploadMetrics(datasetId, tableId, metrics) {
|
||||
try {
|
||||
if (!metrics || metrics.length === 0) {
|
||||
logger.warn('No metrics to upload');
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info(
|
||||
`Uploading ${metrics.length} metrics to BigQuery table ${tableId}`
|
||||
);
|
||||
|
||||
// Get metric type from the first metric to determine schema
|
||||
const metricType = metrics[0]?.metricType;
|
||||
if (!metricType) {
|
||||
throw new Error('Metrics must have a metricType field');
|
||||
}
|
||||
|
||||
// Ensure the table exists with the correct schema
|
||||
const schema = this.getSchemaForMetricType(metricType);
|
||||
await this.createTableIfNotExists(datasetId, tableId, schema, metricType);
|
||||
|
||||
// Get all PR numbers from the metrics
|
||||
const prNumbers = metrics.map((metric) => metric.prNumber);
|
||||
|
||||
// Check which PR numbers already exist in BigQuery
|
||||
const existingMetrics = await this.checkExistingMetrics(
|
||||
datasetId,
|
||||
tableId,
|
||||
prNumbers
|
||||
);
|
||||
|
||||
// Filter out metrics that already exist
|
||||
const newMetrics = metrics.filter(
|
||||
(metric) => !existingMetrics[metric.prNumber]
|
||||
);
|
||||
|
||||
if (newMetrics.length === 0) {
|
||||
logger.info('All metrics already exist in BigQuery, nothing to upload');
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info(
|
||||
`Uploading ${newMetrics.length} new metrics to BigQuery (${
|
||||
metrics.length - newMetrics.length
|
||||
} already exist)`
|
||||
);
|
||||
|
||||
// Transform metrics to BigQuery row format
|
||||
const rows = newMetrics.map((metric) =>
|
||||
this.transformMetricsToRow(metric)
|
||||
);
|
||||
|
||||
// Get a reference to the table
|
||||
const table = this.bigquery.dataset(datasetId).table(tableId);
|
||||
|
||||
// Upload the rows to BigQuery
|
||||
const [apiResponse] = await table.insert(rows);
|
||||
|
||||
logger.info(
|
||||
`Successfully uploaded ${newMetrics.length} metrics to BigQuery`,
|
||||
{
|
||||
datasetId,
|
||||
tableId,
|
||||
insertedRows: newMetrics.length,
|
||||
skippedRows: metrics.length - newMetrics.length,
|
||||
}
|
||||
);
|
||||
|
||||
return apiResponse;
|
||||
} catch (err) {
|
||||
logger.error(
|
||||
`Error uploading metrics to BigQuery ${datasetId}.${tableId}`,
|
||||
{},
|
||||
err
|
||||
);
|
||||
|
||||
// Log more details about the error if it's an insertion error
|
||||
if (
|
||||
err.name === 'PartialFailureError' &&
|
||||
err.errors &&
|
||||
err.errors.length > 0
|
||||
) {
|
||||
err.errors.forEach((error, index) => {
|
||||
logger.error(`Row ${index} error:`, { error });
|
||||
});
|
||||
}
|
||||
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export default BigQueryClient;
|
||||
273
.github/actions/eng-metrics/src/config.js
vendored
Normal file
273
.github/actions/eng-metrics/src/config.js
vendored
Normal file
|
|
@ -0,0 +1,273 @@
|
|||
/**
|
||||
* Configuration module for engineering metrics collector
|
||||
* Loads and validates configuration from files and environment variables
|
||||
*/
|
||||
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import dotenv from 'dotenv';
|
||||
import logger from './logger.js';
|
||||
|
||||
// Load environment variables from .env file
|
||||
dotenv.config();
|
||||
|
||||
/**
|
||||
* Default configuration values
|
||||
*/
|
||||
const DEFAULT_CONFIG = {
|
||||
// Default target branch to track PRs for
|
||||
targetBranch: 'main',
|
||||
|
||||
// Default BigQuery dataset ID
|
||||
bigQueryDatasetId: 'github_metrics',
|
||||
|
||||
// Default time window for fetching PRs (in days)
|
||||
lookbackDays: 5,
|
||||
|
||||
// Default print-only mode (false = upload to BigQuery, true = print to console)
|
||||
printOnly: false,
|
||||
|
||||
// User group management configuration
|
||||
userGroupEnabled: true,
|
||||
userGroupFilepath: '../../../handbook/company/product-groups.md',
|
||||
|
||||
// Bot filtering configuration
|
||||
excludeBotReviews: true,
|
||||
|
||||
// Multi-table configuration
|
||||
metrics: {
|
||||
timeToFirstReview: {
|
||||
enabled: true,
|
||||
tableName: 'pr_first_review',
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: true,
|
||||
tableName: 'pr_merge',
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Loads configuration from a JSON file
|
||||
* @param {string} configPath - Path to the configuration file
|
||||
* @returns {Object} Configuration object
|
||||
*/
|
||||
const loadConfigFromFile = (configPath) => {
|
||||
try {
|
||||
const resolvedPath = path.resolve(process.cwd(), configPath);
|
||||
logger.info(`Loading configuration from ${resolvedPath}`);
|
||||
|
||||
if (!fs.existsSync(resolvedPath)) {
|
||||
logger.warn(`Configuration file not found at ${resolvedPath}`);
|
||||
return {};
|
||||
}
|
||||
|
||||
const configData = fs.readFileSync(resolvedPath, 'utf8');
|
||||
return JSON.parse(configData);
|
||||
} catch (err) {
|
||||
logger.error(
|
||||
`Error loading configuration from file: ${configPath}`,
|
||||
{},
|
||||
err
|
||||
);
|
||||
return {};
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Loads configuration from environment variables
|
||||
* @returns {Object} Configuration object
|
||||
*/
|
||||
const loadConfigFromEnv = () => {
|
||||
// Create a config object with only defined values
|
||||
const config = {};
|
||||
|
||||
// Parse repositories from environment variable if provided
|
||||
if (process.env.REPOSITORIES) {
|
||||
config.repositories = process.env.REPOSITORIES.split(',').map((repo) =>
|
||||
repo.trim()
|
||||
);
|
||||
}
|
||||
|
||||
// Add other environment variables if they are defined
|
||||
if (process.env.GITHUB_TOKEN) config.githubToken = process.env.GITHUB_TOKEN;
|
||||
if (process.env.BIGQUERY_DATASET_ID)
|
||||
config.bigQueryDatasetId = process.env.BIGQUERY_DATASET_ID;
|
||||
if (process.env.SERVICE_ACCOUNT_KEY_PATH)
|
||||
config.serviceAccountKeyPath = process.env.SERVICE_ACCOUNT_KEY_PATH;
|
||||
if (process.env.TARGET_BRANCH)
|
||||
config.targetBranch = process.env.TARGET_BRANCH;
|
||||
if (process.env.PRINT_ONLY)
|
||||
config.printOnly = process.env.PRINT_ONLY === 'true';
|
||||
if (process.env.USER_GROUP_ENABLED)
|
||||
config.userGroupEnabled = process.env.USER_GROUP_ENABLED === 'true';
|
||||
if (process.env.USER_GROUP_FILEPATH)
|
||||
config.userGroupFilepath = process.env.USER_GROUP_FILEPATH;
|
||||
|
||||
// Handle metrics configuration from environment variables
|
||||
if (process.env.ENABLED_METRICS) {
|
||||
const enabledMetrics = process.env.ENABLED_METRICS.split(
|
||||
','
|
||||
).map((metric) => metric.trim());
|
||||
config.metrics = {
|
||||
timeToFirstReview: {
|
||||
enabled: enabledMetrics.includes('time_to_first_review'),
|
||||
tableName: process.env.TIME_TO_FIRST_REVIEW_TABLE || 'pr_first_review',
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: enabledMetrics.includes('time_to_merge'),
|
||||
tableName: process.env.TIME_TO_MERGE_TABLE || 'pr_merge',
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
return config;
|
||||
};
|
||||
|
||||
/**
|
||||
* Validates the configuration
|
||||
* @param {Object} config - Configuration object
|
||||
* @returns {boolean} True if configuration is valid, false otherwise
|
||||
*/
|
||||
const validateConfig = (config) => {
|
||||
// Always required fields
|
||||
const requiredFields = ['repositories', 'githubToken'];
|
||||
|
||||
// Fields required only when not in print-only mode
|
||||
if (!config.printOnly) {
|
||||
requiredFields.push('serviceAccountKeyPath');
|
||||
}
|
||||
|
||||
const missingFields = requiredFields.filter((field) => !config[field]);
|
||||
|
||||
if (missingFields.length > 0) {
|
||||
logger.error(
|
||||
`Missing required configuration fields: ${missingFields.join(', ')}`
|
||||
);
|
||||
return false;
|
||||
}
|
||||
|
||||
// Validate repositories array
|
||||
if (!Array.isArray(config.repositories) || config.repositories.length === 0) {
|
||||
logger.error('Configuration must include at least one repository');
|
||||
return false;
|
||||
}
|
||||
|
||||
// Validate repository format (owner/repo)
|
||||
const invalidRepos = config.repositories.filter((repo) => {
|
||||
return typeof repo !== 'string' || !repo.includes('/');
|
||||
});
|
||||
|
||||
if (invalidRepos.length > 0) {
|
||||
logger.error(`Invalid repository format: ${invalidRepos.join(', ')}`);
|
||||
return false;
|
||||
}
|
||||
|
||||
// Validate metrics configuration
|
||||
if (!config.metrics || typeof config.metrics !== 'object') {
|
||||
logger.error('Configuration must include metrics configuration');
|
||||
return false;
|
||||
}
|
||||
|
||||
// Validate that at least one metric is enabled
|
||||
const enabledMetrics = Object.values(config.metrics).filter(
|
||||
(metric) => metric.enabled
|
||||
);
|
||||
if (enabledMetrics.length === 0) {
|
||||
logger.error('At least one metric must be enabled');
|
||||
return false;
|
||||
}
|
||||
|
||||
// Validate metric configurations
|
||||
for (const [metricName, metricConfig] of Object.entries(config.metrics)) {
|
||||
if (metricConfig.enabled) {
|
||||
if (
|
||||
!metricConfig.tableName ||
|
||||
typeof metricConfig.tableName !== 'string'
|
||||
) {
|
||||
logger.error(`Metric ${metricName} must have a valid tableName`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Validate userGroupFilepath when userGroupEnabled is true
|
||||
if (config.userGroupEnabled) {
|
||||
if (!config.userGroupFilepath) {
|
||||
logger.error(
|
||||
'userGroupFilepath must be specified when userGroupEnabled is true'
|
||||
);
|
||||
return false;
|
||||
}
|
||||
|
||||
const resolvedUserGroupPath = path.resolve(
|
||||
process.cwd(),
|
||||
config.userGroupFilepath
|
||||
);
|
||||
if (!fs.existsSync(resolvedUserGroupPath)) {
|
||||
logger.error(`User group file not found at ${resolvedUserGroupPath}`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
};
|
||||
|
||||
/**
|
||||
* Loads and validates configuration
|
||||
* @param {string} [configPath='config.json'] - Path to the configuration file
|
||||
* @returns {Object} Configuration object
|
||||
*/
|
||||
export const loadConfig = (configPath = 'config.json') => {
|
||||
// Load configuration from file
|
||||
const fileConfig = loadConfigFromFile(configPath);
|
||||
|
||||
// Load configuration from environment variables
|
||||
const envConfig = loadConfigFromEnv();
|
||||
|
||||
// Merge configurations with precedence: env > file > default
|
||||
const config = {
|
||||
...DEFAULT_CONFIG,
|
||||
...fileConfig,
|
||||
...envConfig,
|
||||
};
|
||||
|
||||
// Filter out undefined values
|
||||
Object.keys(config).forEach((key) => {
|
||||
if (config[key] === undefined) {
|
||||
delete config[key];
|
||||
}
|
||||
});
|
||||
|
||||
// Validate configuration
|
||||
const isValid = validateConfig(config);
|
||||
|
||||
if (!isValid) {
|
||||
throw new Error('Invalid configuration');
|
||||
}
|
||||
|
||||
logger.info('Configuration loaded successfully', {
|
||||
repositories: config.repositories,
|
||||
targetBranch: config.targetBranch,
|
||||
printOnly: config.printOnly,
|
||||
metrics: Object.fromEntries(
|
||||
Object.entries(config.metrics).map(([key, value]) => [
|
||||
key,
|
||||
{ enabled: value.enabled, tableName: value.tableName },
|
||||
])
|
||||
),
|
||||
...(config.printOnly
|
||||
? {}
|
||||
: {
|
||||
bigQueryDatasetId: config.bigQueryDatasetId,
|
||||
}),
|
||||
});
|
||||
|
||||
return config;
|
||||
};
|
||||
|
||||
export { validateConfig };
|
||||
|
||||
export default {
|
||||
loadConfig,
|
||||
};
|
||||
655
.github/actions/eng-metrics/src/github-client.js
vendored
Normal file
655
.github/actions/eng-metrics/src/github-client.js
vendored
Normal file
|
|
@ -0,0 +1,655 @@
|
|||
/**
|
||||
* GitHub client module for engineering metrics collector
|
||||
* Handles interactions with the GitHub API using Octokit.js
|
||||
*/
|
||||
|
||||
import { Octokit } from 'octokit';
|
||||
import logger from './logger.js';
|
||||
|
||||
/**
|
||||
* Identifies if a GitHub user is likely a bot
|
||||
* @param {Object} user - GitHub user object
|
||||
* @returns {Object} Bot analysis result
|
||||
*/
|
||||
function identifyBotUser(user) {
|
||||
const botIndicators = {
|
||||
isBot: false,
|
||||
confidence: 'low',
|
||||
reasons: [],
|
||||
};
|
||||
|
||||
// Check GitHub's bot flag (most reliable)
|
||||
if (user.type === 'Bot') {
|
||||
botIndicators.isBot = true;
|
||||
botIndicators.confidence = 'high';
|
||||
botIndicators.reasons.push('GitHub API type is "Bot"');
|
||||
return botIndicators;
|
||||
}
|
||||
|
||||
// Check username patterns
|
||||
const username = user.login.toLowerCase();
|
||||
const botPatterns = [
|
||||
/\[bot]/, // contains '[bot]'
|
||||
/^dependabot/, // dependabot
|
||||
/^renovate/, // renovate bot
|
||||
/^github-actions/, // GitHub Actions
|
||||
/^codecov/, // codecov bot
|
||||
/^coderabbitai/, // coderabbit AI bot
|
||||
/^sonarcloud/, // sonarcloud bot
|
||||
/^snyk/, // snyk bot
|
||||
/^greenkeeper/, // greenkeeper bot
|
||||
/^semantic-release/, // semantic-release bot
|
||||
/^stale/, // stale bot
|
||||
/^imgbot/, // imgbot
|
||||
/^allcontributors/, // all-contributors bot
|
||||
/^whitesource/, // whitesource bot
|
||||
/^deepsource/, // deepsource bot
|
||||
];
|
||||
|
||||
for (const pattern of botPatterns) {
|
||||
if (pattern.test(username)) {
|
||||
botIndicators.isBot = true;
|
||||
botIndicators.confidence = 'high';
|
||||
botIndicators.reasons.push(`Username matches bot pattern: ${pattern}`);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
return botIndicators;
|
||||
}
|
||||
|
||||
/**
|
||||
* GitHub client class
|
||||
*/
|
||||
export class GitHubClient {
|
||||
/**
|
||||
* Creates a new GitHub client
|
||||
* @param {string} token - GitHub API token
|
||||
*/
|
||||
constructor(token) {
|
||||
if (!token) {
|
||||
throw new Error('GitHub token is required');
|
||||
}
|
||||
this.octokit = null;
|
||||
this.initialize(token);
|
||||
}
|
||||
|
||||
/**
|
||||
* Initializes the GitHub client
|
||||
* @param {string} token - GitHub API token
|
||||
*/
|
||||
initialize(token) {
|
||||
try {
|
||||
this.octokit = new Octokit({
|
||||
auth: token,
|
||||
});
|
||||
logger.info('GitHub client initialized');
|
||||
} catch (err) {
|
||||
logger.error('Failed to initialize GitHub client', {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches pull requests for a repository
|
||||
* @param {string} owner - Repository owner
|
||||
* @param {string} repo - Repository name
|
||||
* @param {string} state - PR state (open, closed, all)
|
||||
* @param {Date} since - Fetch PRs updated since this date
|
||||
* @param {string} targetBranch - Target branch to filter PRs by
|
||||
* @returns {Array} Array of pull requests
|
||||
*/
|
||||
async fetchPullRequests(
|
||||
owner,
|
||||
repo,
|
||||
state = 'all',
|
||||
since,
|
||||
targetBranch = 'main'
|
||||
) {
|
||||
try {
|
||||
logger.info(
|
||||
`Fetching ${state} PRs for ${owner}/${repo} since ${since.toISOString()}`
|
||||
);
|
||||
|
||||
// GitHub API returns paginated results, so we need to fetch all pages
|
||||
const pullRequests = [];
|
||||
let page = 1;
|
||||
let hasMorePages = true;
|
||||
|
||||
while (hasMorePages) {
|
||||
const response = await this.octokit.rest.pulls.list({
|
||||
owner,
|
||||
repo,
|
||||
state,
|
||||
sort: 'updated',
|
||||
direction: 'desc',
|
||||
per_page: 100,
|
||||
page,
|
||||
});
|
||||
|
||||
// Filter PRs by update date and target branch
|
||||
const filteredPRs = response.data.filter((pr) => {
|
||||
const prUpdatedAt = new Date(pr.updated_at);
|
||||
return prUpdatedAt >= since && pr.base.ref === targetBranch;
|
||||
});
|
||||
|
||||
if (filteredPRs.length > 0) {
|
||||
pullRequests.push(...filteredPRs);
|
||||
page++;
|
||||
} else {
|
||||
hasMorePages = false;
|
||||
}
|
||||
|
||||
// If we got fewer results than the page size, there are no more pages
|
||||
if (response.data.length < 100) {
|
||||
hasMorePages = false;
|
||||
}
|
||||
}
|
||||
|
||||
logger.info(`Fetched ${pullRequests.length} PRs for ${owner}/${repo}`);
|
||||
return pullRequests;
|
||||
} catch (err) {
|
||||
logger.error(`Error fetching PRs for ${owner}/${repo}`, {}, err);
|
||||
|
||||
// Implement basic retry for rate limiting
|
||||
if (
|
||||
err.status === 403 &&
|
||||
err.response?.headers?.['x-ratelimit-remaining'] === '0'
|
||||
) {
|
||||
const resetTime =
|
||||
parseInt(err.response.headers['x-ratelimit-reset'], 10) * 1000;
|
||||
const waitTime = resetTime - Date.now();
|
||||
|
||||
if (waitTime > 0 && waitTime < 3600000) {
|
||||
// Only retry if wait time is less than 1 hour
|
||||
logger.info(
|
||||
`Rate limit exceeded. Retrying in ${Math.ceil(
|
||||
waitTime / 1000
|
||||
)} seconds`
|
||||
);
|
||||
await new Promise((resolve) => setTimeout(resolve, waitTime + 1000));
|
||||
return this.fetchPullRequests(
|
||||
owner,
|
||||
repo,
|
||||
state,
|
||||
since,
|
||||
targetBranch
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches PR review events
|
||||
* @param {string} owner - Repository owner
|
||||
* @param {string} repo - Repository name
|
||||
* @param {number} prNumber - PR number
|
||||
* @returns {Array} Array of review events
|
||||
*/
|
||||
async fetchPRReviewEvents(owner, repo, prNumber) {
|
||||
try {
|
||||
logger.info(`Fetching review events for ${owner}/${repo}#${prNumber}`);
|
||||
|
||||
const response = await this.octokit.rest.pulls.listReviews({
|
||||
owner,
|
||||
repo,
|
||||
pull_number: prNumber,
|
||||
});
|
||||
|
||||
logger.info(
|
||||
`Fetched ${response.data.length} review events for ${owner}/${repo}#${prNumber}`
|
||||
);
|
||||
return response.data;
|
||||
} catch (err) {
|
||||
logger.error(
|
||||
`Error fetching review events for ${owner}/${repo}#${prNumber}`,
|
||||
{},
|
||||
err
|
||||
);
|
||||
|
||||
// Implement basic retry for rate limiting
|
||||
if (
|
||||
err.status === 403 &&
|
||||
err.response?.headers?.['x-ratelimit-remaining'] === '0'
|
||||
) {
|
||||
const resetTime =
|
||||
parseInt(err.response.headers['x-ratelimit-reset'], 10) * 1000;
|
||||
const waitTime = resetTime - Date.now();
|
||||
|
||||
if (waitTime > 0 && waitTime < 3600000) {
|
||||
// Only retry if wait time is less than 1 hour
|
||||
logger.info(
|
||||
`Rate limit exceeded. Retrying in ${Math.ceil(
|
||||
waitTime / 1000
|
||||
)} seconds`
|
||||
);
|
||||
await new Promise((resolve) => setTimeout(resolve, waitTime + 1000));
|
||||
return this.fetchPRReviewEvents(owner, repo, prNumber);
|
||||
}
|
||||
}
|
||||
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Filters out bot reviews from review events
|
||||
* @param {Array} reviewEvents - Array of review events
|
||||
* @param {boolean} excludeBots - Whether to exclude bot reviews (default: false)
|
||||
* @returns {Array} Filtered review events
|
||||
*/
|
||||
filterBotReviews(reviewEvents, excludeBots = false) {
|
||||
if (!excludeBots) {
|
||||
return reviewEvents;
|
||||
}
|
||||
|
||||
const filteredReviews = reviewEvents.filter((review) => {
|
||||
const botAnalysis = identifyBotUser(review.user);
|
||||
if (botAnalysis.isBot) {
|
||||
logger.debug(`Filtering out bot review from ${review.user.login}`, {
|
||||
confidence: botAnalysis.confidence,
|
||||
reasons: botAnalysis.reasons,
|
||||
});
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
|
||||
const botCount = reviewEvents.length - filteredReviews.length;
|
||||
if (botCount > 0) {
|
||||
logger.info(
|
||||
`Filtered out ${botCount} bot reviews from ${reviewEvents.length} total reviews`
|
||||
);
|
||||
}
|
||||
|
||||
return filteredReviews;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetches PR timeline events
|
||||
* @param {string} owner - Repository owner
|
||||
* @param {string} repo - Repository name
|
||||
* @param {number} prNumber - PR number
|
||||
* @returns {Array} Array of timeline events
|
||||
*/
|
||||
async fetchPRTimelineEvents(owner, repo, prNumber) {
|
||||
try {
|
||||
logger.info(`Fetching timeline events for ${owner}/${repo}#${prNumber}`);
|
||||
|
||||
// GitHub API returns paginated results, so we need to fetch all pages
|
||||
const timelineEvents = [];
|
||||
let page = 1;
|
||||
let hasMorePages = true;
|
||||
|
||||
while (hasMorePages) {
|
||||
const response = await this.octokit.rest.issues.listEventsForTimeline({
|
||||
owner,
|
||||
repo,
|
||||
issue_number: prNumber,
|
||||
per_page: 100,
|
||||
page,
|
||||
});
|
||||
|
||||
if (response.data.length > 0) {
|
||||
timelineEvents.push(...response.data);
|
||||
page++;
|
||||
} else {
|
||||
hasMorePages = false;
|
||||
}
|
||||
|
||||
// If we got fewer results than the page size, there are no more pages
|
||||
if (response.data.length < 100) {
|
||||
hasMorePages = false;
|
||||
}
|
||||
}
|
||||
|
||||
logger.info(
|
||||
`Fetched ${timelineEvents.length} timeline events for ${owner}/${repo}#${prNumber}`
|
||||
);
|
||||
return timelineEvents;
|
||||
} catch (err) {
|
||||
logger.error(
|
||||
`Error fetching timeline events for ${owner}/${repo}#${prNumber}`,
|
||||
{},
|
||||
err
|
||||
);
|
||||
|
||||
// Implement basic retry for rate limiting
|
||||
if (
|
||||
err.status === 403 &&
|
||||
err.response?.headers?.['x-ratelimit-remaining'] === '0'
|
||||
) {
|
||||
const resetTime =
|
||||
parseInt(err.response.headers['x-ratelimit-reset'], 10) * 1000;
|
||||
const waitTime = resetTime - Date.now();
|
||||
|
||||
if (waitTime > 0 && waitTime < 3600000) {
|
||||
// Only retry if wait time is less than 1 hour
|
||||
logger.info(
|
||||
`Rate limit exceeded. Retrying in ${Math.ceil(
|
||||
waitTime / 1000
|
||||
)} seconds`
|
||||
);
|
||||
await new Promise((resolve) => setTimeout(resolve, waitTime + 1000));
|
||||
return this.fetchPRTimelineEvents(owner, repo, prNumber);
|
||||
}
|
||||
}
|
||||
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculates pickup time for a PR
|
||||
* @param {Object} pr - Pull request object
|
||||
* @param {Array} timelineEvents - PR timeline events
|
||||
* @param {Array} reviewEvents - PR review events
|
||||
* @returns {Object} Pickup time metrics
|
||||
*/
|
||||
calculatePickupTime(pr, timelineEvents, reviewEvents) {
|
||||
try {
|
||||
const result = this.getReadyAndFirstReview(
|
||||
pr,
|
||||
timelineEvents,
|
||||
reviewEvents
|
||||
);
|
||||
if (!result || !result.firstReviewTime) {
|
||||
return null;
|
||||
}
|
||||
const { relevantReadyEvent, firstReviewTime } = result;
|
||||
const readyTime = relevantReadyEvent.time;
|
||||
|
||||
// Calculate pickup time excluding weekends
|
||||
const pickupTimeSeconds = this.calculatePickupTimeExcludingWeekends(
|
||||
readyTime,
|
||||
firstReviewTime
|
||||
);
|
||||
|
||||
// If pickup time is negative, something went wrong
|
||||
if (pickupTimeSeconds < 0) {
|
||||
logger.warn(`Negative pickup time for ${pr.html_url}`, {
|
||||
readyTime,
|
||||
firstReviewTime,
|
||||
pickupTimeSeconds,
|
||||
});
|
||||
return null;
|
||||
}
|
||||
|
||||
// Log which ready event was used
|
||||
const readyEventType =
|
||||
relevantReadyEvent.event.event === 'created_not_draft'
|
||||
? 'PR creation (not draft)'
|
||||
: 'ready_for_review event';
|
||||
|
||||
logger.info(`Calculated pickup time for ${pr.html_url}`, {
|
||||
pickupTimeSeconds,
|
||||
readyEventType,
|
||||
readyTime: readyTime.toISOString(),
|
||||
firstReviewTime: firstReviewTime.toISOString(),
|
||||
});
|
||||
|
||||
// We already have readyEventType defined above, so we can use it here
|
||||
|
||||
return {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: `${pr.base.repo.owner.login}/${pr.base.repo.name}`,
|
||||
prNumber: pr.number,
|
||||
prUrl: pr.html_url,
|
||||
prCreator: pr.user.login,
|
||||
targetBranch: pr.base.ref,
|
||||
readyTime,
|
||||
firstReviewTime,
|
||||
reviewDate: firstReviewTime.toISOString().split('T')[0], // YYYY-MM-DD
|
||||
pickupTimeSeconds,
|
||||
readyEventType,
|
||||
};
|
||||
} catch (err) {
|
||||
logger.error(`Error calculating pickup time for ${pr.html_url}`, {}, err);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculates pickup time for a PR
|
||||
* @param {Object} pr - Pull request object
|
||||
* @param {Array} timelineEvents - PR timeline events
|
||||
* @param {Array} reviewEvents - PR review events
|
||||
* @returns {Object} ready event and first review time
|
||||
*/
|
||||
getReadyAndFirstReview(pr, timelineEvents, reviewEvents) {
|
||||
const mergeTime = pr.merged_at ? new Date(pr.merged_at) : null;
|
||||
|
||||
// Find all ready_for_review events that occurred before merge time (if merged)
|
||||
const readyForReviewEvents = timelineEvents
|
||||
.filter((event) => event.event === 'ready_for_review')
|
||||
.map((event) => ({
|
||||
time: new Date(event.created_at),
|
||||
event,
|
||||
}))
|
||||
.filter((readyEvent) => !mergeTime || readyEvent.time <= mergeTime);
|
||||
|
||||
// Add PR creation time as a ready event if PR was not created as draft
|
||||
if (!pr.draft) {
|
||||
readyForReviewEvents.push({
|
||||
time: new Date(pr.created_at),
|
||||
event: { event: 'created_not_draft', created_at: pr.created_at },
|
||||
});
|
||||
}
|
||||
|
||||
// Sort ready events by time (ascending)
|
||||
readyForReviewEvents.sort((a, b) => a.time - b.time);
|
||||
|
||||
// If we couldn't find any ready events, return null
|
||||
if (readyForReviewEvents.length === 0) {
|
||||
logger.warn(`No ready_for_review events found for ${pr.html_url}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
// If there is no review events, the PR may have been merged without a review.
|
||||
if (reviewEvents.length === 0) {
|
||||
const relevantReadyEvent =
|
||||
readyForReviewEvents[readyForReviewEvents.length - 1];
|
||||
return {
|
||||
relevantReadyEvent,
|
||||
firstReviewTime: null,
|
||||
};
|
||||
}
|
||||
|
||||
// Sort review events by submitted_at (ascending)
|
||||
const sortedReviewEvents = [...reviewEvents].sort(
|
||||
(a, b) => new Date(a.submitted_at) - new Date(b.submitted_at)
|
||||
);
|
||||
|
||||
const firstReview = sortedReviewEvents[0];
|
||||
const firstReviewTime = new Date(firstReview.submitted_at);
|
||||
|
||||
// Find the most recent ready event that occurred before the first review
|
||||
const relevantReadyEvent = readyForReviewEvents
|
||||
.filter((readyEvent) => readyEvent.time < firstReviewTime)
|
||||
.pop();
|
||||
|
||||
// If no ready event occurred before the first review, return null
|
||||
if (!relevantReadyEvent) {
|
||||
logger.warn(
|
||||
`No ready_for_review event found before first review for ${pr.html_url}`
|
||||
);
|
||||
return null;
|
||||
}
|
||||
|
||||
return {
|
||||
relevantReadyEvent,
|
||||
firstReviewTime,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculates pickup time excluding weekends
|
||||
* @param {Date} readyTimeOrig - Time when PR was marked as ready for review
|
||||
* @param {Date} reviewTimeOrig - Time when the first review occurred
|
||||
* @returns {number} Pickup time in seconds, excluding weekends
|
||||
*/
|
||||
calculatePickupTimeExcludingWeekends(readyTimeOrig, reviewTimeOrig) {
|
||||
const readyTime = new Date(readyTimeOrig);
|
||||
const reviewTime = new Date(reviewTimeOrig);
|
||||
|
||||
// Get day of week (0 = Sunday, 1 = Monday, ..., 6 = Saturday)
|
||||
const readyDay = readyTime.getUTCDay();
|
||||
const reviewDay = reviewTime.getUTCDay();
|
||||
|
||||
// Case: Both ready time and review time are on the same weekend
|
||||
if (
|
||||
(readyDay === 0 || readyDay === 6) &&
|
||||
(reviewDay === 0 || reviewDay === 6) &&
|
||||
Math.floor(reviewTime / (24 * 60 * 60 * 1000)) -
|
||||
Math.floor(readyTime / (24 * 60 * 60 * 1000)) <=
|
||||
2
|
||||
) {
|
||||
// Return 0 seconds pickup time
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Set to start of Monday if ready time is on weekend
|
||||
if (readyDay === 0) {
|
||||
// Sunday
|
||||
readyTime.setUTCDate(readyTime.getUTCDate() + 1);
|
||||
readyTime.setUTCHours(0, 0, 0, 0);
|
||||
} else if (readyDay === 6) {
|
||||
// Saturday
|
||||
readyTime.setUTCDate(readyTime.getUTCDate() + 2);
|
||||
readyTime.setUTCHours(0, 0, 0, 0);
|
||||
}
|
||||
// Set to start of Saturday if review time is on Sunday
|
||||
if (reviewDay === 0) {
|
||||
// Sunday
|
||||
reviewTime.setUTCDate(reviewTime.getUTCDate() - 1);
|
||||
reviewTime.setUTCHours(0, 0, 0, 0);
|
||||
} else if (reviewDay === 6) {
|
||||
// Saturday
|
||||
reviewTime.setUTCHours(0, 0, 0, 0);
|
||||
}
|
||||
|
||||
// Calculate raw time difference in milliseconds
|
||||
const weekendDays = countWeekendDays(readyTime, reviewTime);
|
||||
const diffMs = reviewTime - readyTime - weekendDays * 24 * 60 * 60 * 1000;
|
||||
|
||||
// Ensure we don't return negative values
|
||||
return Math.max(0, Math.floor(diffMs / 1000));
|
||||
}
|
||||
|
||||
/**
|
||||
* Calculate time to merge metrics
|
||||
* @param {Object} pr - Pull request object
|
||||
* @param {Array} timelineEvents - Timeline events
|
||||
* @param {Array} reviewEvents - PR review events
|
||||
* @returns {Object|null} Time to merge metrics or null if not applicable
|
||||
*/
|
||||
calculateTimeToMerge(pr, timelineEvents, reviewEvents) {
|
||||
try {
|
||||
// Only process merged PRs
|
||||
if (!pr.merged_at) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Find the ready time using the same algorithm as we use for Time to First Review
|
||||
const result = this.getReadyAndFirstReview(
|
||||
pr,
|
||||
timelineEvents,
|
||||
reviewEvents
|
||||
);
|
||||
if (!result) {
|
||||
return null;
|
||||
}
|
||||
const relevantReadyEvent = result.relevantReadyEvent;
|
||||
const readyTime = relevantReadyEvent.time;
|
||||
const mergeTime = new Date(pr.merged_at);
|
||||
|
||||
// Calculate merge time excluding weekends
|
||||
const mergeTimeSeconds = this.calculatePickupTimeExcludingWeekends(
|
||||
readyTime,
|
||||
mergeTime
|
||||
);
|
||||
|
||||
// If merge time is negative, something went wrong
|
||||
if (mergeTimeSeconds < 0) {
|
||||
logger.warn(`Negative merge time for ${pr.html_url}`, {
|
||||
readyTime,
|
||||
mergeTime,
|
||||
mergeTimeSeconds,
|
||||
});
|
||||
return null;
|
||||
}
|
||||
|
||||
// Log which ready event was used
|
||||
const readyEventType =
|
||||
relevantReadyEvent.event.event === 'created_not_draft'
|
||||
? 'PR creation (not draft)'
|
||||
: 'ready_for_review event';
|
||||
|
||||
logger.info(`Calculated merge time for ${pr.html_url}`, {
|
||||
mergeTimeSeconds,
|
||||
readyEventType,
|
||||
readyTime: readyTime.toISOString(),
|
||||
mergeTime: mergeTime.toISOString(),
|
||||
});
|
||||
|
||||
return {
|
||||
metricType: 'time_to_merge',
|
||||
repository: `${pr.base.repo.owner.login}/${pr.base.repo.name}`,
|
||||
prNumber: pr.number,
|
||||
prUrl: pr.html_url,
|
||||
prCreator: pr.user.login,
|
||||
targetBranch: pr.base.ref,
|
||||
readyTime,
|
||||
mergeTime,
|
||||
mergeDate: mergeTime.toISOString().split('T')[0], // YYYY-MM-DD
|
||||
mergeTimeSeconds,
|
||||
readyEventType,
|
||||
};
|
||||
} catch (err) {
|
||||
logger.error(`Error calculating merge time for ${pr.html_url}`, {}, err);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function countWeekendDays(startDate, endDate) {
|
||||
// Make local copies of dates
|
||||
startDate = new Date(startDate);
|
||||
endDate = new Date(endDate);
|
||||
|
||||
// Ensure startDate is before endDate
|
||||
if (startDate > endDate) {
|
||||
[startDate, endDate] = [endDate, startDate];
|
||||
}
|
||||
|
||||
// Make sure start dates and end dates are not on weekends. We just want to count the weekend days between them.
|
||||
if (startDate.getUTCDay() === 0) {
|
||||
startDate.setUTCDate(startDate.getUTCDate() + 1);
|
||||
} else if (startDate.getUTCDay() === 6) {
|
||||
startDate.setUTCDate(startDate.getUTCDate() + 2);
|
||||
}
|
||||
if (endDate.getUTCDay() === 0) {
|
||||
endDate.setUTCDate(endDate.getUTCDate() - 2);
|
||||
} else if (endDate.getUTCDay() === 6) {
|
||||
endDate.setUTCDate(endDate.getUTCDate() - 1);
|
||||
}
|
||||
|
||||
let count = 0;
|
||||
const current = new Date(startDate);
|
||||
|
||||
while (current <= endDate) {
|
||||
const day = current.getUTCDay();
|
||||
if (day === 0 || day === 6) {
|
||||
// Sunday (0) or Saturday (6)
|
||||
count++;
|
||||
}
|
||||
current.setUTCDate(current.getUTCDate() + 1);
|
||||
}
|
||||
|
||||
return count;
|
||||
}
|
||||
|
||||
export default GitHubClient;
|
||||
109
.github/actions/eng-metrics/src/github-validator.js
vendored
Normal file
109
.github/actions/eng-metrics/src/github-validator.js
vendored
Normal file
|
|
@ -0,0 +1,109 @@
|
|||
/**
|
||||
* GitHub username validator
|
||||
* Validates that extracted usernames are real GitHub accounts
|
||||
*/
|
||||
|
||||
import { Octokit } from 'octokit';
|
||||
import logger from './logger.js';
|
||||
|
||||
/**
|
||||
* Validates a single GitHub username
|
||||
* @param {Octokit} octokit - GitHub API client
|
||||
* @param {string} username - GitHub username to validate
|
||||
* @returns {Promise<boolean>} True if username exists, false otherwise
|
||||
*/
|
||||
const validateUsername = async (octokit, username) => {
|
||||
try {
|
||||
await octokit.rest.users.getByUsername({ username });
|
||||
return true;
|
||||
} catch (error) {
|
||||
if (error.status === 404) {
|
||||
logger.warn(`GitHub username not found: ${username}`);
|
||||
return false;
|
||||
}
|
||||
|
||||
// For other errors (rate limiting, network issues), log but assume valid
|
||||
logger.warn(`Error validating username ${username}: ${error.message}`);
|
||||
return true; // Assume valid to avoid false negatives
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Validates multiple GitHub usernames
|
||||
* @param {string} githubToken - GitHub API token
|
||||
* @param {Array<string>} usernames - Array of usernames to validate
|
||||
* @returns {Promise<Array<string>>} Array of valid usernames
|
||||
*/
|
||||
export const validateUsernames = async (githubToken, usernames) => {
|
||||
if (!githubToken) {
|
||||
throw new Error('GitHub token is required for username validation');
|
||||
}
|
||||
|
||||
const octokit = new Octokit({ auth: githubToken });
|
||||
const validUsernames = [];
|
||||
const invalidUsernames = [];
|
||||
|
||||
logger.info(`Validating ${usernames.length} GitHub usernames...`);
|
||||
|
||||
// Process usernames with a small delay to respect rate limits
|
||||
for (const username of usernames) {
|
||||
const isValid = await validateUsername(octokit, username);
|
||||
|
||||
if (isValid) {
|
||||
validUsernames.push(username);
|
||||
} else {
|
||||
invalidUsernames.push(username);
|
||||
}
|
||||
|
||||
// Small delay to avoid hitting rate limits too aggressively.
|
||||
// 2025/07/03: GitHub's authenticated rate limit is 5000 requests/hour (~1.4 requests/second). This could lead to rate limit errors with larger username lists.
|
||||
await new Promise((resolve) => setTimeout(resolve, 100));
|
||||
}
|
||||
|
||||
if (invalidUsernames.length > 0) {
|
||||
logger.warn(
|
||||
`Found ${
|
||||
invalidUsernames.length
|
||||
} invalid GitHub usernames: ${invalidUsernames.join(', ')}`
|
||||
);
|
||||
}
|
||||
|
||||
logger.info(
|
||||
`Validated ${validUsernames.length} out of ${usernames.length} GitHub usernames`
|
||||
);
|
||||
return validUsernames;
|
||||
};
|
||||
|
||||
/**
|
||||
* Filters user groups to only include valid usernames
|
||||
* @param {string} githubToken - GitHub API token
|
||||
* @param {Array<{group: string, username: string}>} userGroups - Array of user group mappings
|
||||
* @returns {Promise<Array<{group: string, username: string}>>} Array of user group mappings with valid usernames only
|
||||
*/
|
||||
export const filterValidUserGroups = async (githubToken, userGroups) => {
|
||||
// Get unique usernames for validation
|
||||
const uniqueUsernames = [...new Set(userGroups.map((ug) => ug.username))];
|
||||
|
||||
// Validate usernames
|
||||
const validUsernames = await validateUsernames(githubToken, uniqueUsernames);
|
||||
const validUsernameSet = new Set(validUsernames);
|
||||
|
||||
// Filter user groups to only include valid usernames
|
||||
const validUserGroups = userGroups.filter((ug) =>
|
||||
validUsernameSet.has(ug.username)
|
||||
);
|
||||
|
||||
const removedCount = userGroups.length - validUserGroups.length;
|
||||
if (removedCount > 0) {
|
||||
logger.info(
|
||||
`Removed ${removedCount} user group mappings due to invalid usernames`
|
||||
);
|
||||
}
|
||||
|
||||
return validUserGroups;
|
||||
};
|
||||
|
||||
export default {
|
||||
validateUsernames,
|
||||
filterValidUserGroups,
|
||||
};
|
||||
74
.github/actions/eng-metrics/src/index.js
vendored
Executable file
74
.github/actions/eng-metrics/src/index.js
vendored
Executable file
|
|
@ -0,0 +1,74 @@
|
|||
#!/usr/bin/env node
|
||||
|
||||
/**
|
||||
* Main entry point for engineering metrics collector
|
||||
* Collects comprehensive GitHub engineering metrics including:
|
||||
* - Time to First Review (currently implemented)
|
||||
* - Time to Merge (planned)
|
||||
* - Time to QA Ready (planned)
|
||||
* - Time to Production Ready (planned)
|
||||
*/
|
||||
|
||||
import { loadConfig } from './config.js';
|
||||
import { MetricsCollector } from './metrics-collector.js';
|
||||
import logger from './logger.js';
|
||||
import { fileURLToPath } from 'url';
|
||||
import { dirname, join, resolve } from 'path';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = dirname(__filename);
|
||||
|
||||
/**
|
||||
* Main function
|
||||
*/
|
||||
async function main() {
|
||||
try {
|
||||
// Set the working directory to one level up from the current file location, which is the root of the action.
|
||||
process.chdir(resolve(__dirname, '..'));
|
||||
|
||||
// Parse command line arguments
|
||||
const args = process.argv.slice(2);
|
||||
|
||||
// Check for print-only flag in command line arguments
|
||||
const printOnlyFlag = args.includes('--print-only');
|
||||
|
||||
// Get the configuration path from command line arguments
|
||||
// Filter out the --print-only flag if present
|
||||
const configPath =
|
||||
args.filter((arg) => arg !== '--print-only')[0] ||
|
||||
join(__dirname, '..', 'config.json');
|
||||
|
||||
// Load configuration
|
||||
const config = loadConfig(configPath);
|
||||
|
||||
// Override printOnly setting if flag is provided
|
||||
if (printOnlyFlag) {
|
||||
config.printOnly = true;
|
||||
}
|
||||
|
||||
// Create and run metrics collector
|
||||
const metricsCollector = new MetricsCollector(config);
|
||||
const metrics = await metricsCollector.run();
|
||||
|
||||
if (config.printOnly) {
|
||||
logger.info(
|
||||
`Successfully collected and printed ${metrics.length} engineering metrics`
|
||||
);
|
||||
} else {
|
||||
logger.info(
|
||||
`Successfully collected and uploaded ${metrics.length} engineering metrics to BigQuery`
|
||||
);
|
||||
}
|
||||
|
||||
// Exit with success
|
||||
process.exit(0);
|
||||
} catch (err) {
|
||||
logger.error('Error running engineering metrics collector', {}, err);
|
||||
|
||||
// Exit with error
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Run the main function
|
||||
await main();
|
||||
112
.github/actions/eng-metrics/src/logger.js
vendored
Normal file
112
.github/actions/eng-metrics/src/logger.js
vendored
Normal file
|
|
@ -0,0 +1,112 @@
|
|||
/**
|
||||
* Logger module for engineering metrics collector
|
||||
* Provides structured JSON logging
|
||||
*/
|
||||
|
||||
/**
|
||||
* Log levels
|
||||
*/
|
||||
const LOG_LEVELS = {
|
||||
DEBUG: 'debug',
|
||||
INFO: 'info',
|
||||
WARN: 'warn',
|
||||
ERROR: 'error',
|
||||
};
|
||||
|
||||
/**
|
||||
* Creates a log entry with the specified level, message, and optional data
|
||||
* @param {string} level - Log level
|
||||
* @param {string} message - Log message
|
||||
* @param {Object} [data] - Optional data to include in the log
|
||||
* @param {Error} [error] - Optional error object
|
||||
* @returns {Object} Log entry object
|
||||
*/
|
||||
const createLogEntry = (level, message, data = {}, error = null) => {
|
||||
const logEntry = {
|
||||
timestamp: new Date().toISOString(),
|
||||
level,
|
||||
message,
|
||||
...(Object.keys(data).length > 0 && { data }),
|
||||
};
|
||||
|
||||
if (error) {
|
||||
logEntry.error = {
|
||||
name: error.name,
|
||||
message: error.message,
|
||||
stack: error.stack,
|
||||
};
|
||||
}
|
||||
|
||||
return logEntry;
|
||||
};
|
||||
|
||||
/**
|
||||
* Logs a message at the specified level
|
||||
* @param {string} level - Log level
|
||||
* @param {string} message - Log message
|
||||
* @param {Object} [data] - Optional data to include in the log
|
||||
* @param {Error} [error] - Optional error object
|
||||
*/
|
||||
const log = (level, message, data = {}, error = null) => {
|
||||
const logEntry = createLogEntry(level, message, data, error);
|
||||
try {
|
||||
console.log(JSON.stringify(logEntry));
|
||||
} catch (serializationError) {
|
||||
console.log(
|
||||
JSON.stringify({
|
||||
timestamp: new Date().toISOString(),
|
||||
level: 'error',
|
||||
message: 'Failed to serialize log entry',
|
||||
originalMessage: message,
|
||||
error: {
|
||||
name: serializationError.name,
|
||||
message: serializationError.message,
|
||||
},
|
||||
})
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Logs a debug message
|
||||
* @param {string} message - Log message
|
||||
* @param {Object} [data] - Optional data to include in the log
|
||||
*/
|
||||
export const debug = (message, data = {}) => {
|
||||
log(LOG_LEVELS.DEBUG, message, data);
|
||||
};
|
||||
|
||||
/**
|
||||
* Logs an info message
|
||||
* @param {string} message - Log message
|
||||
* @param {Object} [data] - Optional data to include in the log
|
||||
*/
|
||||
export const info = (message, data = {}) => {
|
||||
log(LOG_LEVELS.INFO, message, data);
|
||||
};
|
||||
|
||||
/**
|
||||
* Logs a warning message
|
||||
* @param {string} message - Log message
|
||||
* @param {Object} [data] - Optional data to include in the log
|
||||
*/
|
||||
export const warn = (message, data = {}) => {
|
||||
log(LOG_LEVELS.WARN, message, data);
|
||||
};
|
||||
|
||||
/**
|
||||
* Logs an error message
|
||||
* @param {string} message - Log message
|
||||
* @param {Object} [data] - Optional data to include in the log
|
||||
* @param {Error} [error] - Optional error object
|
||||
*/
|
||||
export const error = (message, data = {}, error) => {
|
||||
log(LOG_LEVELS.ERROR, message, data, error);
|
||||
};
|
||||
|
||||
export default {
|
||||
debug,
|
||||
info,
|
||||
warn,
|
||||
error,
|
||||
};
|
||||
156
.github/actions/eng-metrics/src/markdown-parser.js
vendored
Normal file
156
.github/actions/eng-metrics/src/markdown-parser.js
vendored
Normal file
|
|
@ -0,0 +1,156 @@
|
|||
/**
|
||||
* Markdown parser for extracting GitHub usernames from product groups
|
||||
* Parses the product-groups.md file to extract developer usernames by group
|
||||
*/
|
||||
|
||||
import fs from 'fs';
|
||||
import path from 'path';
|
||||
import logger from './logger.js';
|
||||
|
||||
/**
|
||||
* Parses the product groups markdown file and extracts GitHub usernames
|
||||
* @param {string} filePath - Path to the product-groups.md file
|
||||
* @returns {Array<{group: string, username: string}>} Array of user group mappings
|
||||
*/
|
||||
export const parseProductGroups = (filePath) => {
|
||||
try {
|
||||
const resolvedPath = path.resolve(process.cwd(), filePath);
|
||||
logger.info(`Parsing product groups from ${resolvedPath}`);
|
||||
|
||||
if (!fs.existsSync(resolvedPath)) {
|
||||
logger.error(`Product groups file not found at ${resolvedPath}`);
|
||||
return [];
|
||||
}
|
||||
|
||||
const content = fs.readFileSync(resolvedPath, 'utf8');
|
||||
return extractUsernamesFromMarkdown(content);
|
||||
} catch (err) {
|
||||
logger.error(`Error parsing product groups file: ${filePath}`, {}, err);
|
||||
return [];
|
||||
}
|
||||
};
|
||||
|
||||
/**
|
||||
* Extracts usernames from markdown content
|
||||
* @param {string} content - Markdown content
|
||||
* @returns {Array<{group: string, username: string}>} Array of user group mappings
|
||||
*/
|
||||
const extractUsernamesFromMarkdown = (content) => {
|
||||
const userGroups = [];
|
||||
|
||||
// Define the groups we're looking for and their corresponding database group names
|
||||
const groupMappings = {
|
||||
'MDM group': 'mdm',
|
||||
'Orchestration group': 'orchestration',
|
||||
'Software group': 'software',
|
||||
};
|
||||
|
||||
// For each group, find its section and extract usernames
|
||||
for (const [sectionName, groupName] of Object.entries(groupMappings)) {
|
||||
// Find the section for this group
|
||||
const sectionRegex = new RegExp(
|
||||
`### ${sectionName}([\\s\\S]*?)(?=### |$)`,
|
||||
'i'
|
||||
);
|
||||
const sectionMatch = content.match(sectionRegex);
|
||||
|
||||
if (sectionMatch) {
|
||||
const sectionContent = sectionMatch[1];
|
||||
const usernames = extractUsernamesFromSection(sectionContent, groupName);
|
||||
userGroups.push(...usernames);
|
||||
} else {
|
||||
logger.warn(`Section not found: ${sectionName}`);
|
||||
}
|
||||
}
|
||||
|
||||
logger.info(
|
||||
`Extracted ${userGroups.length} user-group mappings from markdown`
|
||||
);
|
||||
return userGroups;
|
||||
};
|
||||
|
||||
/**
|
||||
* Extracts usernames from a specific section
|
||||
* @param {string} sectionContent - Content of the section
|
||||
* @param {string} groupName - Name of the group (mdm, orchestration, software)
|
||||
* @returns {Array<{group: string, username: string}>} Array of user group mappings
|
||||
*/
|
||||
const extractUsernamesFromSection = (sectionContent, groupName) => {
|
||||
const userGroups = [];
|
||||
|
||||
// Look for the Developer row in the table
|
||||
// The pattern needs to handle multi-line content in the cell
|
||||
const developerRowMatch = sectionContent.match(
|
||||
/\|\s*Developer\s*\|\s*([\s\S]*?)(?=\n\||\n\n|$)/
|
||||
);
|
||||
|
||||
if (!developerRowMatch) {
|
||||
logger.warn(`No Developer row found in ${groupName} group section`);
|
||||
return userGroups;
|
||||
}
|
||||
|
||||
const developerCell = developerRowMatch[1];
|
||||
|
||||
// Extract GitHub usernames from the developer cell
|
||||
// Look for patterns like [@username](https://github.com/username)
|
||||
// Note: This match could fail with slight variations in formatting (extra spaces, different brackets, etc.).
|
||||
const usernameMatches = developerCell.match(/\[@([a-zA-Z0-9-]+)]\([^)]+\)/g);
|
||||
|
||||
if (!usernameMatches) {
|
||||
logger.warn(
|
||||
`No GitHub usernames found in ${groupName} group Developer row`
|
||||
);
|
||||
return userGroups;
|
||||
}
|
||||
|
||||
const usernames = usernameMatches
|
||||
.map((match) => {
|
||||
// Extract username from _([@username](url))_ format
|
||||
const usernameMatch = match.match(/\[@([a-zA-Z0-9-]+)]/);
|
||||
return usernameMatch ? usernameMatch[1] : null;
|
||||
})
|
||||
.filter(Boolean);
|
||||
|
||||
logger.info(
|
||||
`Found ${usernames.length
|
||||
} developers in ${groupName} group: ${usernames.join(', ')}`
|
||||
);
|
||||
|
||||
// Create user group mappings for both the specific group and engineering
|
||||
for (const username of usernames) {
|
||||
// Add to specific group (mdm, orchestration, software)
|
||||
userGroups.push({ group: groupName, username });
|
||||
|
||||
// Add to engineering group (all developers are in engineering)
|
||||
userGroups.push({ group: 'engineering', username });
|
||||
}
|
||||
|
||||
return userGroups;
|
||||
};
|
||||
|
||||
/**
|
||||
* Validates the structure of the markdown content
|
||||
* @param {string} content - Markdown content to validate
|
||||
* @returns {boolean} True if structure is valid, false otherwise
|
||||
*/
|
||||
export const validateMarkdownStructure = (content) => {
|
||||
const requiredSections = [
|
||||
'MDM group',
|
||||
'Orchestration group',
|
||||
'Software group',
|
||||
];
|
||||
|
||||
for (const section of requiredSections) {
|
||||
if (!content.includes(`### ${section}`)) {
|
||||
logger.warn(`Missing required section: ${section}`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
};
|
||||
|
||||
export default {
|
||||
parseProductGroups,
|
||||
validateMarkdownStructure,
|
||||
};
|
||||
494
.github/actions/eng-metrics/src/metrics-collector.js
vendored
Normal file
494
.github/actions/eng-metrics/src/metrics-collector.js
vendored
Normal file
|
|
@ -0,0 +1,494 @@
|
|||
/**
|
||||
* Engineering metrics collector module
|
||||
* Orchestrates the collection and uploading of comprehensive GitHub engineering metrics:
|
||||
* - Time to First Review (currently implemented)
|
||||
* - Time to Merge (planned)
|
||||
* - Time to QA Ready (planned)
|
||||
* - Time to Production Ready (planned)
|
||||
*/
|
||||
|
||||
import GitHubClient from './github-client.js';
|
||||
import BigQueryClient from './bigquery-client.js';
|
||||
import { UserGroupClient } from './user-group-client.js';
|
||||
import { parseProductGroups } from './markdown-parser.js';
|
||||
import { filterValidUserGroups } from './github-validator.js';
|
||||
import logger from './logger.js';
|
||||
|
||||
/**
|
||||
* Metrics collector class
|
||||
*/
|
||||
export class MetricsCollector {
|
||||
/**
|
||||
* Creates a new metrics collector
|
||||
* @param {Object} config - Configuration object
|
||||
*/
|
||||
constructor(config) {
|
||||
this.config = config;
|
||||
this.githubClient = null;
|
||||
this.bigqueryClient = null;
|
||||
this.userGroupClient = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Initializes the metrics collector
|
||||
*/
|
||||
async initialize() {
|
||||
try {
|
||||
logger.info('Initializing metrics collector');
|
||||
|
||||
// Initialize GitHub client
|
||||
this.githubClient = new GitHubClient(this.config.githubToken);
|
||||
|
||||
// Initialize BigQuery client only if not in print-only mode
|
||||
if (!this.config.printOnly) {
|
||||
this.bigqueryClient = new BigQueryClient(this.config.serviceAccountKeyPath);
|
||||
} else {
|
||||
logger.info('Running in print-only mode, BigQuery client not initialized');
|
||||
}
|
||||
|
||||
// Initialize User Group client if user group processing is enabled
|
||||
if (this.config.userGroupEnabled) {
|
||||
if (!this.config.printOnly) {
|
||||
// Get project ID from BigQuery client
|
||||
const projectId = this.bigqueryClient.getProjectId();
|
||||
this.userGroupClient = new UserGroupClient(
|
||||
projectId,
|
||||
this.config.bigQueryDatasetId,
|
||||
this.config.serviceAccountKeyPath,
|
||||
this.config.printOnly
|
||||
);
|
||||
} else {
|
||||
// For print-only mode, we don't need a real project ID
|
||||
this.userGroupClient = new UserGroupClient(
|
||||
'print-only-project',
|
||||
this.config.bigQueryDatasetId,
|
||||
this.config.serviceAccountKeyPath,
|
||||
this.config.printOnly
|
||||
);
|
||||
}
|
||||
logger.info('User group client initialized');
|
||||
}
|
||||
|
||||
logger.info('Metrics collector initialized');
|
||||
} catch (err) {
|
||||
logger.error('Failed to initialize metrics collector', {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Collects metrics for a single repository
|
||||
* @param {string} repository - Repository in the format owner/repo
|
||||
* @returns {Array} Array of engineering metrics
|
||||
*/
|
||||
async collectRepositoryMetrics(repository) {
|
||||
const [owner, repo] = repository.split('/');
|
||||
if (!owner || !repo) {
|
||||
const err = new Error(`Invalid repository format: ${repository}`);
|
||||
logger.error(`Error collecting metrics for ${repository}`, {}, err);
|
||||
return [];
|
||||
}
|
||||
logger.info(`Collecting metrics for ${repository}`);
|
||||
|
||||
try {
|
||||
// Calculate the date to fetch PRs from (lookbackDays ago)
|
||||
const since = new Date();
|
||||
since.setDate(since.getDate() - this.config.lookbackDays);
|
||||
|
||||
// Fetch PRs updated since the lookback date
|
||||
const pullRequests = await this.githubClient.fetchPullRequests(
|
||||
owner,
|
||||
repo,
|
||||
'all',
|
||||
since,
|
||||
this.config.targetBranch
|
||||
);
|
||||
|
||||
logger.info(`Found ${pullRequests.length} PRs for ${repository}`);
|
||||
|
||||
// Collect metrics for each PR
|
||||
const metrics = [];
|
||||
|
||||
for (const pr of pullRequests) {
|
||||
try {
|
||||
// Fetch PR timeline events (shared for all metrics)
|
||||
const timelineEvents = await this.githubClient.fetchPRTimelineEvents(
|
||||
owner,
|
||||
repo,
|
||||
pr.number
|
||||
);
|
||||
|
||||
// Fetch PR review events (needed for Time to First Review)
|
||||
const rawReviewEvents = await this.githubClient.fetchPRReviewEvents(
|
||||
owner,
|
||||
repo,
|
||||
pr.number
|
||||
);
|
||||
|
||||
// Filter bot reviews if configured
|
||||
const reviewEvents = this.githubClient.filterBotReviews(
|
||||
rawReviewEvents,
|
||||
this.config.excludeBotReviews
|
||||
);
|
||||
|
||||
// Collect enabled metrics for this PR
|
||||
const prMetrics = await this.collectPRMetrics(pr, timelineEvents, reviewEvents);
|
||||
metrics.push(...prMetrics);
|
||||
} catch (err) {
|
||||
logger.error(`Error collecting metrics for PR ${repository}#${pr.number}`, {}, err);
|
||||
}
|
||||
}
|
||||
|
||||
logger.info(`Collected ${metrics.length} metrics for ${repository}`);
|
||||
return metrics;
|
||||
} catch (err) {
|
||||
logger.error(`Error collecting metrics for ${repository}`, {}, err);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Collects enabled metrics for a single PR
|
||||
* @param {Object} pr - Pull request object
|
||||
* @param {Array} timelineEvents - PR timeline events
|
||||
* @param {Array} reviewEvents - PR review events
|
||||
* @returns {Array} Array of metrics for this PR
|
||||
*/
|
||||
async collectPRMetrics(pr, timelineEvents, reviewEvents) {
|
||||
const metrics = [];
|
||||
|
||||
// Collect Time to First Review if enabled
|
||||
if (this.config.metrics.timeToFirstReview.enabled) {
|
||||
try {
|
||||
const pickupTimeMetrics = this.githubClient.calculatePickupTime(
|
||||
pr,
|
||||
timelineEvents,
|
||||
reviewEvents
|
||||
);
|
||||
|
||||
if (pickupTimeMetrics) {
|
||||
metrics.push(pickupTimeMetrics);
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error(`Error calculating Time to First Review for PR #${pr.number}`, {}, err);
|
||||
}
|
||||
}
|
||||
|
||||
// Collect Time to Merge if enabled
|
||||
if (this.config.metrics.timeToMerge.enabled) {
|
||||
try {
|
||||
const mergeTimeMetrics = this.githubClient.calculateTimeToMerge(
|
||||
pr,
|
||||
timelineEvents,
|
||||
reviewEvents
|
||||
);
|
||||
|
||||
if (mergeTimeMetrics) {
|
||||
metrics.push(mergeTimeMetrics);
|
||||
}
|
||||
} catch (err) {
|
||||
logger.error(`Error calculating Time to Merge for PR #${pr.number}`, {}, err);
|
||||
}
|
||||
}
|
||||
|
||||
return metrics;
|
||||
}
|
||||
|
||||
/**
|
||||
* Collects metrics for all repositories
|
||||
* @returns {Array} Array of engineering metrics
|
||||
*/
|
||||
async collectMetrics() {
|
||||
try {
|
||||
logger.info('Collecting metrics for all repositories');
|
||||
|
||||
const allMetrics = [];
|
||||
|
||||
// Collect metrics for each repository
|
||||
for (const repository of this.config.repositories) {
|
||||
const metrics = await this.collectRepositoryMetrics(repository);
|
||||
allMetrics.push(...metrics);
|
||||
}
|
||||
|
||||
logger.info(`Collected ${allMetrics.length} metrics in total`);
|
||||
return allMetrics;
|
||||
} catch (err) {
|
||||
logger.error('Error collecting metrics', {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Processes user groups from the markdown file
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
async processUserGroups() {
|
||||
if (!this.config.userGroupEnabled) {
|
||||
logger.info('User group processing is disabled');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
logger.info('Processing user groups');
|
||||
|
||||
// Parse user groups from markdown file
|
||||
const userGroups = parseProductGroups(this.config.userGroupFilepath);
|
||||
|
||||
if (userGroups.length === 0) {
|
||||
logger.warn('No user groups found in markdown file');
|
||||
return;
|
||||
}
|
||||
|
||||
// Validate GitHub usernames
|
||||
const validUserGroups = await filterValidUserGroups(
|
||||
this.config.githubToken,
|
||||
userGroups
|
||||
);
|
||||
|
||||
if (validUserGroups.length === 0) {
|
||||
logger.warn('No valid user groups found after validation');
|
||||
return;
|
||||
}
|
||||
|
||||
// Sync user groups to BigQuery
|
||||
await this.userGroupClient.syncUserGroups(validUserGroups);
|
||||
|
||||
logger.info(`Successfully processed ${validUserGroups.length} user group mappings`);
|
||||
} catch (err) {
|
||||
logger.error('Error processing user groups', {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Prints metrics to the console in a readable format
|
||||
* @param {Array} metrics - Array of engineering metrics
|
||||
*/
|
||||
printMetrics(metrics) {
|
||||
try {
|
||||
if (!metrics || metrics.length === 0) {
|
||||
logger.warn('No metrics to print');
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info(`Printing ${metrics.length} metrics to console`);
|
||||
|
||||
// Group metrics by type for organized display
|
||||
const metricsByType = this.groupMetricsByType(metrics);
|
||||
|
||||
console.log('\n=== Engineering Metrics ===\n');
|
||||
|
||||
// Print each metric type separately
|
||||
for (const [metricType, typeMetrics] of Object.entries(metricsByType)) {
|
||||
if (typeMetrics.length === 0) continue;
|
||||
|
||||
console.log(`--- ${this.getMetricTypeDisplayName(metricType)} (${typeMetrics.length} metrics) ---\n`);
|
||||
|
||||
// Sort metrics by time (descending)
|
||||
const sortedMetrics = [...typeMetrics].sort((a, b) => {
|
||||
const timeFieldA = this.getTimeFieldForMetricType(metricType, a);
|
||||
const timeFieldB = this.getTimeFieldForMetricType(metricType, b);
|
||||
return timeFieldB - timeFieldA;
|
||||
});
|
||||
|
||||
// Print each metric
|
||||
sortedMetrics.forEach((metric, index) => {
|
||||
this.printSingleMetric(metric, index + 1);
|
||||
});
|
||||
|
||||
// Print summary statistics for this metric type
|
||||
this.printMetricTypeSummary(metricType, typeMetrics);
|
||||
console.log('');
|
||||
}
|
||||
|
||||
logger.info('Metrics printed successfully');
|
||||
} catch (err) {
|
||||
logger.error('Error printing metrics', {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Prints a single metric to the console
|
||||
* @param {Object} metric - Single metric object
|
||||
* @param {number} index - Index for display
|
||||
*/
|
||||
printSingleMetric(metric, index) {
|
||||
console.log(`[${index}] PR: ${metric.repository}#${metric.prNumber}`);
|
||||
console.log(` URL: ${metric.prUrl}`);
|
||||
console.log(` Creator: ${metric.prCreator}`);
|
||||
console.log(` Ready Time: ${metric.readyTime.toISOString()}${metric.readyEventType ? ` (${metric.readyEventType})` : ''}`);
|
||||
|
||||
if (metric.metricType === 'time_to_first_review') {
|
||||
const hours = Math.floor(metric.pickupTimeSeconds / 3600);
|
||||
const minutes = Math.floor((metric.pickupTimeSeconds % 3600) / 60);
|
||||
const seconds = metric.pickupTimeSeconds % 60;
|
||||
|
||||
console.log(` First Review Time: ${metric.firstReviewTime.toISOString()}`);
|
||||
console.log(` Pickup Time: ${hours}h ${minutes}m ${seconds}s (${metric.pickupTimeSeconds} seconds)`);
|
||||
} else if (metric.metricType === 'time_to_merge') {
|
||||
const hours = Math.floor(metric.mergeTimeSeconds / 3600);
|
||||
const minutes = Math.floor((metric.mergeTimeSeconds % 3600) / 60);
|
||||
const seconds = metric.mergeTimeSeconds % 60;
|
||||
|
||||
console.log(` Merge Time: ${metric.mergeTime.toISOString()}`);
|
||||
console.log(` Time to Merge: ${hours}h ${minutes}m ${seconds}s (${metric.mergeTimeSeconds} seconds)`);
|
||||
}
|
||||
|
||||
console.log('');
|
||||
}
|
||||
|
||||
/**
|
||||
* Prints summary statistics for a metric type
|
||||
* @param {string} metricType - Type of metric
|
||||
* @param {Array} metrics - Array of metrics of this type
|
||||
*/
|
||||
printMetricTypeSummary(metricType, metrics) {
|
||||
const timeField = metricType === 'time_to_first_review' ? 'pickupTimeSeconds' : 'mergeTimeSeconds';
|
||||
const totalTime = metrics.reduce((sum, metric) => sum + metric[timeField], 0);
|
||||
const avgTime = totalTime / metrics.length;
|
||||
const avgHours = Math.floor(avgTime / 3600);
|
||||
const avgMinutes = Math.floor((avgTime % 3600) / 60);
|
||||
const avgSeconds = Math.floor(avgTime % 60);
|
||||
|
||||
console.log(`=== ${this.getMetricTypeDisplayName(metricType)} Summary ===`);
|
||||
console.log(`Total PRs: ${metrics.length}`);
|
||||
console.log(`Average Time: ${avgHours}h ${avgMinutes}m ${avgSeconds}s (${Math.floor(avgTime)} seconds)`);
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets display name for metric type
|
||||
* @param {string} metricType - Type of metric
|
||||
* @returns {string} Display name
|
||||
*/
|
||||
getMetricTypeDisplayName(metricType) {
|
||||
switch (metricType) {
|
||||
case 'time_to_first_review':
|
||||
return 'Time to First Review';
|
||||
case 'time_to_merge':
|
||||
return 'Time to Merge';
|
||||
default:
|
||||
return metricType;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the time field value for sorting metrics
|
||||
* @param {string} metricType - Type of metric
|
||||
* @param {Object} metric - Metric object
|
||||
* @returns {number} Time value in seconds
|
||||
*/
|
||||
getTimeFieldForMetricType(metricType, metric) {
|
||||
switch (metricType) {
|
||||
case 'time_to_first_review':
|
||||
return metric.pickupTimeSeconds;
|
||||
case 'time_to_merge':
|
||||
return metric.mergeTimeSeconds;
|
||||
default:
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Uploads metrics to BigQuery, grouped by metric type
|
||||
* @param {Array} metrics - Array of engineering metrics
|
||||
*/
|
||||
async uploadMetrics(metrics) {
|
||||
try {
|
||||
if (!metrics || metrics.length === 0) {
|
||||
logger.warn('No metrics to upload');
|
||||
return;
|
||||
}
|
||||
|
||||
logger.info(`Uploading ${metrics.length} metrics to BigQuery`);
|
||||
|
||||
// Group metrics by type
|
||||
const metricsByType = this.groupMetricsByType(metrics);
|
||||
|
||||
// Upload each metric type to its respective table
|
||||
for (const [metricType, typeMetrics] of Object.entries(metricsByType)) {
|
||||
if (typeMetrics.length === 0) continue;
|
||||
|
||||
const tableName = this.getTableNameForMetricType(metricType);
|
||||
logger.info(`Uploading ${typeMetrics.length} ${metricType} metrics to table ${tableName}`);
|
||||
|
||||
await this.bigqueryClient.uploadMetrics(
|
||||
this.config.bigQueryDatasetId,
|
||||
tableName,
|
||||
typeMetrics
|
||||
);
|
||||
}
|
||||
|
||||
logger.info('All metrics uploaded successfully');
|
||||
} catch (err) {
|
||||
logger.error('Error uploading metrics to BigQuery', {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Groups metrics by their type
|
||||
* @param {Array} metrics - Array of metrics
|
||||
* @returns {Object} Object with metric types as keys and arrays of metrics as values
|
||||
*/
|
||||
groupMetricsByType(metrics) {
|
||||
return metrics.reduce((groups, metric) => {
|
||||
const type = metric.metricType;
|
||||
if (!groups[type]) {
|
||||
groups[type] = [];
|
||||
}
|
||||
groups[type].push(metric);
|
||||
return groups;
|
||||
}, {});
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets the table name for a specific metric type
|
||||
* @param {string} metricType - Type of metric
|
||||
* @returns {string} Table name
|
||||
*/
|
||||
getTableNameForMetricType(metricType) {
|
||||
switch (metricType) {
|
||||
case 'time_to_first_review':
|
||||
return this.config.metrics.timeToFirstReview.tableName;
|
||||
case 'time_to_merge':
|
||||
return this.config.metrics.timeToMerge.tableName;
|
||||
default:
|
||||
throw new Error(`Unknown metric type: ${metricType}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Runs the metrics collection and upload process
|
||||
*/
|
||||
async run() {
|
||||
try {
|
||||
logger.info('Starting engineering metrics collection');
|
||||
|
||||
// Initialize the metrics collector
|
||||
await this.initialize();
|
||||
|
||||
// Process user groups if enabled
|
||||
await this.processUserGroups();
|
||||
|
||||
// Collect metrics
|
||||
const metrics = await this.collectMetrics();
|
||||
|
||||
if (this.config.printOnly) {
|
||||
// Print metrics to console
|
||||
this.printMetrics(metrics);
|
||||
} else {
|
||||
// Upload metrics to BigQuery
|
||||
await this.uploadMetrics(metrics);
|
||||
}
|
||||
|
||||
logger.info('Engineering metrics collection completed successfully');
|
||||
return metrics;
|
||||
} catch (err) {
|
||||
logger.error('Error running engineering metrics collection', {}, err);
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export default MetricsCollector;
|
||||
374
.github/actions/eng-metrics/src/user-group-client.js
vendored
Normal file
374
.github/actions/eng-metrics/src/user-group-client.js
vendored
Normal file
|
|
@ -0,0 +1,374 @@
|
|||
/**
|
||||
* User Group BigQuery Client
|
||||
* Manages the user_group BigQuery table for team organization tracking
|
||||
*/
|
||||
|
||||
import { BigQuery } from '@google-cloud/bigquery';
|
||||
import logger from './logger.js';
|
||||
|
||||
/**
|
||||
* User Group BigQuery Client class
|
||||
*/
|
||||
export class UserGroupClient {
|
||||
/**
|
||||
* Creates a new UserGroupClient instance
|
||||
* @param {string} projectId - Google Cloud project ID
|
||||
* @param {string} datasetId - BigQuery dataset ID
|
||||
* @param {string} serviceAccountKeyPath - Path to service account key file
|
||||
* @param {boolean} printOnly - If true, print operations instead of executing them
|
||||
*/
|
||||
constructor(projectId, datasetId, serviceAccountKeyPath, printOnly = false) {
|
||||
this.projectId = projectId;
|
||||
this.datasetId = datasetId;
|
||||
this.tableId = 'user_group';
|
||||
this.printOnly = printOnly;
|
||||
|
||||
if (!printOnly) {
|
||||
this.bigquery = new BigQuery({
|
||||
projectId,
|
||||
keyFilename: serviceAccountKeyPath,
|
||||
});
|
||||
this.dataset = this.bigquery.dataset(datasetId);
|
||||
this.table = this.dataset.table(this.tableId);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates the user_group table if it doesn't exist
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
async createUserGroupTable() {
|
||||
if (this.printOnly) {
|
||||
logger.info(
|
||||
'[USER GROUPS] Print-only mode: would create user_group table with schema:'
|
||||
);
|
||||
logger.info('[USER GROUPS] - group (STRING, cluster key)');
|
||||
logger.info('[USER GROUPS] - username (STRING)');
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
const [exists] = await this.table.exists();
|
||||
|
||||
if (exists) {
|
||||
logger.info('user_group table already exists');
|
||||
return;
|
||||
}
|
||||
|
||||
const schema = [
|
||||
{ name: 'group', type: 'STRING', mode: 'REQUIRED' },
|
||||
{ name: 'username', type: 'STRING', mode: 'REQUIRED' },
|
||||
];
|
||||
|
||||
const options = {
|
||||
schema,
|
||||
clustering: {
|
||||
fields: ['group'],
|
||||
},
|
||||
};
|
||||
|
||||
await this.table.create(options);
|
||||
logger.info('Created user_group table with clustering on group field');
|
||||
} catch (error) {
|
||||
logger.error('Error creating user_group table:', {}, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Syncs user groups to the BigQuery table (replaces all data)
|
||||
* @param {Array<{group: string, username: string}>} userGroups - Array of user group mappings
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
async syncUserGroups(userGroups) {
|
||||
if (this.printOnly) {
|
||||
this.printUserGroupsData(userGroups);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
if (userGroups.length === 0) {
|
||||
logger.warn('No user groups to sync');
|
||||
return;
|
||||
}
|
||||
|
||||
// Ensure table exists first
|
||||
await this.createUserGroupTable();
|
||||
|
||||
// Small delay to ensure table is ready
|
||||
await new Promise((resolve) => setTimeout(resolve, 1000));
|
||||
|
||||
// Refresh table reference
|
||||
this.table = this.dataset.table(this.tableId);
|
||||
|
||||
// Get existing entries and perform differential sync
|
||||
await this.syncUserGroupsDifferential(userGroups);
|
||||
|
||||
logger.info(
|
||||
`Successfully synced ${userGroups.length} user group mappings to BigQuery`
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error('Error syncing user groups:', {}, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Performs differential sync of user groups (only insert new, delete removed)
|
||||
* @param {Array<{group: string, username: string}>} newUserGroups - New user group mappings
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
async syncUserGroupsDifferential(newUserGroups) {
|
||||
if (this.printOnly) {
|
||||
logger.info(
|
||||
'[USER GROUPS] Print-only mode: would perform differential sync'
|
||||
);
|
||||
this.printDifferentialSync(newUserGroups);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
// Get existing entries from the table
|
||||
const existingUserGroups = await this.getExistingUserGroups();
|
||||
|
||||
// Create sets for comparison
|
||||
const existingSet = new Set(
|
||||
existingUserGroups.map((ug) => `${ug.group}:${ug.username}`)
|
||||
);
|
||||
const newSet = new Set(
|
||||
newUserGroups.map((ug) => `${ug.group}:${ug.username}`)
|
||||
);
|
||||
|
||||
// Find entries to insert (in new but not in existing)
|
||||
const toInsert = newUserGroups.filter(
|
||||
(ug) => !existingSet.has(`${ug.group}:${ug.username}`)
|
||||
);
|
||||
|
||||
// Find entries to delete (in existing but not in new)
|
||||
const toDelete = existingUserGroups.filter(
|
||||
(ug) => !newSet.has(`${ug.group}:${ug.username}`)
|
||||
);
|
||||
|
||||
logger.info(
|
||||
`Differential sync: ${toInsert.length} to insert, ${toDelete.length} to delete`
|
||||
);
|
||||
|
||||
// Handle deletions with streaming buffer awareness
|
||||
if (toDelete.length > 0) {
|
||||
try {
|
||||
await this.deleteUserGroups(toDelete);
|
||||
} catch (error) {
|
||||
if (error.message?.includes('streaming buffer')) {
|
||||
logger.warn(
|
||||
'Streaming buffer prevents DELETE - this is expected during testing. In production (daily runs), this won\'t occur.'
|
||||
);
|
||||
logger.info(
|
||||
'Skipping deletions for now. Data will be consistent on next daily run.'
|
||||
);
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Insert new entries using load job (no streaming buffer issues for inserts)
|
||||
if (toInsert.length > 0) {
|
||||
await this.insertUserGroups(toInsert);
|
||||
}
|
||||
|
||||
if (toInsert.length === 0 && toDelete.length === 0) {
|
||||
logger.info('No changes needed - user groups are already up to date');
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Error performing differential sync:', {}, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Gets existing user groups from the BigQuery table
|
||||
* @returns {Promise<Array<{group: string, username: string}>>}
|
||||
*/
|
||||
async getExistingUserGroups() {
|
||||
try {
|
||||
const query = `SELECT \`group\`, username FROM \`${this.projectId}.${this.datasetId}.${this.tableId}\``;
|
||||
const [rows] = await this.bigquery.query({ query });
|
||||
|
||||
logger.info(`Found ${rows.length} existing user group entries`);
|
||||
return rows.map((row) => ({
|
||||
group: row.group,
|
||||
username: row.username,
|
||||
}));
|
||||
} catch (error) {
|
||||
if (error.message?.includes('not found')) {
|
||||
logger.info('Table does not exist yet, no existing entries');
|
||||
return [];
|
||||
}
|
||||
logger.error('Error getting existing user groups:', {}, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Deletes specific user groups from the BigQuery table
|
||||
* @param {Array<{group: string, username: string}>} userGroupsToDelete
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
async deleteUserGroups(userGroupsToDelete) {
|
||||
try {
|
||||
const groups = userGroupsToDelete.map((ug) => ug.group);
|
||||
const usernames = userGroupsToDelete.map((ug) => ug.username);
|
||||
|
||||
const query = `
|
||||
DELETE
|
||||
FROM \`${this.projectId}.${this.datasetId}.${this.tableId}\`
|
||||
WHERE EXISTS (SELECT 1
|
||||
FROM UNNEST(@groups) AS g WITH OFFSET AS pos
|
||||
JOIN UNNEST(@usernames) AS u WITH OFFSET AS upos
|
||||
ON pos = upos
|
||||
WHERE \`group\` = g AND username = u
|
||||
)
|
||||
`;
|
||||
|
||||
const options = {
|
||||
query,
|
||||
params: { groups, usernames },
|
||||
};
|
||||
|
||||
await this.bigquery.query(options);
|
||||
|
||||
logger.info(`Deleted ${userGroupsToDelete.length} user group entries`);
|
||||
} catch (error) {
|
||||
logger.error('Error deleting user groups:', {}, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Prints differential sync information for print-only mode
|
||||
* @param {Array<{group: string, username: string}>} newUserGroups
|
||||
*/
|
||||
async printDifferentialSync(newUserGroups) {
|
||||
logger.info(
|
||||
'[USER GROUPS] Print-only mode: would perform differential sync'
|
||||
);
|
||||
logger.info('[USER GROUPS] Would check existing entries in table');
|
||||
logger.info(
|
||||
`[USER GROUPS] Would compare with ${newUserGroups.length} new entries`
|
||||
);
|
||||
logger.info(
|
||||
'[USER GROUPS] Would insert only new entries and delete removed entries'
|
||||
);
|
||||
|
||||
// Group users by their groups for better display
|
||||
const groupedUsers = newUserGroups.reduce((acc, ug) => {
|
||||
if (!acc[ug.group]) {
|
||||
acc[ug.group] = [];
|
||||
}
|
||||
acc[ug.group].push(ug.username);
|
||||
return acc;
|
||||
}, {});
|
||||
|
||||
logger.info('[USER GROUPS] New user groups to sync:');
|
||||
for (const [group, usernames] of Object.entries(groupedUsers)) {
|
||||
logger.info(` ${group}: ${usernames.join(', ')}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Inserts user groups into the BigQuery table using load job (avoids streaming buffer)
|
||||
* @param {Array<{group: string, username: string}>} userGroups - Array of user group mappings
|
||||
* @returns {Promise<void>}
|
||||
*/
|
||||
async insertUserGroups(userGroups) {
|
||||
if (this.printOnly) {
|
||||
logger.info(
|
||||
`[USER GROUPS] Print-only mode: would insert ${userGroups.length} records`
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
// Transform data for BigQuery streaming insert
|
||||
const rows = userGroups.map((ug) => ({
|
||||
group: ug.group,
|
||||
username: ug.username,
|
||||
}));
|
||||
|
||||
// Use streaming insert - simpler and appropriate for small user group datasets
|
||||
// Streaming buffer issues only occur during rapid testing, not daily production runs
|
||||
await this.table.insert(rows);
|
||||
|
||||
logger.info(
|
||||
`Inserted ${rows.length} user group mappings into BigQuery using streaming insert`
|
||||
);
|
||||
} catch (error) {
|
||||
logger.error('Error inserting user groups:', {}, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Prints user groups data in a readable format (for print-only mode)
|
||||
* @param {Array<{group: string, username: string}>} userGroups - Array of user group mappings
|
||||
*/
|
||||
printUserGroupsData(userGroups) {
|
||||
logger.info(
|
||||
'[USER GROUPS] Print-only mode enabled - no BigQuery updates will be performed'
|
||||
);
|
||||
logger.info('');
|
||||
|
||||
if (userGroups.length === 0) {
|
||||
logger.info('[USER GROUPS] No user groups found to process');
|
||||
return;
|
||||
}
|
||||
|
||||
// Group users by their groups for better display
|
||||
const groupedUsers = userGroups.reduce((acc, ug) => {
|
||||
if (!acc[ug.group]) {
|
||||
acc[ug.group] = [];
|
||||
}
|
||||
acc[ug.group].push(ug.username);
|
||||
return acc;
|
||||
}, {});
|
||||
|
||||
logger.info('[USER GROUPS] Extracted user groups:');
|
||||
for (const [group, usernames] of Object.entries(groupedUsers)) {
|
||||
logger.info(` ${group}: ${usernames.join(', ')}`);
|
||||
}
|
||||
|
||||
logger.info('');
|
||||
logger.info(
|
||||
`[USER GROUPS] Would insert ${userGroups.length} records into user_group table:`
|
||||
);
|
||||
|
||||
// Count users per group
|
||||
const groupCounts = Object.entries(groupedUsers).map(
|
||||
([group, usernames]) => ` ${group}: ${usernames.length} users`
|
||||
);
|
||||
logger.info(groupCounts.join('\n'));
|
||||
|
||||
logger.info('');
|
||||
logger.info('[USER GROUPS] Sample records that would be inserted:');
|
||||
logger.info('| group | username |');
|
||||
logger.info('|---------------|-----------------|');
|
||||
|
||||
// Show first few records from each group
|
||||
const sampleRecords = [];
|
||||
for (const [group, usernames] of Object.entries(groupedUsers)) {
|
||||
const sampleUser = usernames[0];
|
||||
sampleRecords.push(`| ${group.padEnd(13)} | ${sampleUser.padEnd(15)} |`);
|
||||
if (sampleRecords.length >= 6) break; // Limit sample output
|
||||
}
|
||||
|
||||
sampleRecords.forEach((record) => logger.info(record));
|
||||
|
||||
if (userGroups.length > sampleRecords.length) {
|
||||
logger.info(
|
||||
`... and ${userGroups.length - sampleRecords.length} more records`
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export default UserGroupClient;
|
||||
275
.github/actions/eng-metrics/test/bigquery-client.test.js
vendored
Normal file
275
.github/actions/eng-metrics/test/bigquery-client.test.js
vendored
Normal file
|
|
@ -0,0 +1,275 @@
|
|||
/**
|
||||
* Tests for BigQuery client module
|
||||
*/
|
||||
|
||||
import { jest } from '@jest/globals';
|
||||
import { BigQueryClient } from '../src/bigquery-client.js';
|
||||
|
||||
// Mock the logger
|
||||
jest.mock('../src/logger.js', () => ({
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn()
|
||||
}));
|
||||
|
||||
// Mock fs
|
||||
jest.mock('fs', () => ({
|
||||
existsSync: jest.fn(() => true),
|
||||
readFileSync: jest.fn(() => JSON.stringify({
|
||||
project_id: 'test-project-id',
|
||||
type: 'service_account',
|
||||
private_key_id: 'key-id',
|
||||
private_key: '-----BEGIN PRIVATE KEY-----\ntest-key\n-----END PRIVATE KEY-----\n',
|
||||
client_email: 'test@test-project-id.iam.gserviceaccount.com',
|
||||
client_id: '123456789',
|
||||
auth_uri: 'https://accounts.google.com/o/oauth2/auth',
|
||||
token_uri: 'https://oauth2.googleapis.com/token'
|
||||
}))
|
||||
}));
|
||||
|
||||
// Mock @google-cloud/bigquery
|
||||
jest.mock('@google-cloud/bigquery', () => ({
|
||||
BigQuery: jest.fn(() => ({
|
||||
dataset: jest.fn(),
|
||||
query: jest.fn()
|
||||
}))
|
||||
}));
|
||||
|
||||
describe('BigQueryClient', () => {
|
||||
let bigqueryClient;
|
||||
let mockBigQuery;
|
||||
let mockDataset;
|
||||
let mockTable;
|
||||
|
||||
beforeEach(() => {
|
||||
mockTable = {
|
||||
exists: jest.fn(() => [true]),
|
||||
create: jest.fn(),
|
||||
insert: jest.fn(() => [{}])
|
||||
};
|
||||
|
||||
mockDataset = {
|
||||
exists: jest.fn(() => [true]),
|
||||
create: jest.fn(),
|
||||
table: jest.fn(() => mockTable)
|
||||
};
|
||||
|
||||
mockBigQuery = {
|
||||
dataset: jest.fn(() => mockDataset),
|
||||
query: jest.fn(() => [[]])
|
||||
};
|
||||
|
||||
// Create client without calling constructor to avoid file check
|
||||
bigqueryClient = Object.create(BigQueryClient.prototype);
|
||||
bigqueryClient.bigquery = mockBigQuery;
|
||||
bigqueryClient.projectId = 'test-project-id';
|
||||
});
|
||||
|
||||
|
||||
describe('getProjectId', () => {
|
||||
test('should return the extracted project ID', () => {
|
||||
expect(bigqueryClient.getProjectId()).toBe('test-project-id');
|
||||
});
|
||||
});
|
||||
|
||||
describe('getSchemaForMetricType', () => {
|
||||
test('should return first_review table schema', () => {
|
||||
const schema = bigqueryClient.getSchemaForMetricType('time_to_first_review');
|
||||
expect(schema.fields).toBeTruthy();
|
||||
});
|
||||
|
||||
test('should return pr_merge table schema', () => {
|
||||
const schema = bigqueryClient.getSchemaForMetricType('time_to_merge');
|
||||
expect(schema.fields).toBeTruthy();
|
||||
});
|
||||
|
||||
test('should throw error for unknown metric type', () => {
|
||||
expect(() => {
|
||||
bigqueryClient.getSchemaForMetricType('unknown_metric');
|
||||
}).toThrow('Unknown metric type: unknown_metric');
|
||||
});
|
||||
});
|
||||
|
||||
describe('getConfigurationForMetricType', () => {
|
||||
test('should return first_review table configuration', () => {
|
||||
const config = bigqueryClient.getConfigurationForMetricType('time_to_first_review');
|
||||
expect(config).toBeTruthy();
|
||||
});
|
||||
|
||||
test('should return pr_merge table configuration', () => {
|
||||
const config = bigqueryClient.getConfigurationForMetricType('time_to_merge');
|
||||
expect(config).toBeTruthy();
|
||||
});
|
||||
|
||||
test('should throw error for unknown metric type configuration', () => {
|
||||
expect(() => {
|
||||
bigqueryClient.getConfigurationForMetricType('unknown_metric');
|
||||
}).toThrow('Unknown metric type for table configuration: unknown_metric');
|
||||
});
|
||||
});
|
||||
|
||||
describe('transformMetricsToRow', () => {
|
||||
test('should transform time_to_first_review metrics', () => {
|
||||
const metrics = {
|
||||
metricType: 'time_to_first_review',
|
||||
reviewDate: '2023-06-15',
|
||||
prCreator: 'testuser',
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
pickupTimeSeconds: 7200,
|
||||
repository: 'owner/repo',
|
||||
prNumber: 123,
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-06-15T10:00:00Z'),
|
||||
firstReviewTime: new Date('2023-06-15T12:00:00Z')
|
||||
};
|
||||
|
||||
const row = bigqueryClient.transformMetricsToRow(metrics);
|
||||
|
||||
expect(row).toEqual({
|
||||
review_date: '2023-06-15',
|
||||
pr_creator: 'testuser',
|
||||
pr_url: 'https://github.com/owner/repo/pull/123',
|
||||
pickup_time_seconds: 7200,
|
||||
repository: 'owner/repo',
|
||||
pr_number: 123,
|
||||
target_branch: 'main',
|
||||
ready_time: '2023-06-15T10:00:00.000Z',
|
||||
first_review_time: '2023-06-15T12:00:00.000Z'
|
||||
});
|
||||
});
|
||||
|
||||
test('should transform time_to_merge metrics', () => {
|
||||
const metrics = {
|
||||
metricType: 'time_to_merge',
|
||||
mergeDate: '2023-06-15',
|
||||
prCreator: 'testuser',
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
mergeTimeSeconds: 16200,
|
||||
repository: 'owner/repo',
|
||||
prNumber: 123,
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-06-15T10:00:00Z'),
|
||||
mergeTime: new Date('2023-06-15T14:30:00Z')
|
||||
};
|
||||
|
||||
const row = bigqueryClient.transformMetricsToRow(metrics);
|
||||
|
||||
expect(row).toEqual({
|
||||
merge_date: '2023-06-15',
|
||||
pr_creator: 'testuser',
|
||||
pr_url: 'https://github.com/owner/repo/pull/123',
|
||||
merge_time_seconds: 16200,
|
||||
repository: 'owner/repo',
|
||||
pr_number: 123,
|
||||
target_branch: 'main',
|
||||
ready_time: '2023-06-15T10:00:00.000Z',
|
||||
merge_time: '2023-06-15T14:30:00.000Z'
|
||||
});
|
||||
});
|
||||
|
||||
test('should throw error for unknown metric type', () => {
|
||||
const metrics = {
|
||||
metricType: 'unknown_type'
|
||||
};
|
||||
|
||||
expect(() => {
|
||||
bigqueryClient.transformMetricsToRow(metrics);
|
||||
}).toThrow('Unknown metric type: unknown_type');
|
||||
});
|
||||
});
|
||||
|
||||
describe('createTableIfNotExists', () => {
|
||||
test('should create table with correct configuration for first_review', async () => {
|
||||
mockTable.exists.mockResolvedValue([false]);
|
||||
const schema = { fields: [] };
|
||||
|
||||
await bigqueryClient.createTableIfNotExists('test_dataset', 'first_review', schema, 'time_to_first_review');
|
||||
|
||||
expect(mockTable.create).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('should create table with correct configuration for pr_merge', async () => {
|
||||
mockTable.exists.mockResolvedValue([false]);
|
||||
const schema = { fields: [] };
|
||||
|
||||
await bigqueryClient.createTableIfNotExists('test_dataset', 'pr_merge', schema, 'time_to_merge');
|
||||
|
||||
expect(mockTable.create).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('should not create table if it already exists', async () => {
|
||||
mockTable.exists.mockResolvedValue([true]);
|
||||
const schema = { fields: [] };
|
||||
|
||||
await bigqueryClient.createTableIfNotExists('test_dataset', 'first_review', schema, 'time_to_first_review');
|
||||
|
||||
expect(mockTable.create).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('uploadMetrics', () => {
|
||||
test('should upload metrics with correct schema', async () => {
|
||||
const metrics = [
|
||||
{
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 123,
|
||||
reviewDate: '2023-06-15',
|
||||
prCreator: 'testuser',
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
pickupTimeSeconds: 7200,
|
||||
repository: 'owner/repo',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-06-15T10:00:00Z'),
|
||||
firstReviewTime: new Date('2023-06-15T12:00:00Z')
|
||||
}
|
||||
];
|
||||
|
||||
await bigqueryClient.uploadMetrics('test_dataset', 'pr_first_review', metrics);
|
||||
|
||||
expect(mockTable.insert).toHaveBeenCalledWith([
|
||||
{
|
||||
review_date: '2023-06-15',
|
||||
pr_creator: 'testuser',
|
||||
pr_url: 'https://github.com/owner/repo/pull/123',
|
||||
pickup_time_seconds: 7200,
|
||||
repository: 'owner/repo',
|
||||
pr_number: 123,
|
||||
target_branch: 'main',
|
||||
ready_time: '2023-06-15T10:00:00.000Z',
|
||||
first_review_time: '2023-06-15T12:00:00.000Z'
|
||||
}
|
||||
]);
|
||||
});
|
||||
|
||||
test('should handle empty metrics array', async () => {
|
||||
await bigqueryClient.uploadMetrics('test_dataset', 'pr_first_review', []);
|
||||
|
||||
expect(mockTable.insert).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('should filter out existing metrics', async () => {
|
||||
const metrics = [
|
||||
{
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 123,
|
||||
reviewDate: '2023-06-15',
|
||||
prCreator: 'testuser',
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
pickupTimeSeconds: 7200,
|
||||
repository: 'owner/repo',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-06-15T10:00:00Z'),
|
||||
firstReviewTime: new Date('2023-06-15T12:00:00Z')
|
||||
}
|
||||
];
|
||||
|
||||
// Mock existing metrics check to return that PR 123 already exists
|
||||
mockBigQuery.query.mockResolvedValue([[{ pr_number: 123 }]]);
|
||||
|
||||
await bigqueryClient.uploadMetrics('test_dataset', 'pr_first_review', metrics);
|
||||
|
||||
expect(mockTable.insert).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
210
.github/actions/eng-metrics/test/bot-detection.test.js
vendored
Normal file
210
.github/actions/eng-metrics/test/bot-detection.test.js
vendored
Normal file
|
|
@ -0,0 +1,210 @@
|
|||
/**
|
||||
* Tests for bot detection functionality
|
||||
*/
|
||||
|
||||
import { jest } from '@jest/globals';
|
||||
|
||||
// Mock the logger
|
||||
const mockLogger = {
|
||||
info: jest.fn(),
|
||||
debug: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn()
|
||||
};
|
||||
|
||||
jest.unstable_mockModule('../src/logger.js', () => ({
|
||||
default: mockLogger
|
||||
}));
|
||||
|
||||
const { GitHubClient } = await import('../src/github-client.js');
|
||||
|
||||
describe('Bot Detection', () => {
|
||||
let githubClient;
|
||||
|
||||
beforeEach(() => {
|
||||
githubClient = new GitHubClient('fake-token');
|
||||
jest.clearAllMocks();
|
||||
});
|
||||
|
||||
describe('identifyBotUser', () => {
|
||||
test('should identify GitHub API Bot type', () => {
|
||||
const botUser = {
|
||||
login: 'coderabbitai[bot]',
|
||||
type: 'Bot',
|
||||
name: 'CodeRabbit AI',
|
||||
bio: 'AI-powered code review assistant'
|
||||
};
|
||||
|
||||
// Access the function through the module (it's not exported as a method)
|
||||
// We'll test it indirectly through filterBotReviews
|
||||
const reviews = [
|
||||
{
|
||||
user: botUser,
|
||||
state: 'COMMENTED',
|
||||
submitted_at: '2023-06-15T12:00:00Z'
|
||||
}
|
||||
];
|
||||
|
||||
const filtered = githubClient.filterBotReviews(reviews, true);
|
||||
expect(filtered).toHaveLength(0);
|
||||
});
|
||||
|
||||
test('should identify bot by username patterns', () => {
|
||||
const testCases = [
|
||||
{ login: 'dependabot[bot]', type: 'User' },
|
||||
{ login: 'renovate[bot]', type: 'User' },
|
||||
{ login: 'github-actions[bot]', type: 'User' },
|
||||
{ login: 'codecov-commenter', type: 'User' },
|
||||
{ login: 'coderabbitai', type: 'User' },
|
||||
{ login: 'sonarcloud[bot]', type: 'User' },
|
||||
{ login: 'snyk-bot', type: 'User' },
|
||||
{ login: 'greenkeeper[bot]', type: 'User' },
|
||||
{ login: 'semantic-release-bot', type: 'User' },
|
||||
{ login: 'stale[bot]', type: 'User' },
|
||||
{ login: 'imgbot[bot]', type: 'User' },
|
||||
{ login: 'allcontributors[bot]', type: 'User' },
|
||||
{ login: 'whitesource-bolt', type: 'User' },
|
||||
{ login: 'deepsource-autofix[bot]', type: 'User' }
|
||||
];
|
||||
|
||||
testCases.forEach(({ login, type }) => {
|
||||
const reviews = [
|
||||
{
|
||||
user: { login, type },
|
||||
state: 'COMMENTED',
|
||||
submitted_at: '2023-06-15T12:00:00Z'
|
||||
}
|
||||
];
|
||||
|
||||
const filtered = githubClient.filterBotReviews(reviews, true);
|
||||
expect(filtered).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
|
||||
test('should not filter human users', () => {
|
||||
const humanUser = {
|
||||
login: 'johndoe',
|
||||
type: 'User',
|
||||
name: 'John Doe',
|
||||
bio: 'Software engineer'
|
||||
};
|
||||
|
||||
const reviews = [
|
||||
{
|
||||
user: humanUser,
|
||||
state: 'APPROVED',
|
||||
submitted_at: '2023-06-15T12:00:00Z'
|
||||
}
|
||||
];
|
||||
|
||||
const filtered = githubClient.filterBotReviews(reviews, true);
|
||||
expect(filtered).toHaveLength(1);
|
||||
expect(filtered[0].user.login).toBe('johndoe');
|
||||
});
|
||||
|
||||
test('should not filter when excludeBots is false', () => {
|
||||
const botUser = {
|
||||
login: 'coderabbitai[bot]',
|
||||
type: 'Bot'
|
||||
};
|
||||
|
||||
const humanUser = {
|
||||
login: 'johndoe',
|
||||
type: 'User'
|
||||
};
|
||||
|
||||
const reviews = [
|
||||
{
|
||||
user: botUser,
|
||||
state: 'COMMENTED',
|
||||
submitted_at: '2023-06-15T12:00:00Z'
|
||||
},
|
||||
{
|
||||
user: humanUser,
|
||||
state: 'APPROVED',
|
||||
submitted_at: '2023-06-15T13:00:00Z'
|
||||
}
|
||||
];
|
||||
|
||||
const filtered = githubClient.filterBotReviews(reviews, false);
|
||||
expect(filtered).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('should handle mixed bot and human reviews', () => {
|
||||
const reviews = [
|
||||
{
|
||||
user: { login: 'coderabbitai[bot]', type: 'Bot' },
|
||||
state: 'COMMENTED',
|
||||
submitted_at: '2023-06-15T12:00:00Z'
|
||||
},
|
||||
{
|
||||
user: { login: 'johndoe', type: 'User' },
|
||||
state: 'APPROVED',
|
||||
submitted_at: '2023-06-15T13:00:00Z'
|
||||
},
|
||||
{
|
||||
user: { login: 'dependabot[bot]', type: 'User' },
|
||||
state: 'COMMENTED',
|
||||
submitted_at: '2023-06-15T14:00:00Z'
|
||||
},
|
||||
{
|
||||
user: { login: 'janedoe', type: 'User' },
|
||||
state: 'CHANGES_REQUESTED',
|
||||
submitted_at: '2023-06-15T15:00:00Z'
|
||||
}
|
||||
];
|
||||
|
||||
const filtered = githubClient.filterBotReviews(reviews, true);
|
||||
expect(filtered).toHaveLength(2);
|
||||
expect(filtered[0].user.login).toBe('johndoe');
|
||||
expect(filtered[1].user.login).toBe('janedoe');
|
||||
});
|
||||
|
||||
test('should log filtering activity', () => {
|
||||
const reviews = [
|
||||
{
|
||||
user: { login: 'coderabbitai[bot]', type: 'Bot' },
|
||||
state: 'COMMENTED',
|
||||
submitted_at: '2023-06-15T12:00:00Z'
|
||||
},
|
||||
{
|
||||
user: { login: 'johndoe', type: 'User' },
|
||||
state: 'APPROVED',
|
||||
submitted_at: '2023-06-15T13:00:00Z'
|
||||
}
|
||||
];
|
||||
|
||||
githubClient.filterBotReviews(reviews, true);
|
||||
|
||||
expect(mockLogger.info).toHaveBeenCalledWith(
|
||||
'Filtered out 1 bot reviews from 2 total reviews'
|
||||
);
|
||||
});
|
||||
|
||||
test('should handle empty review array', () => {
|
||||
const filtered = githubClient.filterBotReviews([], true);
|
||||
expect(filtered).toHaveLength(0);
|
||||
});
|
||||
|
||||
test('should handle reviews with no bots', () => {
|
||||
const reviews = [
|
||||
{
|
||||
user: { login: 'johndoe', type: 'User' },
|
||||
state: 'APPROVED',
|
||||
submitted_at: '2023-06-15T13:00:00Z'
|
||||
},
|
||||
{
|
||||
user: { login: 'janedoe', type: 'User' },
|
||||
state: 'CHANGES_REQUESTED',
|
||||
submitted_at: '2023-06-15T15:00:00Z'
|
||||
}
|
||||
];
|
||||
|
||||
const filtered = githubClient.filterBotReviews(reviews, true);
|
||||
expect(filtered).toHaveLength(2);
|
||||
expect(mockLogger.info).not.toHaveBeenCalledWith(
|
||||
expect.stringContaining('Filtered out')
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
421
.github/actions/eng-metrics/test/config.test.js
vendored
Normal file
421
.github/actions/eng-metrics/test/config.test.js
vendored
Normal file
|
|
@ -0,0 +1,421 @@
|
|||
/**
|
||||
* Tests for configuration module
|
||||
*/
|
||||
|
||||
import { jest } from "@jest/globals";
|
||||
|
||||
// Mock the logger
|
||||
const mockLogger = {
|
||||
default: {
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn(),
|
||||
},
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn(),
|
||||
};
|
||||
|
||||
// Mock fs
|
||||
const mockFs = {
|
||||
default: {
|
||||
existsSync: jest.fn(),
|
||||
readFileSync: jest.fn(),
|
||||
},
|
||||
existsSync: jest.fn(),
|
||||
readFileSync: jest.fn(),
|
||||
};
|
||||
|
||||
// Mock dotenv
|
||||
const mockDotenv = {
|
||||
default: {
|
||||
config: jest.fn(),
|
||||
},
|
||||
config: jest.fn(),
|
||||
};
|
||||
|
||||
// Mock modules before importing
|
||||
jest.unstable_mockModule("../src/logger.js", () => mockLogger);
|
||||
jest.unstable_mockModule("fs", () => mockFs);
|
||||
jest.unstable_mockModule("dotenv", () => mockDotenv);
|
||||
|
||||
// Now import the module under test
|
||||
const { loadConfig, validateConfig } = await import("../src/config.js");
|
||||
|
||||
describe("Config", () => {
|
||||
let originalEnv;
|
||||
|
||||
beforeEach(() => {
|
||||
// Save original environment
|
||||
originalEnv = { ...process.env };
|
||||
|
||||
// Clear environment variables. This is the recommended approach to prevent performance hit.
|
||||
Reflect.deleteProperty(process.env, "GITHUB_TOKEN");
|
||||
Reflect.deleteProperty(process.env, "SERVICE_ACCOUNT_KEY_PATH");
|
||||
Reflect.deleteProperty(process.env, "BIGQUERY_DATASET_ID");
|
||||
Reflect.deleteProperty(process.env, "REPOSITORIES");
|
||||
Reflect.deleteProperty(process.env, "LOOKBACK_DAYS");
|
||||
Reflect.deleteProperty(process.env, "TARGET_BRANCH");
|
||||
Reflect.deleteProperty(process.env, "PRINT_ONLY");
|
||||
Reflect.deleteProperty(process.env, "ENABLED_METRICS");
|
||||
Reflect.deleteProperty(process.env, "TIME_TO_FIRST_REVIEW_TABLE");
|
||||
Reflect.deleteProperty(process.env, "TIME_TO_MERGE_TABLE");
|
||||
|
||||
// Reset all mocks
|
||||
jest.clearAllMocks();
|
||||
|
||||
// Mock fs.existsSync to return true by default (for config.json and user group file)
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.default.existsSync.mockReturnValue(true);
|
||||
|
||||
// Mock fs.readFileSync to return a default config
|
||||
mockFs.readFileSync.mockReturnValue(
|
||||
JSON.stringify({
|
||||
repositories: ["owner/repo1", "owner/repo2"],
|
||||
targetBranch: "main",
|
||||
bigQueryDatasetId: "test_dataset",
|
||||
lookbackDays: 5,
|
||||
serviceAccountKeyPath: "./service-account-key.json",
|
||||
printOnly: false,
|
||||
userGroupEnabled: true,
|
||||
userGroupFilepath: "../../../handbook/company/product-groups.md",
|
||||
})
|
||||
);
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
// Restore original environment
|
||||
process.env = originalEnv;
|
||||
});
|
||||
|
||||
describe("loadConfig", () => {
|
||||
test("should load default configuration", () => {
|
||||
process.env.GITHUB_TOKEN = "test-token";
|
||||
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
|
||||
process.env.BIGQUERY_DATASET_ID = "test_dataset";
|
||||
process.env.REPOSITORIES = "owner/repo1,owner/repo2";
|
||||
|
||||
const config = loadConfig();
|
||||
|
||||
expect(config).toEqual({
|
||||
githubToken: "test-token",
|
||||
serviceAccountKeyPath: "/path/to/key.json",
|
||||
bigQueryDatasetId: "test_dataset",
|
||||
repositories: ["owner/repo1", "owner/repo2"],
|
||||
lookbackDays: 5,
|
||||
targetBranch: "main",
|
||||
printOnly: false,
|
||||
userGroupEnabled: true,
|
||||
userGroupFilepath: "../../../handbook/company/product-groups.md",
|
||||
excludeBotReviews: true,
|
||||
metrics: {
|
||||
timeToFirstReview: {
|
||||
enabled: true,
|
||||
tableName: "pr_first_review",
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: true,
|
||||
tableName: "pr_merge",
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test("should override defaults with environment variables", () => {
|
||||
process.env.GITHUB_TOKEN = "test-token";
|
||||
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
|
||||
process.env.BIGQUERY_DATASET_ID = "test_dataset";
|
||||
process.env.REPOSITORIES = "owner/repo";
|
||||
process.env.LOOKBACK_DAYS = "14";
|
||||
process.env.TARGET_BRANCH = "develop";
|
||||
process.env.PRINT_ONLY = "true";
|
||||
process.env.ENABLED_METRICS = "time_to_first_review";
|
||||
process.env.TIME_TO_FIRST_REVIEW_TABLE = "custom_first_review";
|
||||
process.env.TIME_TO_MERGE_TABLE = "custom_pr_merge";
|
||||
|
||||
const config = loadConfig();
|
||||
|
||||
expect(config).toEqual({
|
||||
githubToken: "test-token",
|
||||
serviceAccountKeyPath: "/path/to/key.json",
|
||||
bigQueryDatasetId: "test_dataset",
|
||||
repositories: ["owner/repo"],
|
||||
lookbackDays: 5,
|
||||
targetBranch: "develop",
|
||||
printOnly: true,
|
||||
userGroupEnabled: true,
|
||||
userGroupFilepath: "../../../handbook/company/product-groups.md",
|
||||
excludeBotReviews: true,
|
||||
metrics: {
|
||||
timeToFirstReview: {
|
||||
enabled: true,
|
||||
tableName: "custom_first_review",
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: false,
|
||||
tableName: "custom_pr_merge",
|
||||
},
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test("should handle multiple enabled metrics", () => {
|
||||
process.env.GITHUB_TOKEN = "test-token";
|
||||
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
|
||||
process.env.BIGQUERY_DATASET_ID = "test_dataset";
|
||||
process.env.REPOSITORIES = "owner/repo";
|
||||
process.env.ENABLED_METRICS = "time_to_first_review,time_to_merge";
|
||||
|
||||
const config = loadConfig();
|
||||
|
||||
expect(config.metrics).toEqual({
|
||||
timeToFirstReview: {
|
||||
enabled: true,
|
||||
tableName: "pr_first_review",
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: true,
|
||||
tableName: "pr_merge",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test("should handle only time_to_merge enabled", () => {
|
||||
process.env.GITHUB_TOKEN = "test-token";
|
||||
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
|
||||
process.env.BIGQUERY_DATASET_ID = "test_dataset";
|
||||
process.env.REPOSITORIES = "owner/repo";
|
||||
process.env.ENABLED_METRICS = "time_to_merge";
|
||||
|
||||
const config = loadConfig();
|
||||
|
||||
expect(config.metrics).toEqual({
|
||||
timeToFirstReview: {
|
||||
enabled: false,
|
||||
tableName: "pr_first_review",
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: true,
|
||||
tableName: "pr_merge",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
test("should trim whitespace from repositories", () => {
|
||||
process.env.GITHUB_TOKEN = "test-token";
|
||||
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
|
||||
process.env.BIGQUERY_DATASET_ID = "test_dataset";
|
||||
process.env.REPOSITORIES = " owner/repo1 , owner/repo2 , owner/repo3 ";
|
||||
|
||||
const config = loadConfig();
|
||||
|
||||
expect(config.repositories).toEqual([
|
||||
"owner/repo1",
|
||||
"owner/repo2",
|
||||
"owner/repo3",
|
||||
]);
|
||||
});
|
||||
|
||||
test("should trim whitespace from enabled metrics", () => {
|
||||
process.env.GITHUB_TOKEN = "test-token";
|
||||
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
|
||||
process.env.BIGQUERY_DATASET_ID = "test_dataset";
|
||||
process.env.REPOSITORIES = "owner/repo";
|
||||
process.env.ENABLED_METRICS = " time_to_first_review , time_to_merge ";
|
||||
|
||||
const config = loadConfig();
|
||||
|
||||
expect(config.metrics).toEqual({
|
||||
timeToFirstReview: {
|
||||
enabled: true,
|
||||
tableName: "pr_first_review",
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: true,
|
||||
tableName: "pr_merge",
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("validateConfig", () => {
|
||||
const baseValidConfig = {
|
||||
githubToken: "test-token",
|
||||
serviceAccountKeyPath: "/path/to/key.json",
|
||||
bigQueryDatasetId: "test_dataset",
|
||||
repositories: ["owner/repo"],
|
||||
lookbackDays: 30,
|
||||
targetBranch: "main",
|
||||
printOnly: false,
|
||||
userGroupEnabled: true,
|
||||
userGroupFilepath: "../../../handbook/company/product-groups.md",
|
||||
excludeBotReviews: true,
|
||||
metrics: {
|
||||
timeToFirstReview: {
|
||||
enabled: true,
|
||||
tableName: "pr_first_review",
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: true,
|
||||
tableName: "pr_merge",
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
test("should validate correct configuration", () => {
|
||||
expect(validateConfig(baseValidConfig)).toBe(true);
|
||||
});
|
||||
|
||||
test("should return false for missing GitHub token", () => {
|
||||
const config = { ...baseValidConfig, githubToken: "" };
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
|
||||
test("should return false for missing service account key path in non-print mode", () => {
|
||||
const config = { ...baseValidConfig, serviceAccountKeyPath: "" };
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
|
||||
test("should not require service account key path in print-only mode", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
serviceAccountKeyPath: "",
|
||||
printOnly: true,
|
||||
};
|
||||
expect(validateConfig(config)).toBe(true);
|
||||
});
|
||||
|
||||
test("should not require BigQuery dataset ID in print-only mode", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
bigQueryDatasetId: "",
|
||||
printOnly: true,
|
||||
};
|
||||
expect(validateConfig(config)).toBe(true);
|
||||
});
|
||||
|
||||
test("should return false for empty repositories array", () => {
|
||||
const config = { ...baseValidConfig, repositories: [] };
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
|
||||
test("should return false for invalid repository format", () => {
|
||||
const config = { ...baseValidConfig, repositories: ["invalid-repo"] };
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
|
||||
test("should validate lookback days correctly", () => {
|
||||
const config = { ...baseValidConfig, lookbackDays: 5 };
|
||||
expect(validateConfig(config)).toBe(true);
|
||||
});
|
||||
|
||||
test("should return false when no metrics are enabled", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
metrics: {
|
||||
timeToFirstReview: { enabled: false, tableName: "pr_first_review" },
|
||||
timeToMerge: { enabled: false, tableName: "pr_merge" },
|
||||
},
|
||||
};
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
|
||||
test("should return false for missing table name when metric is enabled", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
metrics: {
|
||||
timeToFirstReview: { enabled: true, tableName: "" },
|
||||
timeToMerge: { enabled: false, tableName: "pr_merge" },
|
||||
},
|
||||
};
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
|
||||
test("should allow missing table name when metric is disabled", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
metrics: {
|
||||
timeToFirstReview: { enabled: false, tableName: "" },
|
||||
timeToMerge: { enabled: true, tableName: "pr_merge" },
|
||||
},
|
||||
};
|
||||
expect(validateConfig(config)).toBe(true);
|
||||
});
|
||||
|
||||
test("should validate multiple valid repositories", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
repositories: ["owner1/repo1", "owner2/repo2", "owner3/repo3"],
|
||||
};
|
||||
expect(validateConfig(config)).toBe(true);
|
||||
});
|
||||
|
||||
test("should throw error for mixed valid and invalid repositories", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
repositories: ["owner1/repo1", "invalid-repo", "owner3/repo3"],
|
||||
};
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
|
||||
test("should validate when userGroupEnabled is true and file exists", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
userGroupEnabled: true,
|
||||
userGroupFilepath: "../../../handbook/company/product-groups.md",
|
||||
};
|
||||
|
||||
// Mock file exists - need to reset and set up the mock properly
|
||||
mockFs.existsSync.mockReset();
|
||||
mockFs.default.existsSync.mockReset();
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.default.existsSync.mockReturnValue(true);
|
||||
|
||||
expect(validateConfig(config)).toBe(true);
|
||||
});
|
||||
|
||||
test("should return false when userGroupEnabled is true but file does not exist", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
userGroupEnabled: true,
|
||||
userGroupFilepath: "../../../handbook/company/product-groups.md",
|
||||
};
|
||||
|
||||
// Mock file does not exist
|
||||
mockFs.existsSync.mockReset();
|
||||
mockFs.default.existsSync.mockReset();
|
||||
mockFs.existsSync.mockReturnValue(false);
|
||||
mockFs.default.existsSync.mockReturnValue(false);
|
||||
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
|
||||
test("should validate when userGroupEnabled is false regardless of file existence", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
userGroupEnabled: false,
|
||||
userGroupFilepath: "../../../handbook/company/product-groups.md",
|
||||
};
|
||||
|
||||
// Mock file does not exist
|
||||
mockFs.existsSync.mockReset();
|
||||
mockFs.default.existsSync.mockReset();
|
||||
mockFs.existsSync.mockReturnValue(false);
|
||||
mockFs.default.existsSync.mockReturnValue(false);
|
||||
|
||||
expect(validateConfig(config)).toBe(true);
|
||||
});
|
||||
|
||||
test("should return false when userGroupEnabled is true but userGroupFilepath is not set", () => {
|
||||
const config = {
|
||||
...baseValidConfig,
|
||||
userGroupEnabled: true,
|
||||
userGroupFilepath: undefined,
|
||||
};
|
||||
|
||||
expect(validateConfig(config)).toBe(false);
|
||||
});
|
||||
});
|
||||
});
|
||||
232
.github/actions/eng-metrics/test/end-to-end.test.js
vendored
Normal file
232
.github/actions/eng-metrics/test/end-to-end.test.js
vendored
Normal file
|
|
@ -0,0 +1,232 @@
|
|||
/**
|
||||
* Tests the complete workflow from configuration to metric collection
|
||||
*/
|
||||
|
||||
import { jest } from '@jest/globals';
|
||||
import { loadConfig } from '../src/config.js';
|
||||
import { MetricsCollector } from '../src/metrics-collector.js';
|
||||
|
||||
// Mock the logger
|
||||
jest.mock('../src/logger.js', () => ({
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn()
|
||||
}));
|
||||
|
||||
// Mock GitHubClient
|
||||
jest.mock('../src/github-client.js', () => {
|
||||
return jest.fn().mockImplementation(() => ({
|
||||
fetchPullRequests: jest.fn(),
|
||||
fetchPRTimelineEvents: jest.fn(),
|
||||
fetchPRReviewEvents: jest.fn(),
|
||||
filterBotReviews: jest.fn((reviews, excludeBots) => excludeBots ? [] : reviews),
|
||||
calculatePickupTime: jest.fn(),
|
||||
calculateTimeToMerge: jest.fn()
|
||||
}));
|
||||
});
|
||||
|
||||
// Mock BigQueryClient
|
||||
jest.mock('../src/bigquery-client.js', () => ({
|
||||
BigQueryClient: jest.fn().mockImplementation(() => ({
|
||||
uploadMetrics: jest.fn()
|
||||
}))
|
||||
}));
|
||||
|
||||
describe('End-to-End Time to Merge Workflow', () => {
|
||||
beforeEach(() => {
|
||||
// Set environment variables for testing
|
||||
process.env.GITHUB_TOKEN = 'test-token';
|
||||
process.env.SERVICE_ACCOUNT_KEY_PATH = '/fake/path';
|
||||
process.env.BIGQUERY_DATASET_ID = 'test_dataset';
|
||||
process.env.REPOSITORIES = 'owner/repo';
|
||||
process.env.PRINT_ONLY = 'true';
|
||||
});
|
||||
|
||||
test('should collect both Time to First Review and Time to Merge metrics', async () => {
|
||||
// Load configuration
|
||||
const config = loadConfig();
|
||||
|
||||
// Verify both metrics are enabled
|
||||
expect(config.metrics.timeToFirstReview.enabled).toBe(true);
|
||||
expect(config.metrics.timeToMerge.enabled).toBe(true);
|
||||
|
||||
// Create metrics collector
|
||||
const metricsCollector = new MetricsCollector(config);
|
||||
|
||||
// Mock the GitHub client methods
|
||||
metricsCollector.githubClient = {
|
||||
fetchPullRequests: jest.fn().mockResolvedValue([
|
||||
{
|
||||
number: 123,
|
||||
html_url: 'https://github.com/owner/repo/pull/123',
|
||||
user: { login: 'testuser' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
owner: { login: 'owner' },
|
||||
name: 'repo'
|
||||
}
|
||||
},
|
||||
head: { repo: { full_name: 'owner/repo' } },
|
||||
state: 'closed',
|
||||
merged_at: '2023-06-15T14:30:00Z'
|
||||
}
|
||||
]),
|
||||
fetchPRTimelineEvents: jest.fn().mockResolvedValue([
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-06-15T10:00:00Z'
|
||||
}
|
||||
]),
|
||||
fetchPRReviewEvents: jest.fn().mockResolvedValue([
|
||||
{
|
||||
submitted_at: '2023-06-15T12:00:00Z',
|
||||
state: 'approved',
|
||||
user: { login: 'reviewer1' }
|
||||
}
|
||||
]),
|
||||
calculatePickupTime: jest.fn().mockReturnValue({
|
||||
metricType: 'time_to_first_review',
|
||||
reviewDate: '2023-06-15',
|
||||
prCreator: 'testuser',
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
pickupTimeSeconds: 7200,
|
||||
repository: 'owner/repo',
|
||||
prNumber: 123,
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-06-15T10:00:00Z'),
|
||||
firstReviewTime: new Date('2023-06-15T12:00:00Z')
|
||||
}),
|
||||
calculateTimeToMerge: jest.fn().mockReturnValue({
|
||||
metricType: 'time_to_merge',
|
||||
mergeDate: '2023-06-15',
|
||||
prCreator: 'testuser',
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
mergeTimeSeconds: 16200,
|
||||
repository: 'owner/repo',
|
||||
prNumber: 123,
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-06-15T10:00:00Z'),
|
||||
mergeTime: new Date('2023-06-15T14:30:00Z')
|
||||
}),
|
||||
filterBotReviews: jest.fn((reviews, excludeBots) => excludeBots ? [] : reviews)
|
||||
};
|
||||
|
||||
// Collect metrics for a single repository
|
||||
const metrics = await metricsCollector.collectRepositoryMetrics('owner/repo');
|
||||
|
||||
// Verify that both metrics were collected
|
||||
expect(metrics).toHaveLength(2);
|
||||
|
||||
const firstReviewMetric = metrics.find(m => m.metricType === 'time_to_first_review');
|
||||
const mergeMetric = metrics.find(m => m.metricType === 'time_to_merge');
|
||||
|
||||
expect(firstReviewMetric).toBeDefined();
|
||||
expect(firstReviewMetric.pickupTimeSeconds).toBe(7200);
|
||||
expect(firstReviewMetric.prNumber).toBe(123);
|
||||
|
||||
expect(mergeMetric).toBeDefined();
|
||||
expect(mergeMetric.mergeTimeSeconds).toBe(16200);
|
||||
expect(mergeMetric.prNumber).toBe(123);
|
||||
|
||||
// Verify that both calculation methods were called
|
||||
expect(metricsCollector.githubClient.calculatePickupTime).toHaveBeenCalled();
|
||||
expect(metricsCollector.githubClient.calculateTimeToMerge).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('should handle configuration with only Time to Merge enabled', async () => {
|
||||
// Override environment to enable only Time to Merge
|
||||
process.env.ENABLED_METRICS = 'time_to_merge';
|
||||
|
||||
const config = loadConfig();
|
||||
|
||||
expect(config.metrics.timeToFirstReview.enabled).toBe(false);
|
||||
expect(config.metrics.timeToMerge.enabled).toBe(true);
|
||||
|
||||
const metricsCollector = new MetricsCollector(config);
|
||||
|
||||
// Mock the GitHub client
|
||||
metricsCollector.githubClient = {
|
||||
calculateTimeToMerge: jest.fn().mockReturnValue({
|
||||
metricType: 'time_to_merge',
|
||||
mergeDate: '2023-06-15',
|
||||
prCreator: 'testuser',
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
mergeTimeSeconds: 16200,
|
||||
repository: 'owner/repo',
|
||||
prNumber: 123,
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-06-15T10:00:00Z'),
|
||||
mergeTime: new Date('2023-06-15T14:30:00Z')
|
||||
}),
|
||||
calculatePickupTime: jest.fn().mockReturnValue(null) // Should not be called
|
||||
};
|
||||
|
||||
// Mock PR data
|
||||
const mockPR = {
|
||||
number: 123,
|
||||
html_url: 'https://github.com/owner/repo/pull/123',
|
||||
user: { login: 'testuser' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
owner: { login: 'owner' },
|
||||
name: 'repo'
|
||||
}
|
||||
},
|
||||
head: { repo: { full_name: 'owner/repo' } },
|
||||
state: 'closed',
|
||||
merged_at: '2023-06-15T14:30:00Z'
|
||||
};
|
||||
|
||||
const mockTimelineEvents = [
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-06-15T10:00:00Z'
|
||||
}
|
||||
];
|
||||
|
||||
const mockReviewEvents = [];
|
||||
|
||||
// Test collectPRMetrics with only merge enabled
|
||||
const metrics = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
|
||||
|
||||
// Should only collect Time to Merge metric
|
||||
expect(metrics).toHaveLength(1);
|
||||
expect(metrics[0].metricType).toBe('time_to_merge');
|
||||
});
|
||||
|
||||
test('should group and route metrics to correct tables', () => {
|
||||
const config = loadConfig();
|
||||
const metricsCollector = new MetricsCollector(config);
|
||||
|
||||
const mixedMetrics = [
|
||||
{
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 123,
|
||||
pickupTimeSeconds: 7200
|
||||
},
|
||||
{
|
||||
metricType: 'time_to_merge',
|
||||
prNumber: 123,
|
||||
mergeTimeSeconds: 16200
|
||||
},
|
||||
{
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 124,
|
||||
pickupTimeSeconds: 3600
|
||||
}
|
||||
];
|
||||
|
||||
// Test grouping
|
||||
const grouped = metricsCollector.groupMetricsByType(mixedMetrics);
|
||||
|
||||
expect(grouped.time_to_first_review).toHaveLength(2);
|
||||
expect(grouped.time_to_merge).toHaveLength(1);
|
||||
|
||||
// Test table name mapping
|
||||
expect(metricsCollector.getTableNameForMetricType('time_to_first_review')).toBe('pr_first_review');
|
||||
expect(metricsCollector.getTableNameForMetricType('time_to_merge')).toBe('pr_merge');
|
||||
});
|
||||
});
|
||||
547
.github/actions/eng-metrics/test/github-client-first-review.test.js
vendored
Normal file
547
.github/actions/eng-metrics/test/github-client-first-review.test.js
vendored
Normal file
|
|
@ -0,0 +1,547 @@
|
|||
import { jest } from '@jest/globals';
|
||||
import GitHubClient from '../src/github-client.js';
|
||||
|
||||
// Mock the logger to avoid console output during tests
|
||||
jest.mock('../src/logger.js', () => ({
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn(),
|
||||
default: {
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn(),
|
||||
}
|
||||
}));
|
||||
|
||||
describe('GitHubClient - Time to first review (pickup time)', () => {
|
||||
let githubClient;
|
||||
|
||||
beforeEach(() => {
|
||||
// Create a new instance of GitHubClient for each test
|
||||
githubClient = new GitHubClient('fake-token');
|
||||
|
||||
// Mock the Octokit instance
|
||||
githubClient.octokit = {
|
||||
rest: {
|
||||
pulls: {
|
||||
list: jest.fn(),
|
||||
listReviews: jest.fn(),
|
||||
},
|
||||
issues: {
|
||||
listEventsForTimeline: jest.fn(),
|
||||
}
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
describe('calculatePickupTime', () => {
|
||||
// Table-driven test cases for calculatePickupTime
|
||||
const testCases = [
|
||||
{
|
||||
name: 'PR created as non-draft with one review',
|
||||
pr: {
|
||||
number: 123,
|
||||
html_url: 'https://github.com/owner/repo/pull/123',
|
||||
draft: false,
|
||||
created_at: '2023-05-10T10:00:00Z',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-10T11:30:00Z' }
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 123,
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-10T10:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-10T11:30:00Z'),
|
||||
reviewDate: '2023-05-10',
|
||||
pickupTimeSeconds: 5400, // 1.5 hours = 5400 seconds
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR created as draft, then marked as ready for review, then reviewed',
|
||||
pr: {
|
||||
number: 124,
|
||||
html_url: 'https://github.com/owner/repo/pull/124',
|
||||
draft: true,
|
||||
created_at: '2023-05-11T09:00:00Z',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-05-11T10:00:00Z'
|
||||
}
|
||||
],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-11T11:00:00Z' }
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 124,
|
||||
prUrl: 'https://github.com/owner/repo/pull/124',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-11T10:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-11T11:00:00Z'),
|
||||
reviewDate: '2023-05-11',
|
||||
pickupTimeSeconds: 3600, // 1 hour = 3600 seconds
|
||||
readyEventType: 'ready_for_review event'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR with multiple ready_for_review events - should use the latest one',
|
||||
pr: {
|
||||
number: 125,
|
||||
html_url: 'https://github.com/owner/repo/pull/125',
|
||||
draft: true,
|
||||
created_at: '2023-05-12T09:00:00Z',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-05-12T10:00:00Z'
|
||||
},
|
||||
{
|
||||
event: 'convert_to_draft',
|
||||
created_at: '2023-05-12T11:00:00Z'
|
||||
},
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-05-12T12:00:00Z'
|
||||
}
|
||||
],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-12T13:00:00Z' }
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 125,
|
||||
prUrl: 'https://github.com/owner/repo/pull/125',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-12T12:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-12T13:00:00Z'),
|
||||
reviewDate: '2023-05-12',
|
||||
pickupTimeSeconds: 3600, // 1 hour = 3600 seconds
|
||||
readyEventType: 'ready_for_review event'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR with ready_for_review event after the first review',
|
||||
pr: {
|
||||
number: 126,
|
||||
html_url: 'https://github.com/owner/repo/pull/126',
|
||||
draft: false,
|
||||
created_at: '2023-05-16T09:00:00Z',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [
|
||||
{
|
||||
event: 'convert_to_draft',
|
||||
created_at: '2023-05-16T10:00:00Z'
|
||||
},
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-05-16T12:00:00Z'
|
||||
}
|
||||
],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-16T11:00:00Z' }
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 126,
|
||||
prUrl: 'https://github.com/owner/repo/pull/126',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-16T09:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-16T11:00:00Z'),
|
||||
reviewDate: '2023-05-16',
|
||||
pickupTimeSeconds: 7200, // 2 hours = 7200 seconds
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR with no ready_for_review events and created as draft',
|
||||
pr: {
|
||||
number: 127,
|
||||
html_url: 'https://github.com/owner/repo/pull/127',
|
||||
draft: true,
|
||||
created_at: '2023-05-14T09:00:00Z',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-14T11:00:00Z' }
|
||||
],
|
||||
expected: null // Should return null because no ready event was found
|
||||
},
|
||||
{
|
||||
name: 'PR with no reviews',
|
||||
pr: {
|
||||
number: 128,
|
||||
html_url: 'https://github.com/owner/repo/pull/128',
|
||||
draft: false,
|
||||
created_at: '2023-05-15T09:00:00Z',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: null // Should return null because no reviews were found
|
||||
},
|
||||
{
|
||||
name: 'PR with multiple reviews - only first one should be counted',
|
||||
pr: {
|
||||
number: 129,
|
||||
html_url: 'https://github.com/owner/repo/pull/129',
|
||||
draft: false,
|
||||
created_at: '2023-05-16T09:00:00Z',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-16T10:00:00Z' }, // First review - should be used
|
||||
{ submitted_at: '2023-05-16T11:00:00Z' }, // Second review - should be ignored
|
||||
{ submitted_at: '2023-05-16T12:00:00Z' } // Third review - should be ignored
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 129,
|
||||
prUrl: 'https://github.com/owner/repo/pull/129',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-16T09:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-16T10:00:00Z'),
|
||||
reviewDate: '2023-05-16',
|
||||
pickupTimeSeconds: 3600, // 1 hour = 3600 seconds
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Saturday, reviewed on Sunday 3 weeks later (should exclude weekend days)',
|
||||
pr: {
|
||||
number: 136,
|
||||
html_url: 'https://github.com/owner/repo/pull/136',
|
||||
draft: false,
|
||||
created_at: '2023-05-20T14:00:00Z', // Saturday
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-06-11T14:00:00Z' } // Sunday, 3 weeks later
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 136,
|
||||
prUrl: 'https://github.com/owner/repo/pull/136',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-20T14:00:00Z'),
|
||||
firstReviewTime: new Date('2023-06-11T14:00:00Z'),
|
||||
reviewDate: '2023-06-11',
|
||||
pickupTimeSeconds: 1296000, // 15 days = 1296000 seconds (3 weeks of 5 working days)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Sunday, reviewed on Monday (should use end of Sunday as ready time)',
|
||||
pr: {
|
||||
number: 134,
|
||||
html_url: 'https://github.com/owner/repo/pull/134',
|
||||
draft: false,
|
||||
created_at: '2023-05-21T14:00:00Z', // Sunday
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-22T10:00:00Z' } // Monday
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 134,
|
||||
prUrl: 'https://github.com/owner/repo/pull/134',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-21T14:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-22T10:00:00Z'),
|
||||
reviewDate: '2023-05-22',
|
||||
pickupTimeSeconds: 36000, // 10 hours = 36000 seconds (from end of Sunday to Monday 10am)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Sunday, reviewed on next Saturday (should exclude weekend days)',
|
||||
pr: {
|
||||
number: 135,
|
||||
html_url: 'https://github.com/owner/repo/pull/135',
|
||||
draft: false,
|
||||
created_at: '2023-05-21T14:00:00Z', // Sunday
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-27T14:00:00Z' } // Saturday
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 135,
|
||||
prUrl: 'https://github.com/owner/repo/pull/135',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-21T14:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-27T14:00:00Z'),
|
||||
reviewDate: '2023-05-27',
|
||||
pickupTimeSeconds: 432000, // 5 days = 432000 seconds (6 days - 1 weekend day)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Friday, reviewed on Monday (should subtract weekend days)',
|
||||
pr: {
|
||||
number: 130,
|
||||
html_url: 'https://github.com/owner/repo/pull/130',
|
||||
draft: false,
|
||||
created_at: '2023-05-19T14:00:00Z', // Friday
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-22T10:00:00Z' } // Monday
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 130,
|
||||
prUrl: 'https://github.com/owner/repo/pull/130',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-19T14:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-22T10:00:00Z'),
|
||||
reviewDate: '2023-05-22',
|
||||
pickupTimeSeconds: 72000, // 20 hours = 72000 seconds (3 days - 2 weekend days)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Saturday, reviewed on Monday (should use end of Sunday as ready time)',
|
||||
pr: {
|
||||
number: 131,
|
||||
html_url: 'https://github.com/owner/repo/pull/131',
|
||||
draft: false,
|
||||
created_at: '2023-05-20T14:00:00Z', // Saturday
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-22T10:00:00Z' } // Monday
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 131,
|
||||
prUrl: 'https://github.com/owner/repo/pull/131',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-20T14:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-22T10:00:00Z'),
|
||||
reviewDate: '2023-05-22',
|
||||
pickupTimeSeconds: 36000, // 10 hours = 36000 seconds (from end of Sunday to Monday 10am)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Saturday, reviewed on Sunday (should have 0 pickup time)',
|
||||
pr: {
|
||||
number: 132,
|
||||
html_url: 'https://github.com/owner/repo/pull/132',
|
||||
draft: false,
|
||||
created_at: '2023-05-20T14:00:00Z', // Saturday
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-21T14:00:00Z' } // Sunday
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 132,
|
||||
prUrl: 'https://github.com/owner/repo/pull/132',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-20T14:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-21T14:00:00Z'),
|
||||
reviewDate: '2023-05-21',
|
||||
pickupTimeSeconds: 0, // 0 seconds (both on weekend)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on weekday, reviewed after multiple weekends (should subtract weekend days)',
|
||||
pr: {
|
||||
number: 133,
|
||||
html_url: 'https://github.com/owner/repo/pull/133',
|
||||
draft: false,
|
||||
created_at: '2023-05-17T14:00:00Z', // Wednesday
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-29T14:00:00Z' } // Monday, 12 days later
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_first_review',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 133,
|
||||
prUrl: 'https://github.com/owner/repo/pull/133',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-17T14:00:00Z'),
|
||||
firstReviewTime: new Date('2023-05-29T14:00:00Z'),
|
||||
reviewDate: '2023-05-29',
|
||||
pickupTimeSeconds: 691200, // 8 days = 691200 seconds (12 days - 4 weekend days)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
}
|
||||
];
|
||||
|
||||
// Run each test case
|
||||
test.each(testCases)('$name', ({ pr, timelineEvents, reviewEvents, expected }) => {
|
||||
const result = githubClient.calculatePickupTime(pr, timelineEvents, reviewEvents);
|
||||
|
||||
if (expected === null) {
|
||||
expect(result).toBeNull();
|
||||
} else {
|
||||
// Compare date objects separately
|
||||
expect(result.readyTime).toEqual(expected.readyTime);
|
||||
expect(result.firstReviewTime).toEqual(expected.firstReviewTime);
|
||||
|
||||
// Compare the rest of the properties
|
||||
expect({
|
||||
...result,
|
||||
readyTime: undefined,
|
||||
firstReviewTime: undefined
|
||||
}).toEqual({
|
||||
...expected,
|
||||
readyTime: undefined,
|
||||
firstReviewTime: undefined
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
136
.github/actions/eng-metrics/test/github-client-general.test.js
vendored
Normal file
136
.github/actions/eng-metrics/test/github-client-general.test.js
vendored
Normal file
|
|
@ -0,0 +1,136 @@
|
|||
/**
|
||||
* General tests for GitHub client functionality
|
||||
*/
|
||||
|
||||
import { jest } from '@jest/globals';
|
||||
import GitHubClient from '../src/github-client.js';
|
||||
|
||||
// Mock the logger
|
||||
jest.mock('../src/logger.js', () => ({
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn()
|
||||
}));
|
||||
|
||||
describe('GitHubClient - General Functionality', () => {
|
||||
let githubClient;
|
||||
let mockOctokit;
|
||||
|
||||
beforeEach(() => {
|
||||
// Mock Octokit
|
||||
mockOctokit = {
|
||||
rest: {
|
||||
pulls: {
|
||||
list: jest.fn(),
|
||||
listReviews: jest.fn()
|
||||
},
|
||||
issues: {
|
||||
listEventsForTimeline: jest.fn()
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
githubClient = new GitHubClient('fake-token');
|
||||
githubClient.octokit = mockOctokit;
|
||||
});
|
||||
|
||||
describe('constructor', () => {
|
||||
test('should initialize with token', () => {
|
||||
const client = new GitHubClient('test-token');
|
||||
expect(client).toBeInstanceOf(GitHubClient);
|
||||
});
|
||||
|
||||
test('should throw error without token', () => {
|
||||
expect(() => new GitHubClient()).toThrow('GitHub token is required');
|
||||
});
|
||||
});
|
||||
|
||||
describe('fetchPullRequests', () => {
|
||||
test('should fetch pull requests successfully', async () => {
|
||||
const mockResponse = {
|
||||
data: [
|
||||
{
|
||||
number: 1,
|
||||
title: 'Test PR',
|
||||
state: 'closed',
|
||||
merged_at: '2023-06-15T14:30:00Z',
|
||||
updated_at: '2023-06-15T14:30:00Z',
|
||||
base: { ref: 'main' }
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
mockOctokit.rest.pulls.list.mockResolvedValue(mockResponse);
|
||||
|
||||
const result = await githubClient.fetchPullRequests('owner', 'repo', 'all', new Date('2023-01-01'));
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].number).toBe(1);
|
||||
});
|
||||
|
||||
test('should handle API errors gracefully', async () => {
|
||||
const mockError = new Error('API Error');
|
||||
mockOctokit.rest.pulls.list.mockRejectedValue(mockError);
|
||||
|
||||
await expect(githubClient.fetchPullRequests('owner', 'repo', 'all', new Date('2023-01-01')))
|
||||
.rejects.toThrow('API Error');
|
||||
});
|
||||
});
|
||||
|
||||
describe('fetchPRTimelineEvents', () => {
|
||||
test('should fetch timeline events successfully', async () => {
|
||||
const mockResponse = {
|
||||
data: [
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-06-15T10:00:00Z'
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
mockOctokit.rest.issues.listEventsForTimeline.mockResolvedValue(mockResponse);
|
||||
|
||||
const result = await githubClient.fetchPRTimelineEvents('owner', 'repo', 123);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].event).toBe('ready_for_review');
|
||||
});
|
||||
|
||||
test('should handle API errors gracefully', async () => {
|
||||
const mockError = new Error('Timeline API Error');
|
||||
mockOctokit.rest.issues.listEventsForTimeline.mockRejectedValue(mockError);
|
||||
|
||||
await expect(githubClient.fetchPRTimelineEvents('owner', 'repo', 123))
|
||||
.rejects.toThrow('Timeline API Error');
|
||||
});
|
||||
});
|
||||
|
||||
describe('fetchPRReviewEvents', () => {
|
||||
test('should fetch reviews successfully', async () => {
|
||||
const mockResponse = {
|
||||
data: [
|
||||
{
|
||||
id: 1,
|
||||
state: 'APPROVED',
|
||||
submitted_at: '2023-06-15T12:00:00Z',
|
||||
user: { login: 'reviewer1' }
|
||||
}
|
||||
]
|
||||
};
|
||||
|
||||
mockOctokit.rest.pulls.listReviews.mockResolvedValue(mockResponse);
|
||||
|
||||
const result = await githubClient.fetchPRReviewEvents('owner', 'repo', 123);
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].state).toBe('APPROVED');
|
||||
});
|
||||
|
||||
test('should handle API errors gracefully', async () => {
|
||||
const mockError = new Error('Reviews API Error');
|
||||
mockOctokit.rest.pulls.listReviews.mockRejectedValue(mockError);
|
||||
|
||||
await expect(githubClient.fetchPRReviewEvents('owner', 'repo', 123))
|
||||
.rejects.toThrow('Reviews API Error');
|
||||
});
|
||||
});
|
||||
|
||||
});
|
||||
547
.github/actions/eng-metrics/test/github-client-merge.test.js
vendored
Normal file
547
.github/actions/eng-metrics/test/github-client-merge.test.js
vendored
Normal file
|
|
@ -0,0 +1,547 @@
|
|||
import { jest } from '@jest/globals';
|
||||
import GitHubClient from '../src/github-client.js';
|
||||
|
||||
// Mock the logger to avoid console output during tests
|
||||
jest.mock('../src/logger.js', () => ({
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn(),
|
||||
default: {
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn(),
|
||||
}
|
||||
}));
|
||||
|
||||
describe('GitHubClient - Time to Merge', () => {
|
||||
let githubClient;
|
||||
|
||||
beforeEach(() => {
|
||||
// Create a new instance of GitHubClient for each test
|
||||
githubClient = new GitHubClient('fake-token');
|
||||
|
||||
// Mock the Octokit instance
|
||||
githubClient.octokit = {
|
||||
rest: {
|
||||
pulls: {
|
||||
list: jest.fn(),
|
||||
listReviews: jest.fn(),
|
||||
},
|
||||
issues: {
|
||||
listEventsForTimeline: jest.fn(),
|
||||
}
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
describe('calculateTimeToMerge', () => {
|
||||
// Table-driven test cases for calculateTimeToMerge
|
||||
const testCases = [
|
||||
{
|
||||
name: 'PR created as non-draft and merged',
|
||||
pr: {
|
||||
number: 123,
|
||||
html_url: 'https://github.com/owner/repo/pull/123',
|
||||
draft: false,
|
||||
created_at: '2023-05-10T10:00:00Z',
|
||||
merged_at: '2023-05-10T11:30:00Z',
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-10T11:30:00Z' }
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 123,
|
||||
prUrl: 'https://github.com/owner/repo/pull/123',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-10T10:00:00Z'),
|
||||
mergeTime: new Date('2023-05-10T11:30:00Z'),
|
||||
mergeDate: '2023-05-10',
|
||||
mergeTimeSeconds: 5400, // 1.5 hours = 5400 seconds
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR created as draft, then marked as ready for review, then merged',
|
||||
pr: {
|
||||
number: 124,
|
||||
html_url: 'https://github.com/owner/repo/pull/124',
|
||||
draft: true,
|
||||
created_at: '2023-05-11T09:00:00Z',
|
||||
merged_at: '2023-05-11T11:00:00Z',
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-05-11T10:00:00Z'
|
||||
}
|
||||
],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-11T11:00:00Z' }
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 124,
|
||||
prUrl: 'https://github.com/owner/repo/pull/124',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-11T10:00:00Z'),
|
||||
mergeTime: new Date('2023-05-11T11:00:00Z'),
|
||||
mergeDate: '2023-05-11',
|
||||
mergeTimeSeconds: 3600, // 1 hour = 3600 seconds
|
||||
readyEventType: 'ready_for_review event'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR with multiple ready_for_review events - should use the latest one',
|
||||
pr: {
|
||||
number: 125,
|
||||
html_url: 'https://github.com/owner/repo/pull/125',
|
||||
draft: true,
|
||||
created_at: '2023-05-12T09:00:00Z',
|
||||
merged_at: '2023-05-12T13:00:00Z',
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-05-12T10:00:00Z'
|
||||
},
|
||||
{
|
||||
event: 'convert_to_draft',
|
||||
created_at: '2023-05-12T11:00:00Z'
|
||||
},
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-05-12T12:00:00Z'
|
||||
}
|
||||
],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-12T13:00:00Z' }
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 125,
|
||||
prUrl: 'https://github.com/owner/repo/pull/125',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-12T12:00:00Z'),
|
||||
mergeTime: new Date('2023-05-12T13:00:00Z'),
|
||||
mergeDate: '2023-05-12',
|
||||
mergeTimeSeconds: 3600, // 1 hour = 3600 seconds
|
||||
readyEventType: 'ready_for_review event'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR with ready_for_review event after merge time - should use PR creation',
|
||||
pr: {
|
||||
number: 126,
|
||||
html_url: 'https://github.com/owner/repo/pull/126',
|
||||
draft: false,
|
||||
created_at: '2023-05-16T09:00:00Z',
|
||||
merged_at: '2023-05-16T11:00:00Z',
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [
|
||||
{
|
||||
event: 'convert_to_draft',
|
||||
created_at: '2023-05-16T10:00:00Z'
|
||||
},
|
||||
{
|
||||
event: 'ready_for_review',
|
||||
created_at: '2023-05-16T12:00:00Z' // After merge
|
||||
}
|
||||
],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-16T11:00:00Z' }
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 126,
|
||||
prUrl: 'https://github.com/owner/repo/pull/126',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-16T09:00:00Z'),
|
||||
mergeTime: new Date('2023-05-16T11:00:00Z'),
|
||||
mergeDate: '2023-05-16',
|
||||
mergeTimeSeconds: 7200, // 2 hours = 7200 seconds
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR with no ready_for_review events and created as draft',
|
||||
pr: {
|
||||
number: 127,
|
||||
html_url: 'https://github.com/owner/repo/pull/127',
|
||||
draft: true,
|
||||
created_at: '2023-05-14T09:00:00Z',
|
||||
merged_at: '2023-05-14T11:00:00Z',
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-05-14T11:00:00Z' }
|
||||
],
|
||||
expected: null // Should return null because no ready event was found
|
||||
},
|
||||
{
|
||||
name: 'PR that was never merged',
|
||||
pr: {
|
||||
number: 128,
|
||||
html_url: 'https://github.com/owner/repo/pull/128',
|
||||
draft: false,
|
||||
created_at: '2023-05-15T09:00:00Z',
|
||||
merged_at: null,
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: null // Should return null because PR was not merged
|
||||
},
|
||||
{
|
||||
name: 'PR that is still open',
|
||||
pr: {
|
||||
number: 129,
|
||||
html_url: 'https://github.com/owner/repo/pull/129',
|
||||
draft: false,
|
||||
created_at: '2023-05-16T09:00:00Z',
|
||||
merged_at: null,
|
||||
state: 'open',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: null // Should return null because PR is not merged
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Saturday, merged on Sunday 3 weeks later (should exclude weekend days)',
|
||||
pr: {
|
||||
number: 136,
|
||||
html_url: 'https://github.com/owner/repo/pull/136',
|
||||
draft: false,
|
||||
created_at: '2023-05-20T14:00:00Z', // Saturday
|
||||
merged_at: '2023-06-11T14:00:00Z', // Sunday, 3 weeks later
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [
|
||||
{ submitted_at: '2023-06-11T13:00:00Z' } // Sunday, 3 weeks later
|
||||
],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 136,
|
||||
prUrl: 'https://github.com/owner/repo/pull/136',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-20T14:00:00Z'),
|
||||
mergeTime: new Date('2023-06-11T14:00:00Z'),
|
||||
mergeDate: '2023-06-11',
|
||||
mergeTimeSeconds: 1296000, // 15 days = 1296000 seconds (3 weeks of 5 working days)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Sunday, merged on Monday (should use end of Sunday as ready time)',
|
||||
pr: {
|
||||
number: 134,
|
||||
html_url: 'https://github.com/owner/repo/pull/134',
|
||||
draft: false,
|
||||
created_at: '2023-05-21T14:00:00Z', // Sunday
|
||||
merged_at: '2023-05-22T10:00:00Z', // Monday
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 134,
|
||||
prUrl: 'https://github.com/owner/repo/pull/134',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-21T14:00:00Z'),
|
||||
mergeTime: new Date('2023-05-22T10:00:00Z'),
|
||||
mergeDate: '2023-05-22',
|
||||
mergeTimeSeconds: 36000, // 10 hours = 36000 seconds (from end of Sunday to Monday 10am)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Sunday, merged on next Saturday (should exclude weekend days)',
|
||||
pr: {
|
||||
number: 135,
|
||||
html_url: 'https://github.com/owner/repo/pull/135',
|
||||
draft: false,
|
||||
created_at: '2023-05-21T14:00:00Z', // Sunday
|
||||
merged_at: '2023-05-27T14:00:00Z', // Saturday
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 135,
|
||||
prUrl: 'https://github.com/owner/repo/pull/135',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-21T14:00:00Z'),
|
||||
mergeTime: new Date('2023-05-27T14:00:00Z'),
|
||||
mergeDate: '2023-05-27',
|
||||
mergeTimeSeconds: 432000, // 5 days = 432000 seconds (6 days - 1 weekend day)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Friday, merged on Monday (should subtract weekend days)',
|
||||
pr: {
|
||||
number: 130,
|
||||
html_url: 'https://github.com/owner/repo/pull/130',
|
||||
draft: false,
|
||||
created_at: '2023-05-19T14:00:00Z', // Friday
|
||||
merged_at: '2023-05-22T10:00:00Z', // Monday
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 130,
|
||||
prUrl: 'https://github.com/owner/repo/pull/130',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-19T14:00:00Z'),
|
||||
mergeTime: new Date('2023-05-22T10:00:00Z'),
|
||||
mergeDate: '2023-05-22',
|
||||
mergeTimeSeconds: 72000, // 20 hours = 72000 seconds (3 days - 2 weekend days)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Saturday, merged on Monday (should use end of Sunday as ready time)',
|
||||
pr: {
|
||||
number: 131,
|
||||
html_url: 'https://github.com/owner/repo/pull/131',
|
||||
draft: false,
|
||||
created_at: '2023-05-20T14:00:00Z', // Saturday
|
||||
merged_at: '2023-05-22T10:00:00Z', // Monday
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 131,
|
||||
prUrl: 'https://github.com/owner/repo/pull/131',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-20T14:00:00Z'),
|
||||
mergeTime: new Date('2023-05-22T10:00:00Z'),
|
||||
mergeDate: '2023-05-22',
|
||||
mergeTimeSeconds: 36000, // 10 hours = 36000 seconds (from end of Sunday to Monday 10am)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on Saturday, merged on Sunday (should have 0 merge time)',
|
||||
pr: {
|
||||
number: 132,
|
||||
html_url: 'https://github.com/owner/repo/pull/132',
|
||||
draft: false,
|
||||
created_at: '2023-05-20T14:00:00Z', // Saturday
|
||||
merged_at: '2023-05-21T14:00:00Z', // Sunday
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 132,
|
||||
prUrl: 'https://github.com/owner/repo/pull/132',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-20T14:00:00Z'),
|
||||
mergeTime: new Date('2023-05-21T14:00:00Z'),
|
||||
mergeDate: '2023-05-21',
|
||||
mergeTimeSeconds: 0, // 0 seconds (both on weekend)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
},
|
||||
{
|
||||
name: 'PR ready on weekday, merged after multiple weekends (should subtract weekend days)',
|
||||
pr: {
|
||||
number: 133,
|
||||
html_url: 'https://github.com/owner/repo/pull/133',
|
||||
draft: false,
|
||||
created_at: '2023-05-17T14:00:00Z', // Wednesday
|
||||
merged_at: '2023-05-29T14:00:00Z', // Monday, 12 days later
|
||||
state: 'closed',
|
||||
user: { login: 'author' },
|
||||
base: {
|
||||
ref: 'main',
|
||||
repo: {
|
||||
name: 'repo',
|
||||
owner: { login: 'owner' }
|
||||
}
|
||||
}
|
||||
},
|
||||
timelineEvents: [],
|
||||
reviewEvents: [],
|
||||
expected: {
|
||||
metricType: 'time_to_merge',
|
||||
repository: 'owner/repo',
|
||||
prNumber: 133,
|
||||
prUrl: 'https://github.com/owner/repo/pull/133',
|
||||
prCreator: 'author',
|
||||
targetBranch: 'main',
|
||||
readyTime: new Date('2023-05-17T14:00:00Z'),
|
||||
mergeTime: new Date('2023-05-29T14:00:00Z'),
|
||||
mergeDate: '2023-05-29',
|
||||
mergeTimeSeconds: 691200, // 8 days = 691200 seconds (12 days - 4 weekend days)
|
||||
readyEventType: 'PR creation (not draft)'
|
||||
}
|
||||
}
|
||||
];
|
||||
|
||||
// Run each test case
|
||||
test.each(testCases)('$name', ({ pr, timelineEvents, reviewEvents, expected }) => {
|
||||
const result = githubClient.calculateTimeToMerge(pr, timelineEvents, reviewEvents);
|
||||
|
||||
if (expected === null) {
|
||||
expect(result).toBeNull();
|
||||
} else {
|
||||
// Compare date objects separately
|
||||
expect(result.readyTime).toEqual(expected.readyTime);
|
||||
expect(result.mergeTime).toEqual(expected.mergeTime);
|
||||
|
||||
// Compare the rest of the properties
|
||||
expect({
|
||||
...result,
|
||||
readyTime: undefined,
|
||||
mergeTime: undefined
|
||||
}).toEqual({
|
||||
...expected,
|
||||
readyTime: undefined,
|
||||
mergeTime: undefined
|
||||
});
|
||||
}
|
||||
});
|
||||
});
|
||||
});
|
||||
354
.github/actions/eng-metrics/test/markdown-parser.test.js
vendored
Normal file
354
.github/actions/eng-metrics/test/markdown-parser.test.js
vendored
Normal file
|
|
@ -0,0 +1,354 @@
|
|||
/**
|
||||
* Tests for markdown parser module
|
||||
*/
|
||||
|
||||
import { jest } from '@jest/globals';
|
||||
|
||||
// Mock the logger
|
||||
const mockLogger = {
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn()
|
||||
};
|
||||
|
||||
// Mock fs
|
||||
const mockFs = {
|
||||
existsSync: jest.fn(),
|
||||
readFileSync: jest.fn()
|
||||
};
|
||||
|
||||
// Mock path
|
||||
const mockPath = {
|
||||
resolve: jest.fn()
|
||||
};
|
||||
|
||||
// Set up module mocks
|
||||
jest.unstable_mockModule('../src/logger.js', () => ({
|
||||
default: mockLogger,
|
||||
...mockLogger
|
||||
}));
|
||||
|
||||
jest.unstable_mockModule('fs', () => ({
|
||||
default: mockFs,
|
||||
...mockFs
|
||||
}));
|
||||
|
||||
jest.unstable_mockModule('path', () => ({
|
||||
default: mockPath,
|
||||
...mockPath
|
||||
}));
|
||||
|
||||
// Import the module after mocking
|
||||
const { parseProductGroups, validateMarkdownStructure } = await import('../src/markdown-parser.js');
|
||||
|
||||
describe('MarkdownParser', () => {
|
||||
beforeEach(() => {
|
||||
jest.clearAllMocks();
|
||||
mockPath.resolve.mockImplementation((cwd, filePath) => `/resolved/${filePath}`);
|
||||
});
|
||||
|
||||
describe('parseProductGroups', () => {
|
||||
it('should parse valid markdown with all groups correctly', () => {
|
||||
const mockMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
## Some other content
|
||||
|
||||
### MDM group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Product Manager | _([@testpm1](https://github.com/testpm1))_ |
|
||||
| Developer | _([@testdev1](https://github.com/testdev1))_, _([@testdev2](https://github.com/testdev2))_, _([@testdev3](https://github.com/testdev3))_ |
|
||||
| Quality Assurance | _([@testqa1](https://github.com/testqa1))_ |
|
||||
|
||||
### Orchestration group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Product Manager | _([@testpm2](https://github.com/testpm2))_ |
|
||||
| Developer | _([@orchdev1](https://github.com/orchdev1))_, _([@orchdev2](https://github.com/orchdev2))_, _([@orchdev3](https://github.com/orchdev3))_ |
|
||||
|
||||
### Software group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Developer | _([@softdev1](https://github.com/softdev1))_, _([@softdev2](https://github.com/softdev2))_ |
|
||||
| Quality Assurance | _([@testqa2](https://github.com/testqa2))_ |
|
||||
`;
|
||||
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.readFileSync.mockReturnValue(mockMarkdown);
|
||||
|
||||
const result = parseProductGroups('test-file.md');
|
||||
|
||||
expect(mockFs.existsSync).toHaveBeenCalledWith('/resolved/test-file.md');
|
||||
expect(mockFs.readFileSync).toHaveBeenCalledWith('/resolved/test-file.md', 'utf8');
|
||||
|
||||
// Should extract usernames and create dual group membership
|
||||
expect(result).toEqual([
|
||||
// MDM group users
|
||||
{ group: 'mdm', username: 'testdev1' },
|
||||
{ group: 'engineering', username: 'testdev1' },
|
||||
{ group: 'mdm', username: 'testdev2' },
|
||||
{ group: 'engineering', username: 'testdev2' },
|
||||
{ group: 'mdm', username: 'testdev3' },
|
||||
{ group: 'engineering', username: 'testdev3' },
|
||||
// Orchestration group users
|
||||
{ group: 'orchestration', username: 'orchdev1' },
|
||||
{ group: 'engineering', username: 'orchdev1' },
|
||||
{ group: 'orchestration', username: 'orchdev2' },
|
||||
{ group: 'engineering', username: 'orchdev2' },
|
||||
{ group: 'orchestration', username: 'orchdev3' },
|
||||
{ group: 'engineering', username: 'orchdev3' },
|
||||
// Software group users
|
||||
{ group: 'software', username: 'softdev1' },
|
||||
{ group: 'engineering', username: 'softdev1' },
|
||||
{ group: 'software', username: 'softdev2' },
|
||||
{ group: 'engineering', username: 'softdev2' }
|
||||
]);
|
||||
|
||||
expect(mockLogger.info).toHaveBeenCalledWith('Parsing product groups from /resolved/test-file.md');
|
||||
expect(mockLogger.info).toHaveBeenCalledWith('Found 3 developers in mdm group: testdev1, testdev2, testdev3');
|
||||
expect(mockLogger.info).toHaveBeenCalledWith('Found 3 developers in orchestration group: orchdev1, orchdev2, orchdev3');
|
||||
expect(mockLogger.info).toHaveBeenCalledWith('Found 2 developers in software group: softdev1, softdev2');
|
||||
expect(mockLogger.info).toHaveBeenCalledWith('Extracted 16 user-group mappings from markdown');
|
||||
});
|
||||
|
||||
it('should handle missing sections gracefully', () => {
|
||||
const mockMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
### MDM group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Developer | _([@testdev1](https://github.com/testdev1))_ |
|
||||
|
||||
### Software group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Developer | _([@softdev1](https://github.com/softdev1))_ |
|
||||
`;
|
||||
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.readFileSync.mockReturnValue(mockMarkdown);
|
||||
|
||||
const result = parseProductGroups('test-file.md');
|
||||
|
||||
expect(result).toEqual([
|
||||
{ group: 'mdm', username: 'testdev1' },
|
||||
{ group: 'engineering', username: 'testdev1' },
|
||||
{ group: 'software', username: 'softdev1' },
|
||||
{ group: 'engineering', username: 'softdev1' }
|
||||
]);
|
||||
|
||||
expect(mockLogger.warn).toHaveBeenCalledWith('Section not found: Orchestration group');
|
||||
});
|
||||
|
||||
it('should handle missing Developer rows', () => {
|
||||
const mockMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
### MDM group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Product Manager | _([@testpm1](https://github.com/testpm1))_ |
|
||||
| Quality Assurance | _([@testqa1](https://github.com/testqa1))_ |
|
||||
|
||||
### Orchestration group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Developer | _([@orchdev1](https://github.com/orchdev1))_ |
|
||||
`;
|
||||
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.readFileSync.mockReturnValue(mockMarkdown);
|
||||
|
||||
const result = parseProductGroups('test-file.md');
|
||||
|
||||
expect(result).toEqual([
|
||||
{ group: 'orchestration', username: 'orchdev1' },
|
||||
{ group: 'engineering', username: 'orchdev1' }
|
||||
]);
|
||||
|
||||
expect(mockLogger.warn).toHaveBeenCalledWith('No Developer row found in mdm group section');
|
||||
});
|
||||
|
||||
it('should handle malformed GitHub username patterns', () => {
|
||||
const mockMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
### MDM group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Developer | Some text without proper format, _([@testvaliduser](https://github.com/testvaliduser))_, invalid format here |
|
||||
|
||||
### Orchestration group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Developer | No valid usernames here at all |
|
||||
`;
|
||||
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.readFileSync.mockReturnValue(mockMarkdown);
|
||||
|
||||
const result = parseProductGroups('test-file.md');
|
||||
|
||||
expect(result).toEqual([
|
||||
{ group: 'mdm', username: 'testvaliduser' },
|
||||
{ group: 'engineering', username: 'testvaliduser' }
|
||||
]);
|
||||
|
||||
expect(mockLogger.warn).toHaveBeenCalledWith('No GitHub usernames found in orchestration group Developer row');
|
||||
});
|
||||
|
||||
it('should handle multi-line developer cells', () => {
|
||||
const mockMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
### MDM group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Developer | _([@testuser1](https://github.com/testuser1))_,
|
||||
_([@testuser2](https://github.com/testuser2))_,
|
||||
_([@testuser3](https://github.com/testuser3))_ |
|
||||
`;
|
||||
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.readFileSync.mockReturnValue(mockMarkdown);
|
||||
|
||||
const result = parseProductGroups('test-file.md');
|
||||
|
||||
expect(result).toEqual([
|
||||
{ group: 'mdm', username: 'testuser1' },
|
||||
{ group: 'engineering', username: 'testuser1' },
|
||||
{ group: 'mdm', username: 'testuser2' },
|
||||
{ group: 'engineering', username: 'testuser2' },
|
||||
{ group: 'mdm', username: 'testuser3' },
|
||||
{ group: 'engineering', username: 'testuser3' }
|
||||
]);
|
||||
});
|
||||
|
||||
it('should handle file not found', () => {
|
||||
mockFs.existsSync.mockReturnValue(false);
|
||||
|
||||
const result = parseProductGroups('nonexistent-file.md');
|
||||
|
||||
expect(result).toEqual([]);
|
||||
expect(mockLogger.error).toHaveBeenCalledWith('Product groups file not found at /resolved/nonexistent-file.md');
|
||||
});
|
||||
|
||||
it('should handle file read errors', () => {
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.readFileSync.mockImplementation(() => {
|
||||
throw new Error('Permission denied');
|
||||
});
|
||||
|
||||
const result = parseProductGroups('error-file.md');
|
||||
|
||||
expect(result).toEqual([]);
|
||||
expect(mockLogger.error).toHaveBeenCalledWith(
|
||||
'Error parsing product groups file: error-file.md',
|
||||
{},
|
||||
expect.any(Error)
|
||||
);
|
||||
});
|
||||
|
||||
it('should handle usernames with hyphens and numbers', () => {
|
||||
const mockMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
### MDM group
|
||||
|
||||
| Role | Contributor |
|
||||
|------|-------------|
|
||||
| Developer | _([@testuser-123](https://github.com/testuser-123))_, _([@fakeuser-456](https://github.com/fakeuser-456))_ |
|
||||
`;
|
||||
|
||||
mockFs.existsSync.mockReturnValue(true);
|
||||
mockFs.readFileSync.mockReturnValue(mockMarkdown);
|
||||
|
||||
const result = parseProductGroups('test-file.md');
|
||||
|
||||
expect(result).toEqual([
|
||||
{ group: 'mdm', username: 'testuser-123' },
|
||||
{ group: 'engineering', username: 'testuser-123' },
|
||||
{ group: 'mdm', username: 'fakeuser-456' },
|
||||
{ group: 'engineering', username: 'fakeuser-456' }
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('validateMarkdownStructure', () => {
|
||||
it('should return true for valid markdown with all required sections', () => {
|
||||
const validMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
### MDM group
|
||||
Some content
|
||||
|
||||
### Orchestration group
|
||||
Some content
|
||||
|
||||
### Software group
|
||||
Some content
|
||||
`;
|
||||
|
||||
const result = validateMarkdownStructure(validMarkdown);
|
||||
expect(result).toBe(true);
|
||||
});
|
||||
|
||||
it('should return false and warn for missing sections', () => {
|
||||
const invalidMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
### MDM group
|
||||
Some content
|
||||
|
||||
### Software group
|
||||
Some content
|
||||
`;
|
||||
|
||||
const result = validateMarkdownStructure(invalidMarkdown);
|
||||
expect(result).toBe(false);
|
||||
expect(mockLogger.warn).toHaveBeenCalledWith('Missing required section: Orchestration group');
|
||||
});
|
||||
|
||||
it('should return false for completely empty content', () => {
|
||||
const result = validateMarkdownStructure('');
|
||||
expect(result).toBe(false);
|
||||
expect(mockLogger.warn).toHaveBeenCalledWith('Missing required section: MDM group');
|
||||
// The function returns false on first missing section, so other warnings may not be called
|
||||
expect(mockLogger.warn).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('should handle case-sensitive section matching', () => {
|
||||
const invalidMarkdown = `
|
||||
# Product Groups
|
||||
|
||||
### mdm group
|
||||
Some content
|
||||
|
||||
### orchestration group
|
||||
Some content
|
||||
|
||||
### software group
|
||||
Some content
|
||||
`;
|
||||
|
||||
const result = validateMarkdownStructure(invalidMarkdown);
|
||||
expect(result).toBe(false);
|
||||
expect(mockLogger.warn).toHaveBeenCalledWith('Missing required section: MDM group');
|
||||
// The function returns false on first missing section, so other warnings may not be called
|
||||
expect(mockLogger.warn).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
});
|
||||
321
.github/actions/eng-metrics/test/metrics-collector.test.js
vendored
Normal file
321
.github/actions/eng-metrics/test/metrics-collector.test.js
vendored
Normal file
|
|
@ -0,0 +1,321 @@
|
|||
/**
|
||||
* Tests for metrics collector module
|
||||
*/
|
||||
|
||||
import { jest } from '@jest/globals';
|
||||
import { MetricsCollector } from '../src/metrics-collector.js';
|
||||
|
||||
// Mock the logger
|
||||
jest.mock('../src/logger.js', () => ({
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
debug: jest.fn()
|
||||
}));
|
||||
|
||||
// Mock GitHubClient
|
||||
jest.mock('../src/github-client.js', () => {
|
||||
return jest.fn().mockImplementation(() => ({
|
||||
fetchPullRequests: jest.fn(),
|
||||
fetchPRTimelineEvents: jest.fn(),
|
||||
fetchPRReviewEvents: jest.fn(),
|
||||
calculatePickupTime: jest.fn(),
|
||||
calculateTimeToMerge: jest.fn()
|
||||
}));
|
||||
});
|
||||
|
||||
// Mock BigQueryClient
|
||||
jest.mock('../src/bigquery-client.js', () => ({
|
||||
BigQueryClient: jest.fn().mockImplementation(() => ({
|
||||
uploadMetrics: jest.fn()
|
||||
}))
|
||||
}));
|
||||
|
||||
describe('MetricsCollector', () => {
|
||||
let metricsCollector;
|
||||
let mockConfig;
|
||||
let mockGitHubClient;
|
||||
let mockBigQueryClient;
|
||||
|
||||
beforeEach(() => {
|
||||
mockConfig = {
|
||||
githubToken: 'fake-token',
|
||||
serviceAccountKeyPath: '/fake/path',
|
||||
repositories: ['owner/repo'],
|
||||
lookbackDays: 7,
|
||||
targetBranch: 'main',
|
||||
bigQueryDatasetId: 'test_dataset',
|
||||
printOnly: false,
|
||||
metrics: {
|
||||
timeToFirstReview: {
|
||||
enabled: true,
|
||||
tableName: 'pr_first_review'
|
||||
},
|
||||
timeToMerge: {
|
||||
enabled: true,
|
||||
tableName: 'pr_merge'
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
metricsCollector = new MetricsCollector(mockConfig);
|
||||
|
||||
// Mock the clients
|
||||
mockGitHubClient = {
|
||||
fetchPullRequests: jest.fn(),
|
||||
fetchPRTimelineEvents: jest.fn(),
|
||||
fetchPRReviewEvents: jest.fn(),
|
||||
calculatePickupTime: jest.fn(),
|
||||
calculateTimeToMerge: jest.fn()
|
||||
};
|
||||
|
||||
mockBigQueryClient = {
|
||||
uploadMetrics: jest.fn()
|
||||
};
|
||||
|
||||
metricsCollector.githubClient = mockGitHubClient;
|
||||
metricsCollector.bigqueryClient = mockBigQueryClient;
|
||||
});
|
||||
|
||||
describe('collectPRMetrics', () => {
|
||||
const mockPR = {
|
||||
number: 123,
|
||||
html_url: 'https://github.com/owner/repo/pull/123',
|
||||
user: { login: 'testuser' },
|
||||
base: { ref: 'main' },
|
||||
head: { repo: { full_name: 'owner/repo' } }
|
||||
};
|
||||
|
||||
const mockTimelineEvents = [
|
||||
{ event: 'ready_for_review', created_at: '2023-06-15T10:00:00Z' }
|
||||
];
|
||||
|
||||
const mockReviewEvents = [
|
||||
{ submitted_at: '2023-06-15T12:00:00Z', state: 'approved', user: { login: 'reviewer1' } }
|
||||
];
|
||||
|
||||
test('should collect both metrics when both are enabled', async () => {
|
||||
const firstReviewMetric = {
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 123,
|
||||
pickupTimeSeconds: 7200
|
||||
};
|
||||
|
||||
const mergeMetric = {
|
||||
metricType: 'time_to_merge',
|
||||
prNumber: 123,
|
||||
mergeTimeSeconds: 16200
|
||||
};
|
||||
|
||||
mockGitHubClient.calculatePickupTime.mockReturnValue(firstReviewMetric);
|
||||
mockGitHubClient.calculateTimeToMerge.mockReturnValue(mergeMetric);
|
||||
|
||||
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
|
||||
|
||||
expect(result).toEqual([firstReviewMetric, mergeMetric]);
|
||||
expect(mockGitHubClient.calculatePickupTime).toHaveBeenCalledWith(mockPR, mockTimelineEvents, mockReviewEvents);
|
||||
expect(mockGitHubClient.calculateTimeToMerge).toHaveBeenCalledWith(mockPR, mockTimelineEvents, mockReviewEvents);
|
||||
});
|
||||
|
||||
test('should only collect first review metric when merge is disabled', async () => {
|
||||
metricsCollector.config.metrics.timeToMerge.enabled = false;
|
||||
|
||||
const firstReviewMetric = {
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 123,
|
||||
pickupTimeSeconds: 7200
|
||||
};
|
||||
|
||||
mockGitHubClient.calculatePickupTime.mockReturnValue(firstReviewMetric);
|
||||
|
||||
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
|
||||
|
||||
expect(result).toEqual([firstReviewMetric]);
|
||||
expect(mockGitHubClient.calculatePickupTime).toHaveBeenCalled();
|
||||
expect(mockGitHubClient.calculateTimeToMerge).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('should only collect merge metric when first review is disabled', async () => {
|
||||
metricsCollector.config.metrics.timeToFirstReview.enabled = false;
|
||||
|
||||
const mergeMetric = {
|
||||
metricType: 'time_to_merge',
|
||||
prNumber: 123,
|
||||
mergeTimeSeconds: 16200
|
||||
};
|
||||
|
||||
mockGitHubClient.calculateTimeToMerge.mockReturnValue(mergeMetric);
|
||||
|
||||
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
|
||||
|
||||
expect(result).toEqual([mergeMetric]);
|
||||
expect(mockGitHubClient.calculatePickupTime).not.toHaveBeenCalled();
|
||||
expect(mockGitHubClient.calculateTimeToMerge).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('should handle null metrics gracefully', async () => {
|
||||
mockGitHubClient.calculatePickupTime.mockReturnValue(null);
|
||||
mockGitHubClient.calculateTimeToMerge.mockReturnValue(null);
|
||||
|
||||
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
|
||||
|
||||
expect(result).toEqual([]);
|
||||
});
|
||||
|
||||
test('should handle errors in metric calculation', async () => {
|
||||
mockGitHubClient.calculatePickupTime.mockImplementation(() => {
|
||||
throw new Error('Calculation error');
|
||||
});
|
||||
|
||||
const mergeMetric = {
|
||||
metricType: 'time_to_merge',
|
||||
prNumber: 123,
|
||||
mergeTimeSeconds: 16200
|
||||
};
|
||||
|
||||
mockGitHubClient.calculateTimeToMerge.mockReturnValue(mergeMetric);
|
||||
|
||||
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
|
||||
|
||||
expect(result).toEqual([mergeMetric]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('groupMetricsByType', () => {
|
||||
test('should group metrics by type correctly', () => {
|
||||
const metrics = [
|
||||
{ metricType: 'time_to_first_review', prNumber: 123 },
|
||||
{ metricType: 'time_to_merge', prNumber: 123 },
|
||||
{ metricType: 'time_to_first_review', prNumber: 124 },
|
||||
{ metricType: 'time_to_merge', prNumber: 124 }
|
||||
];
|
||||
|
||||
const grouped = metricsCollector.groupMetricsByType(metrics);
|
||||
|
||||
expect(grouped).toEqual({
|
||||
time_to_first_review: [
|
||||
{ metricType: 'time_to_first_review', prNumber: 123 },
|
||||
{ metricType: 'time_to_first_review', prNumber: 124 }
|
||||
],
|
||||
time_to_merge: [
|
||||
{ metricType: 'time_to_merge', prNumber: 123 },
|
||||
{ metricType: 'time_to_merge', prNumber: 124 }
|
||||
]
|
||||
});
|
||||
});
|
||||
|
||||
test('should handle empty metrics array', () => {
|
||||
const grouped = metricsCollector.groupMetricsByType([]);
|
||||
expect(grouped).toEqual({});
|
||||
});
|
||||
});
|
||||
|
||||
describe('getTableNameForMetricType', () => {
|
||||
test('should return correct table name for time_to_first_review', () => {
|
||||
const tableName = metricsCollector.getTableNameForMetricType('time_to_first_review');
|
||||
expect(tableName).toBe('pr_first_review');
|
||||
});
|
||||
|
||||
test('should return correct table name for time_to_merge', () => {
|
||||
const tableName = metricsCollector.getTableNameForMetricType('time_to_merge');
|
||||
expect(tableName).toBe('pr_merge');
|
||||
});
|
||||
|
||||
test('should throw error for unknown metric type', () => {
|
||||
expect(() => {
|
||||
metricsCollector.getTableNameForMetricType('unknown_type');
|
||||
}).toThrow('Unknown metric type: unknown_type');
|
||||
});
|
||||
});
|
||||
|
||||
describe('uploadMetrics', () => {
|
||||
test('should upload metrics to correct tables', async () => {
|
||||
const metrics = [
|
||||
{
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 123,
|
||||
pickupTimeSeconds: 7200
|
||||
},
|
||||
{
|
||||
metricType: 'time_to_merge',
|
||||
prNumber: 123,
|
||||
mergeTimeSeconds: 16200
|
||||
},
|
||||
{
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 124,
|
||||
pickupTimeSeconds: 3600
|
||||
}
|
||||
];
|
||||
|
||||
await metricsCollector.uploadMetrics(metrics);
|
||||
|
||||
expect(mockBigQueryClient.uploadMetrics).toHaveBeenCalledTimes(2);
|
||||
|
||||
// Check first call (time_to_first_review)
|
||||
expect(mockBigQueryClient.uploadMetrics).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
'test_dataset',
|
||||
'pr_first_review',
|
||||
[
|
||||
{ metricType: 'time_to_first_review', prNumber: 123, pickupTimeSeconds: 7200 },
|
||||
{ metricType: 'time_to_first_review', prNumber: 124, pickupTimeSeconds: 3600 }
|
||||
]
|
||||
);
|
||||
|
||||
// Check second call (time_to_merge)
|
||||
expect(mockBigQueryClient.uploadMetrics).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
'test_dataset',
|
||||
'pr_merge',
|
||||
[
|
||||
{ metricType: 'time_to_merge', prNumber: 123, mergeTimeSeconds: 16200 }
|
||||
]
|
||||
);
|
||||
});
|
||||
|
||||
test('should handle empty metrics array', async () => {
|
||||
await metricsCollector.uploadMetrics([]);
|
||||
|
||||
expect(mockBigQueryClient.uploadMetrics).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('should skip empty metric groups', async () => {
|
||||
const metrics = [
|
||||
{
|
||||
metricType: 'time_to_first_review',
|
||||
prNumber: 123,
|
||||
pickupTimeSeconds: 7200
|
||||
}
|
||||
];
|
||||
|
||||
await metricsCollector.uploadMetrics(metrics);
|
||||
|
||||
expect(mockBigQueryClient.uploadMetrics).toHaveBeenCalledTimes(1);
|
||||
expect(mockBigQueryClient.uploadMetrics).toHaveBeenCalledWith(
|
||||
'test_dataset',
|
||||
'pr_first_review',
|
||||
[{ metricType: 'time_to_first_review', prNumber: 123, pickupTimeSeconds: 7200 }]
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('getMetricTypeDisplayName', () => {
|
||||
test('should return correct display names', () => {
|
||||
expect(metricsCollector.getMetricTypeDisplayName('time_to_first_review')).toBe('Time to First Review');
|
||||
expect(metricsCollector.getMetricTypeDisplayName('time_to_merge')).toBe('Time to Merge');
|
||||
expect(metricsCollector.getMetricTypeDisplayName('unknown_type')).toBe('unknown_type');
|
||||
});
|
||||
});
|
||||
|
||||
describe('getTimeFieldForMetricType', () => {
|
||||
test('should return correct time fields', () => {
|
||||
const firstReviewMetric = { pickupTimeSeconds: 7200, mergeTimeSeconds: 16200 };
|
||||
const mergeMetric = { pickupTimeSeconds: 7200, mergeTimeSeconds: 16200 };
|
||||
|
||||
expect(metricsCollector.getTimeFieldForMetricType('time_to_first_review', firstReviewMetric)).toBe(7200);
|
||||
expect(metricsCollector.getTimeFieldForMetricType('time_to_merge', mergeMetric)).toBe(16200);
|
||||
expect(metricsCollector.getTimeFieldForMetricType('unknown_type', {})).toBe(0);
|
||||
});
|
||||
});
|
||||
});
|
||||
16
.github/actions/eng-metrics/test/setup.js
vendored
Normal file
16
.github/actions/eng-metrics/test/setup.js
vendored
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
/**
|
||||
* Jest setup file for engineering metrics tests
|
||||
*/
|
||||
import { jest } from "@jest/globals";
|
||||
|
||||
// Set test environment variables
|
||||
process.env.NODE_ENV = "test";
|
||||
|
||||
// Mock Date.now for consistent testing
|
||||
beforeAll(() => {
|
||||
jest.useFakeTimers().setSystemTime(new Date("2023-06-15T12:00:00Z"));
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
jest.useRealTimers();
|
||||
});
|
||||
57
.github/workflows/collect-eng-metrics-test.yml
vendored
Normal file
57
.github/workflows/collect-eng-metrics-test.yml
vendored
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
name: Collect engineering metrics test
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
- patch-*
|
||||
- prepare-*
|
||||
paths:
|
||||
- '.github/actions/eng-metrics/**'
|
||||
pull_request:
|
||||
paths:
|
||||
- '.github/actions/eng-metrics/**'
|
||||
workflow_dispatch:
|
||||
|
||||
# This allows a subsequently queued workflow run to interrupt previous runs
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id}}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
test:
|
||||
name: Test Engineering Metrics Action
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Checkout repository
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # 4.2.2
|
||||
|
||||
- name: Setup Node.js 20
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # 4.4.0
|
||||
with:
|
||||
node-version: 20
|
||||
cache: 'npm'
|
||||
cache-dependency-path: '.github/actions/eng-metrics/package-lock.json'
|
||||
|
||||
- name: Install dependencies
|
||||
run: npm ci
|
||||
working-directory: .github/actions/eng-metrics
|
||||
|
||||
- name: Run linting
|
||||
run: npm run lint
|
||||
working-directory: .github/actions/eng-metrics
|
||||
|
||||
- name: Run tests
|
||||
run: npm test
|
||||
working-directory: .github/actions/eng-metrics
|
||||
env:
|
||||
NODE_ENV: test
|
||||
55
.github/workflows/collect-eng-metrics.yml
vendored
Normal file
55
.github/workflows/collect-eng-metrics.yml
vendored
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
name: Collect engineering metrics
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: '0 9 * * *' # Run at 4am CDT (9am UTC)
|
||||
workflow_dispatch: # Allow manual triggering
|
||||
|
||||
# This allows a subsequently queued workflow run to interrupt previous runs
|
||||
concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id}}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read # fetch repo metadata
|
||||
pull-requests: read # read PR timelines
|
||||
|
||||
jobs:
|
||||
collect-metrics:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Harden Runner
|
||||
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
|
||||
with:
|
||||
egress-policy: audit
|
||||
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # 4.2.2
|
||||
|
||||
- name: Set up Node.js
|
||||
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # 4.4.0
|
||||
with:
|
||||
node-version: '20.18.1'
|
||||
cache: 'npm'
|
||||
cache-dependency-path: .github/actions/eng-metrics/package-lock.json
|
||||
|
||||
- name: Install dependencies
|
||||
working-directory: .github/actions/eng-metrics
|
||||
run: npm ci
|
||||
|
||||
- name: Create service account key file
|
||||
working-directory: .github/actions/eng-metrics
|
||||
run: |
|
||||
echo '${{ secrets.ENG_METRICS_GCP_SERVICE_ACCOUNT_KEY }}' > service-account-key.json
|
||||
# Verify the file is valid JSON and is a top-level object
|
||||
if ! jq 'type == "object"' service-account-key.json | grep -q true; then
|
||||
echo "Error: service-account-key.json is either invalid JSON or not a JSON object"
|
||||
echo "Is ENG_METRICS_GCP_SERVICE_ACCOUNT_KEY secret properly set?"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Collect and upload engineering metrics
|
||||
uses: ./.github/actions/eng-metrics
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
SERVICE_ACCOUNT_KEY_PATH: './service-account-key.json'
|
||||
Loading…
Reference in a new issue