Add workflows to collect engineering metrics. (#30540)

Fixes #29140 

This is an engineering initiated story that does not impact product.
This code has been running and manually tested in my own repo:
https://github.com/getvictor/eng-metrics

See
[README.md](https://github.com/fleetdm/fleet/blob/victor/29140-eng-metrics/.github/actions/eng-metrics/README.md)
in this branch for details.

The metrics can be viewed on
https://fleeteng.grafana.net/d/b97a629f-3626-4a28-9781-0fa3c8427897/engineering-metrics
(credentials in 1Password)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Introduced an engineering metrics collection tool that gathers GitHub
metrics (e.g., Time to First Review, Time to Merge) and uploads them to
BigQuery.
* Added support for user group management and product group mapping via
markdown parsing.
* Enabled print-only mode for testing metrics output without uploading
to BigQuery.
* Added automatic handling of bot filtering, weekend-aware time
calculations, and differential syncing of user groups.
* Implemented robust GitHub username validation and retry logic for API
rate limits.

* **Documentation**
* Added comprehensive usage and configuration documentation for the
engineering metrics tool.

* **Chores**
* Added configuration, environment example, and workflow files for
automated metrics collection and testing.
* Specified Node.js version and set up project dependencies and scripts.

* **Tests**
* Added extensive unit and end-to-end test suites to ensure reliability
of metrics collection, configuration, and integrations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
This commit is contained in:
Victor Lyuboslavsky 2025-07-03 16:59:25 -05:00 committed by GitHub
parent dce722cc07
commit 0d095b3778
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
30 changed files with 12885 additions and 0 deletions

View file

@ -0,0 +1,17 @@
# GitHub API token with repo scope
GITHUB_TOKEN=your_github_token_here
# Repositories to track (comma-separated list)
# REPOSITORIES=octocat/Hello-World,example-org/example-repo
# BigQuery dataset and table IDs (optional, defaults in config.json)
# BIGQUERY_DATASET_ID=github_metrics
# Path to the service account key file (optional, default in config.json)
# SERVICE_ACCOUNT_KEY_PATH=./service-account-key.json
# Target branch to track PRs for (optional, default: main)
# TARGET_BRANCH=main
# Print-only mode (set to 'true' to print metrics to console instead of uploading to BigQuery)
# PRINT_ONLY=true

1
.github/actions/eng-metrics/.nvmrc vendored Normal file
View file

@ -0,0 +1 @@
20.18.1

308
.github/actions/eng-metrics/README.md vendored Normal file
View file

@ -0,0 +1,308 @@
# Engineering metrics collector
A comprehensive tool to collect GitHub engineering metrics and upload them to BigQuery. This tool can be used as a standalone application or as a GitHub Action to track various development workflow metrics.
## Supported Metrics
```mermaid
---
title: Software development life cycle (SDLC)
---
graph TD
requirements[Understand the why]
design[Design solution]
implement[Implement code]
subgraph Time_to_Merge["⏱️ Time to Merge"]
subgraph Time_to_First_Review["⏱️ Time to First Review"]
pr_ready_for_review[PR ready for review]
first_review[Receive first review]
end
first_review --> rework["Changes requested &<br/>re-review (optional)"]
rework --> code_approved[Code approved]
code_approved --> pr_merged[Merge pull request]
end
testing[QA]
ready_for_release[Ready for release]
release[Release]
monitoring[Monitoring & feedback]
requirements --> design
design --> implement
implement --> pr_ready_for_review
pr_ready_for_review --> first_review
pr_merged --> testing
testing --> ready_for_release
ready_for_release --> release
release --> monitoring
```
This tool collects the following engineering metrics:
### 1. Time to First Review
Previously known as "PR Pickup Time" - measures the time between when a PR is marked as "Ready for Review" and when a reviewer has submitted a review.
- **Start Time**: When a PR is marked as "Ready for Review" - this can be:
- When a PR is created as a non-draft PR
- When a draft PR is converted to ready for review
- If multiple ready_for_review events exist, the tool uses the most recent one that occurred before the first review
- **End Time**: When the first review submission occurs (comment, approval, or changes requested)
- **Metric**: The time difference between these two events, excluding weekends
### 2. Time to Merge
Measures the time from when a PR is marked as "Ready for Review" to when it is merged.
- **Start Time**: When a PR is marked as "Ready for Review"
- **End Time**: When the PR is merged into the target branch
- **Metric**: The time difference between these two events, excluding weekends
### 3. User Group Management
Extracts GitHub usernames from a product groups markdown file and syncs them to BigQuery for team analytics.
- **Source**: Markdown file containing product group tables with GitHub usernames
- **Processing**: Validates GitHub usernames and creates dual group membership (specific group + engineering)
- **Storage**: Syncs to BigQuery `user_group` table with differential updates
- **Groups**: Supports MDM, Orchestration, and Software groups with automatic engineering group inclusion
### 4. Time to QA Ready (Planned)
Measures the time from issue reaching "In Progress" status to when it reaches "Awaiting QA" status in GitHub Projects.
- **Start Time**: When an issue is moved to "In Progress"
- **End Time**: When the issue status changes to "Awaiting QA" or "Ready for release" in GitHub Projects
- **Metric**: The time difference between these two events, excluding weekends
### 5. Time to Production Ready (Planned)
Measures the time from issue reaching "In Progress" status to when it reaches "Ready for Release" status in GitHub Projects.
- **Start Time**: When an issue is moved to "In Progress"
- **End Time**: When the issue status changes to "Ready for Release" in GitHub Projects
- **Metric**: The time difference between these two events, excluding weekends
## Current Implementation Status
- ✅ **Time to First Review**: Fully implemented and active
- ✅ **Time to Merge**: Fully implemented and active
- ✅ **User Group Management**: Fully implemented and active
- 🚧 **Time to QA Ready**: Planned for future implementation
- 🚧 **Time to Production Ready**: Planned for future implementation
## Features
- Collects engineering metrics from GitHub repositories
- Uploads metrics to Google BigQuery for analysis
- Configurable via JSON file and environment variables
- Can run as a standalone application or as a GitHub Action
- Supports multiple repositories (not tested)
- Only tracks PRs targeting the main branch
- Excludes weekends from time calculations
- Supports print-only mode for testing without BigQuery
## Prerequisites
- Node.js 20 or higher
- A GitHub token with `public_repo` scope or `repo` scope (for private repos)
- A Google Cloud project with BigQuery enabled
- A Google Cloud service account with BigQuery permissions
## Installation
```bash
# Clone the repo and go here
git clone <repo.git>
cd <repo_dir>/.github/actions/eng-metrics
# Install dependencies
npm install
```
## Configuration
### Configuration File
Create a `config.json` file with the following structure:
```json
{
"repositories": [
"owner/repo1",
"owner/repo2"
],
"targetBranch": "main",
"bigQueryDatasetId": "github_metrics",
"lookbackDays": 5,
"serviceAccountKeyPath": "./service-account-key.json",
"printOnly": false,
"userGroupEnabled": true,
"userGroupFilepath": "../../../handbook/company/product-groups.md",
"metrics": {
"timeToFirstReview": {
"enabled": true,
"tableName": "pr_first_review"
},
"timeToMerge": {
"enabled": true,
"tableName": "pr_merge"
}
}
}
```
### Environment Variables
You can also configure the tool using environment variables:
- `GITHUB_TOKEN`: GitHub token with repo scope
- `REPOSITORIES`: Comma-separated list of repositories to track (optional, overrides config.json)
- `BIGQUERY_DATASET_ID`: BigQuery dataset ID (optional, defaults to config.json)
- `SERVICE_ACCOUNT_KEY_PATH`: Path to the service account key file (optional, overrides config.json)
- `TARGET_BRANCH`: Target branch to track PRs for (optional, default: main)
- `PRINT_ONLY`: Set to 'true' to print metrics to console instead of uploading to BigQuery
- `ENABLED_METRICS`: Comma-separated list of metrics to collect (e.g., "time_to_first_review,time_to_merge")
- `TIME_TO_FIRST_REVIEW_TABLE`: Override table name for Time to First Review metrics (optional, defaults to "pr_first_review")
- `TIME_TO_MERGE_TABLE`: Override table name for Time to Merge metrics (optional, defaults to "pr_merge")
- `USER_GROUP_ENABLED`: Set to 'true' to enable user group processing (optional, defaults to false)
- `USER_GROUP_FILEPATH`: Path to the product groups markdown file (optional, defaults to "../../../handbook/company/product-groups.md")
Create a `.env` file based on the provided `.env.example` to set these variables.
### Configuration Priority
The tool uses the following configuration priority order (highest priority overrides lower priority):
1. **Environment Variables** (highest priority) - Values from `.env` file or system environment
2. **JSON Configuration File** (medium priority) - Values from `config.json` or specified config file
3. **Default Configuration** (lowest priority) - Built-in default values
This means that environment variables will always override values in the JSON configuration file, and both will override any default values. For example, if you have `"printOnly": false` in your `config.json` file but set `PRINT_ONLY=true` in your `.env` file, the tool will run in print-only mode.
## Usage
### As a Standalone Application
```bash
# Run with default config.json
npm start
# Run with a custom config file
npm start -- path/to/config.json
# Run in print-only mode (no BigQuery upload)
npm start -- --print-only
# Run with a custom config file in print-only mode
npm start -- path/to/config.json --print-only
```
### As a GitHub Action
See example in .github/workflows/collect-eng-metrics.yml
#### How the GitHub Action Works
1. **Service Account Key Handling**:
- The workflow writes the `ENG_METRICS_GCP_SERVICE_ACCOUNT_KEY` secret directly to a file
- It verifies that the file contains valid JSON using `jq`
- It then sets the `SERVICE_ACCOUNT_KEY_PATH` environment variable to point to this file
- The application uses this environment variable to locate the service account key file
**Important**: The service account key should be stored as a JSON string in the GitHub secret. Copy the entire contents of your service account key JSON file to the secret value.
2. **Configuration**:
- The workflow passes configuration values as environment variables
- Environment variables are used directly by the application
Make sure to set the following secrets in your repository:
- `GITHUB_TOKEN`: GitHub token with repo scope (automatically provided by GitHub Actions)
- `ENG_METRICS_GCP_SERVICE_ACCOUNT_KEY`: JSON service account key as a string
## BigQuery Schema
### Multi-Table Architecture
The tool uses separate BigQuery tables for different metric types to optimize performance and enable independent analysis:
#### Table 1: `pr_first_review` (Time to First Review)
| Field | Type | Description |
|---------------------|-----------|------------------------------------------------------------------------------|
| review_date | DATE | Date when the reviewer started looking at the PR |
| pr_creator | STRING | GitHub username of the PR creator (cluster key) |
| pr_url | STRING | HTTP link to the PR |
| pickup_time_seconds | INTEGER | Time in seconds from "Ready for Review" to first review (excluding weekends) |
| repository | STRING | Repository name (owner/repo) |
| pr_number | INTEGER | PR number (cluster key) |
| target_branch | STRING | Branch the PR is targeting (always "main") |
| ready_time | TIMESTAMP | Timestamp when PR was marked ready for review |
| first_review_time | TIMESTAMP | Timestamp of first review activity (partition key) |
#### Table 2: `pr_merge` (Time to Merge)
| Field | Type | Description |
|--------------------|-----------|-----------------------------------------------------------------------|
| merge_date | DATE | Date when the PR was merged |
| pr_creator | STRING | GitHub username of the PR creator (cluster key) |
| pr_url | STRING | HTTP link to the PR |
| merge_time_seconds | INTEGER | Time in seconds from "Ready for Review" to merge (excluding weekends) |
| repository | STRING | Repository name (owner/repo) |
| pr_number | INTEGER | PR number (cluster key) |
| target_branch | STRING | Branch the PR is targeting (always "main") |
| ready_time | TIMESTAMP | Timestamp when PR was marked ready for review |
| merge_time | TIMESTAMP | Timestamp when PR was merged (partition key) |
#### Table 3: `user_group` (User Group Management)
| Field | Type | Description |
|----------|--------|----------------------------------------------------------------------|
| group | STRING | Group name (mdm, orchestration, software, engineering) - cluster key |
| username | STRING | GitHub username |
**Table Optimizations:**
- `user_group` table is clustered by `group` for efficient group-based queries
- Supports dual group membership (specific group + engineering group for all users)
- Uses differential sync to minimize BigQuery operations
- Records are managed through intelligent insert/delete operations
- Due to BigQuery buffering, records cannot be updated/deleted within a 90-minute timeframe. This is OK as this update should run once per day.
**Multi-Table Optimizations:**
- `pr_first_review` table is partitioned by `DATE(first_review_time)` for efficient date-range queries
- `pr_merge` table is partitioned by `DATE(merge_time)` for efficient date-range queries
- `pr_first_review` and `pr_merge` tables are clustered by `pr_creator` and `pr_number` for efficient user-based analysis
- Each table uses `pr_number` as unique identifier (enforced at application level)
- Records are insert-only (no updates) to preserve historical data integrity
## Print-Only Mode
The tool supports a print-only mode that prints metrics to the console instead of uploading them to BigQuery. This is useful for testing without setting up BigQuery.
To enable print-only mode:
1. Set `printOnly: true` in your config.json file, OR
2. Set the `PRINT_ONLY=true` environment variable, OR
3. Use the `--print-only` command line flag
When running in print-only mode, you don't need to provide BigQuery credentials or configuration.
## Development
### Running Tests
```bash
npm test
```
### Linting
```bash
npm run lint
```
## Contributing
When contributing to this project, please:
1. Update tests for any new metrics or functionality
2. Update documentation to reflect changes
3. Follow the existing code style and patterns
4. Consider backward compatibility for BigQuery schema changes

10
.github/actions/eng-metrics/action.yml vendored Normal file
View file

@ -0,0 +1,10 @@
name: 'Engineering metrics collector'
description: 'Collects comprehensive GitHub engineering metrics including time to first review, time to merge, and GitHub Projects workflow metrics, then uploads them to BigQuery for analysis'
runs:
using: 'node20'
main: 'src/index.js'
branding:
icon: 'bar-chart-2'
color: 'blue'

22
.github/actions/eng-metrics/config.json vendored Normal file
View file

@ -0,0 +1,22 @@
{
"repositories": [
"fleetdm/fleet"
],
"targetBranch": "main",
"bigQueryDatasetId": "github_metrics",
"lookbackDays": 5,
"serviceAccountKeyPath": "./service-account-key.json",
"printOnly": false,
"userGroupEnabled": true,
"userGroupFilepath": "../../../handbook/company/product-groups.md",
"metrics": {
"timeToFirstReview": {
"enabled": true,
"tableName": "pr_first_review"
},
"timeToMerge": {
"enabled": true,
"tableName": "pr_merge"
}
}
}

View file

@ -0,0 +1,53 @@
import js from '@eslint/js';
export default [
js.configs.recommended,
{
languageOptions: {
ecmaVersion: 2022,
sourceType: 'module',
globals: {
console: 'readonly',
process: 'readonly',
Buffer: 'readonly',
__dirname: 'readonly',
__filename: 'readonly',
global: 'readonly',
setTimeout: 'readonly',
clearTimeout: 'readonly',
setInterval: 'readonly',
clearInterval: 'readonly'
}
},
rules: {
'indent': ['error', 2],
'linebreak-style': ['error', 'unix'],
'quotes': ['error', 'single'],
'semi': ['error', 'always'],
'no-unused-vars': ['error', { 'argsIgnorePattern': '^_' }],
'no-console': 'off',
'prefer-const': 'error',
'no-var': 'error',
'object-shorthand': 'error',
'prefer-arrow-callback': 'error'
}
},
{
files: ['test/**/*.js'],
languageOptions: {
globals: {
describe: 'readonly',
test: 'readonly',
expect: 'readonly',
beforeEach: 'readonly',
afterEach: 'readonly',
beforeAll: 'readonly',
afterAll: 'readonly',
jest: 'readonly'
}
},
rules: {
'no-unused-vars': ['error', { 'argsIgnorePattern': '^_', 'varsIgnorePattern': '^_' }]
}
}
];

View file

@ -0,0 +1,41 @@
/**
* Jest configuration for engineering metrics tests
*/
export default {
// Test environment
testEnvironment: "node",
// Transform configuration for ES modules
transform: {},
// File extensions to consider
moduleFileExtensions: ["js", "json"],
// Test file patterns
testMatch: ["**/test/**/*.test.js"],
// Coverage configuration
collectCoverage: false, // Disable for now to focus on functionality
coverageDirectory: "coverage",
coverageReporters: ["text", "lcov", "html"],
// Files to collect coverage from
collectCoverageFrom: [
"src/**/*.js",
"!src/index.js", // Exclude main entry point
"!src/logger.js", // Exclude logger (simple utility)
],
// Setup files
setupFilesAfterEnv: ["<rootDir>/test/setup.js"],
// Clear mocks between tests
clearMocks: true,
// Restore mocks after each test
restoreMocks: true,
// Verbose output
verbose: true,
};

6571
.github/actions/eng-metrics/package-lock.json generated vendored Normal file

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,44 @@
{
"name": "engineering-metrics-collector",
"version": "1.0.0",
"description": "Comprehensive GitHub engineering metrics collector - tracks time to first review, time to merge, and GitHub Projects workflow metrics",
"type": "module",
"main": "src/index.js",
"directories": {
"test": "test"
},
"scripts": {
"start": "node src/index.js",
"start:print": "node src/index.js --print-only",
"test": "node --experimental-vm-modules ./node_modules/jest/bin/jest.js",
"test:watch": "node --experimental-vm-modules ./node_modules/jest/bin/jest.js --watch",
"test:coverage": "node --experimental-vm-modules ./node_modules/jest/bin/jest.js --coverage",
"test:ci": "node --experimental-vm-modules ./node_modules/jest/bin/jest.js --ci --coverage --watchAll=false",
"lint": "./node_modules/.bin/eslint src/**/*.js",
"lint:fix": "./node_modules/.bin/eslint src/**/*.js --fix"
},
"keywords": [
"github",
"engineering-metrics",
"developer-productivity",
"time-to-first-review",
"time-to-merge",
"github-projects",
"workflow-metrics",
"bigquery",
"analytics"
],
"dependencies": {
"@google-cloud/bigquery": "^8.1.0",
"dotenv": "^17.0.0",
"octokit": "^5.0.3"
},
"devDependencies": {
"@eslint/js": "^9.30.0",
"eslint": "^9.30.0",
"jest": "^30.0.3"
},
"engines": {
"node": "20.18.1"
}
}

View file

@ -0,0 +1,400 @@
/**
* BigQuery client module for PR pickup time metrics collector
* Handles authentication and data upload to BigQuery
*/
import { BigQuery } from '@google-cloud/bigquery';
import fs from 'fs';
import logger from './logger.js';
/**
* BigQuery client class
*/
export class BigQueryClient {
/**
* Creates a new BigQuery client
* @param {string} keyFilePath - Path to the service account key file
*/
constructor(keyFilePath) {
this.bigquery = null;
this.projectId = null;
this.initialize(keyFilePath);
}
/**
* Initializes the BigQuery client
* @param {string} keyFilePath - Path to the service account key file
*/
initialize(keyFilePath) {
// Check if the key file exists
if (!fs.existsSync(keyFilePath)) {
const err = new Error(
`Service account key file not found at ${keyFilePath}`
);
logger.error('Failed to initialize BigQuery client', {}, err);
throw err;
}
try {
// Read and parse the service account key to extract project_id
const keyFileContent = fs.readFileSync(keyFilePath, 'utf8');
const serviceAccountKey = JSON.parse(keyFileContent);
if (!serviceAccountKey.project_id) {
throw new Error(
'Service account key file must contain a project_id field'
);
}
this.projectId = serviceAccountKey.project_id;
logger.info(
`Extracted project ID from service account key: ${this.projectId}`
);
this.bigquery = new BigQuery({
keyFilename: keyFilePath,
projectId: this.projectId,
});
logger.info('BigQuery client initialized');
} catch (err) {
logger.error('Failed to initialize BigQuery client', {}, err);
throw err;
}
}
/**
* Gets the project ID extracted from the service account key
* @returns {string} Project ID
*/
getProjectId() {
return this.projectId;
}
/**
* Creates a table if it doesn't exist with table-specific configuration
* @param {string} datasetId - BigQuery dataset ID
* @param {string} tableId - BigQuery table ID
* @param {Object} schema - BigQuery table schema
* @param {string} metricType - Type of metric (e.g., 'time_to_first_review', 'time_to_merge')
*/
async createTableIfNotExists(datasetId, tableId, schema, metricType) {
try {
// Get a reference to the dataset
const dataset = this.bigquery.dataset(datasetId);
// Check if the dataset exists, create it if it doesn't
const [datasetExists] = await dataset.exists();
if (!datasetExists) {
logger.info(`Dataset ${datasetId} does not exist, creating it`);
await dataset.create();
logger.info(`Dataset ${datasetId} created`);
}
// Get a reference to the table
const table = dataset.table(tableId);
// Check if the table exists, create it if it doesn't
const [tableExists] = await table.exists();
if (!tableExists) {
logger.info(`Table ${tableId} does not exist, creating it`);
// Get table-specific configuration based on metric type
const tableConfig = this.getConfigurationForMetricType(metricType);
const options = {
schema,
timePartitioning: tableConfig.timePartitioning,
clustering: tableConfig.clustering,
};
await table.create(options);
logger.info(`Table ${tableId} created`);
}
} catch (err) {
logger.error(`Error creating table ${datasetId}.${tableId}`, {}, err);
throw err;
}
}
/**
* Gets table-specific configuration for partitioning and clustering based on metric type
* @param {string} metricType - Type of metric (e.g., 'time_to_first_review', 'time_to_merge')
* @returns {Object} Table configuration
*/
getConfigurationForMetricType(metricType) {
switch (metricType) {
case 'time_to_first_review':
return {
timePartitioning: {
type: 'DAY',
field: 'first_review_time',
},
clustering: {
fields: ['pr_creator', 'pr_number'],
},
};
case 'time_to_merge':
return {
timePartitioning: {
type: 'DAY',
field: 'merge_time',
},
clustering: {
fields: ['pr_creator', 'pr_number'],
},
};
default:
throw new Error(
`Unknown metric type for table configuration: ${metricType}`
);
}
}
/**
* Gets the BigQuery table schema for a specific metric type
* @param {string} metricType - Type of metric (e.g., 'time_to_first_review', 'time_to_merge')
* @returns {Object} BigQuery table schema
*/
getSchemaForMetricType(metricType) {
switch (metricType) {
case 'time_to_first_review':
return {
fields: [
{ name: 'review_date', type: 'DATE', mode: 'REQUIRED' },
{ name: 'pr_creator', type: 'STRING', mode: 'REQUIRED' },
{ name: 'pr_url', type: 'STRING', mode: 'REQUIRED' },
{ name: 'pickup_time_seconds', type: 'INTEGER', mode: 'REQUIRED' },
{ name: 'repository', type: 'STRING', mode: 'REQUIRED' },
{ name: 'pr_number', type: 'INTEGER', mode: 'REQUIRED' },
{ name: 'target_branch', type: 'STRING', mode: 'REQUIRED' },
{ name: 'ready_time', type: 'TIMESTAMP', mode: 'REQUIRED' },
{ name: 'first_review_time', type: 'TIMESTAMP', mode: 'REQUIRED' },
],
};
case 'time_to_merge':
return {
fields: [
{ name: 'merge_date', type: 'DATE', mode: 'REQUIRED' },
{ name: 'pr_creator', type: 'STRING', mode: 'REQUIRED' },
{ name: 'pr_url', type: 'STRING', mode: 'REQUIRED' },
{ name: 'merge_time_seconds', type: 'INTEGER', mode: 'REQUIRED' },
{ name: 'repository', type: 'STRING', mode: 'REQUIRED' },
{ name: 'pr_number', type: 'INTEGER', mode: 'REQUIRED' },
{ name: 'target_branch', type: 'STRING', mode: 'REQUIRED' },
{ name: 'ready_time', type: 'TIMESTAMP', mode: 'REQUIRED' },
{ name: 'merge_time', type: 'TIMESTAMP', mode: 'REQUIRED' },
],
};
default:
throw new Error(`Unknown metric type: ${metricType}`);
}
}
/**
* Transforms metrics to BigQuery row format based on metric type
* @param {Object} metrics - Metrics object
* @returns {Object} BigQuery row
*/
transformMetricsToRow(metrics) {
switch (metrics.metricType) {
case 'time_to_first_review':
return {
review_date: metrics.reviewDate,
pr_creator: metrics.prCreator,
pr_url: metrics.prUrl,
pickup_time_seconds: metrics.pickupTimeSeconds,
repository: metrics.repository,
pr_number: metrics.prNumber,
target_branch: metrics.targetBranch,
ready_time: metrics.readyTime.toISOString(),
first_review_time: metrics.firstReviewTime.toISOString(),
};
case 'time_to_merge':
return {
merge_date: metrics.mergeDate,
pr_creator: metrics.prCreator,
pr_url: metrics.prUrl,
merge_time_seconds: metrics.mergeTimeSeconds,
repository: metrics.repository,
pr_number: metrics.prNumber,
target_branch: metrics.targetBranch,
ready_time: metrics.readyTime.toISOString(),
merge_time: metrics.mergeTime.toISOString(),
};
default:
throw new Error(`Unknown metric type: ${metrics.metricType}`);
}
}
/**
* Checks if metrics already exist in BigQuery for the given PR numbers
* @param {string} datasetId - BigQuery dataset ID
* @param {string} tableId - BigQuery table ID
* @param {Array} prNumbers - Array of PR numbers to check
* @returns {Object} Object with PR numbers as keys and boolean values indicating if they exist
*/
async checkExistingMetrics(datasetId, tableId, prNumbers) {
try {
// Get a reference to the table
const table = this.bigquery.dataset(datasetId).table(tableId);
// Check if the table exists
const [tableExists] = await table.exists();
if (!tableExists) {
// If the table doesn't exist, no metrics exist
return prNumbers.reduce((acc, prNumber) => {
acc[prNumber] = false;
return acc;
}, {});
}
// Create a parameterized query to check for existing PR numbers
const query = `
SELECT pr_number
FROM \`${this.projectId}.${datasetId}.${tableId}\`
WHERE pr_number IN UNNEST(@prNumbers)
`;
const options = {
query,
params: {
prNumbers,
},
};
// Run the query
const [rows] = await this.bigquery.query(options);
// Create a map of existing PR numbers
const existingPRs = rows.reduce((acc, row) => {
acc[row.pr_number] = true;
return acc;
}, {});
// Return a map of all PR numbers with their existence status
return prNumbers.reduce((acc, prNumber) => {
acc[prNumber] = !!existingPRs[prNumber];
return acc;
}, {});
} catch (err) {
logger.error(
`Error checking existing metrics in BigQuery ${datasetId}.${tableId}`,
{},
err
);
// If there's an error, assume no metrics exist
return prNumbers.reduce((acc, prNumber) => {
acc[prNumber] = false;
return acc;
}, {});
}
}
/**
* Uploads metrics to BigQuery
* @param {string} datasetId - BigQuery dataset ID
* @param {string} tableId - BigQuery table ID
* @param {Array} metrics - Array of metrics
*/
async uploadMetrics(datasetId, tableId, metrics) {
try {
if (!metrics || metrics.length === 0) {
logger.warn('No metrics to upload');
return;
}
logger.info(
`Uploading ${metrics.length} metrics to BigQuery table ${tableId}`
);
// Get metric type from the first metric to determine schema
const metricType = metrics[0]?.metricType;
if (!metricType) {
throw new Error('Metrics must have a metricType field');
}
// Ensure the table exists with the correct schema
const schema = this.getSchemaForMetricType(metricType);
await this.createTableIfNotExists(datasetId, tableId, schema, metricType);
// Get all PR numbers from the metrics
const prNumbers = metrics.map((metric) => metric.prNumber);
// Check which PR numbers already exist in BigQuery
const existingMetrics = await this.checkExistingMetrics(
datasetId,
tableId,
prNumbers
);
// Filter out metrics that already exist
const newMetrics = metrics.filter(
(metric) => !existingMetrics[metric.prNumber]
);
if (newMetrics.length === 0) {
logger.info('All metrics already exist in BigQuery, nothing to upload');
return;
}
logger.info(
`Uploading ${newMetrics.length} new metrics to BigQuery (${
metrics.length - newMetrics.length
} already exist)`
);
// Transform metrics to BigQuery row format
const rows = newMetrics.map((metric) =>
this.transformMetricsToRow(metric)
);
// Get a reference to the table
const table = this.bigquery.dataset(datasetId).table(tableId);
// Upload the rows to BigQuery
const [apiResponse] = await table.insert(rows);
logger.info(
`Successfully uploaded ${newMetrics.length} metrics to BigQuery`,
{
datasetId,
tableId,
insertedRows: newMetrics.length,
skippedRows: metrics.length - newMetrics.length,
}
);
return apiResponse;
} catch (err) {
logger.error(
`Error uploading metrics to BigQuery ${datasetId}.${tableId}`,
{},
err
);
// Log more details about the error if it's an insertion error
if (
err.name === 'PartialFailureError' &&
err.errors &&
err.errors.length > 0
) {
err.errors.forEach((error, index) => {
logger.error(`Row ${index} error:`, { error });
});
}
throw err;
}
}
}
export default BigQueryClient;

View file

@ -0,0 +1,273 @@
/**
* Configuration module for engineering metrics collector
* Loads and validates configuration from files and environment variables
*/
import fs from 'fs';
import path from 'path';
import dotenv from 'dotenv';
import logger from './logger.js';
// Load environment variables from .env file
dotenv.config();
/**
* Default configuration values
*/
const DEFAULT_CONFIG = {
// Default target branch to track PRs for
targetBranch: 'main',
// Default BigQuery dataset ID
bigQueryDatasetId: 'github_metrics',
// Default time window for fetching PRs (in days)
lookbackDays: 5,
// Default print-only mode (false = upload to BigQuery, true = print to console)
printOnly: false,
// User group management configuration
userGroupEnabled: true,
userGroupFilepath: '../../../handbook/company/product-groups.md',
// Bot filtering configuration
excludeBotReviews: true,
// Multi-table configuration
metrics: {
timeToFirstReview: {
enabled: true,
tableName: 'pr_first_review',
},
timeToMerge: {
enabled: true,
tableName: 'pr_merge',
},
},
};
/**
* Loads configuration from a JSON file
* @param {string} configPath - Path to the configuration file
* @returns {Object} Configuration object
*/
const loadConfigFromFile = (configPath) => {
try {
const resolvedPath = path.resolve(process.cwd(), configPath);
logger.info(`Loading configuration from ${resolvedPath}`);
if (!fs.existsSync(resolvedPath)) {
logger.warn(`Configuration file not found at ${resolvedPath}`);
return {};
}
const configData = fs.readFileSync(resolvedPath, 'utf8');
return JSON.parse(configData);
} catch (err) {
logger.error(
`Error loading configuration from file: ${configPath}`,
{},
err
);
return {};
}
};
/**
* Loads configuration from environment variables
* @returns {Object} Configuration object
*/
const loadConfigFromEnv = () => {
// Create a config object with only defined values
const config = {};
// Parse repositories from environment variable if provided
if (process.env.REPOSITORIES) {
config.repositories = process.env.REPOSITORIES.split(',').map((repo) =>
repo.trim()
);
}
// Add other environment variables if they are defined
if (process.env.GITHUB_TOKEN) config.githubToken = process.env.GITHUB_TOKEN;
if (process.env.BIGQUERY_DATASET_ID)
config.bigQueryDatasetId = process.env.BIGQUERY_DATASET_ID;
if (process.env.SERVICE_ACCOUNT_KEY_PATH)
config.serviceAccountKeyPath = process.env.SERVICE_ACCOUNT_KEY_PATH;
if (process.env.TARGET_BRANCH)
config.targetBranch = process.env.TARGET_BRANCH;
if (process.env.PRINT_ONLY)
config.printOnly = process.env.PRINT_ONLY === 'true';
if (process.env.USER_GROUP_ENABLED)
config.userGroupEnabled = process.env.USER_GROUP_ENABLED === 'true';
if (process.env.USER_GROUP_FILEPATH)
config.userGroupFilepath = process.env.USER_GROUP_FILEPATH;
// Handle metrics configuration from environment variables
if (process.env.ENABLED_METRICS) {
const enabledMetrics = process.env.ENABLED_METRICS.split(
','
).map((metric) => metric.trim());
config.metrics = {
timeToFirstReview: {
enabled: enabledMetrics.includes('time_to_first_review'),
tableName: process.env.TIME_TO_FIRST_REVIEW_TABLE || 'pr_first_review',
},
timeToMerge: {
enabled: enabledMetrics.includes('time_to_merge'),
tableName: process.env.TIME_TO_MERGE_TABLE || 'pr_merge',
},
};
}
return config;
};
/**
* Validates the configuration
* @param {Object} config - Configuration object
* @returns {boolean} True if configuration is valid, false otherwise
*/
const validateConfig = (config) => {
// Always required fields
const requiredFields = ['repositories', 'githubToken'];
// Fields required only when not in print-only mode
if (!config.printOnly) {
requiredFields.push('serviceAccountKeyPath');
}
const missingFields = requiredFields.filter((field) => !config[field]);
if (missingFields.length > 0) {
logger.error(
`Missing required configuration fields: ${missingFields.join(', ')}`
);
return false;
}
// Validate repositories array
if (!Array.isArray(config.repositories) || config.repositories.length === 0) {
logger.error('Configuration must include at least one repository');
return false;
}
// Validate repository format (owner/repo)
const invalidRepos = config.repositories.filter((repo) => {
return typeof repo !== 'string' || !repo.includes('/');
});
if (invalidRepos.length > 0) {
logger.error(`Invalid repository format: ${invalidRepos.join(', ')}`);
return false;
}
// Validate metrics configuration
if (!config.metrics || typeof config.metrics !== 'object') {
logger.error('Configuration must include metrics configuration');
return false;
}
// Validate that at least one metric is enabled
const enabledMetrics = Object.values(config.metrics).filter(
(metric) => metric.enabled
);
if (enabledMetrics.length === 0) {
logger.error('At least one metric must be enabled');
return false;
}
// Validate metric configurations
for (const [metricName, metricConfig] of Object.entries(config.metrics)) {
if (metricConfig.enabled) {
if (
!metricConfig.tableName ||
typeof metricConfig.tableName !== 'string'
) {
logger.error(`Metric ${metricName} must have a valid tableName`);
return false;
}
}
}
// Validate userGroupFilepath when userGroupEnabled is true
if (config.userGroupEnabled) {
if (!config.userGroupFilepath) {
logger.error(
'userGroupFilepath must be specified when userGroupEnabled is true'
);
return false;
}
const resolvedUserGroupPath = path.resolve(
process.cwd(),
config.userGroupFilepath
);
if (!fs.existsSync(resolvedUserGroupPath)) {
logger.error(`User group file not found at ${resolvedUserGroupPath}`);
return false;
}
}
return true;
};
/**
* Loads and validates configuration
* @param {string} [configPath='config.json'] - Path to the configuration file
* @returns {Object} Configuration object
*/
export const loadConfig = (configPath = 'config.json') => {
// Load configuration from file
const fileConfig = loadConfigFromFile(configPath);
// Load configuration from environment variables
const envConfig = loadConfigFromEnv();
// Merge configurations with precedence: env > file > default
const config = {
...DEFAULT_CONFIG,
...fileConfig,
...envConfig,
};
// Filter out undefined values
Object.keys(config).forEach((key) => {
if (config[key] === undefined) {
delete config[key];
}
});
// Validate configuration
const isValid = validateConfig(config);
if (!isValid) {
throw new Error('Invalid configuration');
}
logger.info('Configuration loaded successfully', {
repositories: config.repositories,
targetBranch: config.targetBranch,
printOnly: config.printOnly,
metrics: Object.fromEntries(
Object.entries(config.metrics).map(([key, value]) => [
key,
{ enabled: value.enabled, tableName: value.tableName },
])
),
...(config.printOnly
? {}
: {
bigQueryDatasetId: config.bigQueryDatasetId,
}),
});
return config;
};
export { validateConfig };
export default {
loadConfig,
};

View file

@ -0,0 +1,655 @@
/**
* GitHub client module for engineering metrics collector
* Handles interactions with the GitHub API using Octokit.js
*/
import { Octokit } from 'octokit';
import logger from './logger.js';
/**
* Identifies if a GitHub user is likely a bot
* @param {Object} user - GitHub user object
* @returns {Object} Bot analysis result
*/
function identifyBotUser(user) {
const botIndicators = {
isBot: false,
confidence: 'low',
reasons: [],
};
// Check GitHub's bot flag (most reliable)
if (user.type === 'Bot') {
botIndicators.isBot = true;
botIndicators.confidence = 'high';
botIndicators.reasons.push('GitHub API type is "Bot"');
return botIndicators;
}
// Check username patterns
const username = user.login.toLowerCase();
const botPatterns = [
/\[bot]/, // contains '[bot]'
/^dependabot/, // dependabot
/^renovate/, // renovate bot
/^github-actions/, // GitHub Actions
/^codecov/, // codecov bot
/^coderabbitai/, // coderabbit AI bot
/^sonarcloud/, // sonarcloud bot
/^snyk/, // snyk bot
/^greenkeeper/, // greenkeeper bot
/^semantic-release/, // semantic-release bot
/^stale/, // stale bot
/^imgbot/, // imgbot
/^allcontributors/, // all-contributors bot
/^whitesource/, // whitesource bot
/^deepsource/, // deepsource bot
];
for (const pattern of botPatterns) {
if (pattern.test(username)) {
botIndicators.isBot = true;
botIndicators.confidence = 'high';
botIndicators.reasons.push(`Username matches bot pattern: ${pattern}`);
break;
}
}
return botIndicators;
}
/**
* GitHub client class
*/
export class GitHubClient {
/**
* Creates a new GitHub client
* @param {string} token - GitHub API token
*/
constructor(token) {
if (!token) {
throw new Error('GitHub token is required');
}
this.octokit = null;
this.initialize(token);
}
/**
* Initializes the GitHub client
* @param {string} token - GitHub API token
*/
initialize(token) {
try {
this.octokit = new Octokit({
auth: token,
});
logger.info('GitHub client initialized');
} catch (err) {
logger.error('Failed to initialize GitHub client', {}, err);
throw err;
}
}
/**
* Fetches pull requests for a repository
* @param {string} owner - Repository owner
* @param {string} repo - Repository name
* @param {string} state - PR state (open, closed, all)
* @param {Date} since - Fetch PRs updated since this date
* @param {string} targetBranch - Target branch to filter PRs by
* @returns {Array} Array of pull requests
*/
async fetchPullRequests(
owner,
repo,
state = 'all',
since,
targetBranch = 'main'
) {
try {
logger.info(
`Fetching ${state} PRs for ${owner}/${repo} since ${since.toISOString()}`
);
// GitHub API returns paginated results, so we need to fetch all pages
const pullRequests = [];
let page = 1;
let hasMorePages = true;
while (hasMorePages) {
const response = await this.octokit.rest.pulls.list({
owner,
repo,
state,
sort: 'updated',
direction: 'desc',
per_page: 100,
page,
});
// Filter PRs by update date and target branch
const filteredPRs = response.data.filter((pr) => {
const prUpdatedAt = new Date(pr.updated_at);
return prUpdatedAt >= since && pr.base.ref === targetBranch;
});
if (filteredPRs.length > 0) {
pullRequests.push(...filteredPRs);
page++;
} else {
hasMorePages = false;
}
// If we got fewer results than the page size, there are no more pages
if (response.data.length < 100) {
hasMorePages = false;
}
}
logger.info(`Fetched ${pullRequests.length} PRs for ${owner}/${repo}`);
return pullRequests;
} catch (err) {
logger.error(`Error fetching PRs for ${owner}/${repo}`, {}, err);
// Implement basic retry for rate limiting
if (
err.status === 403 &&
err.response?.headers?.['x-ratelimit-remaining'] === '0'
) {
const resetTime =
parseInt(err.response.headers['x-ratelimit-reset'], 10) * 1000;
const waitTime = resetTime - Date.now();
if (waitTime > 0 && waitTime < 3600000) {
// Only retry if wait time is less than 1 hour
logger.info(
`Rate limit exceeded. Retrying in ${Math.ceil(
waitTime / 1000
)} seconds`
);
await new Promise((resolve) => setTimeout(resolve, waitTime + 1000));
return this.fetchPullRequests(
owner,
repo,
state,
since,
targetBranch
);
}
}
throw err;
}
}
/**
* Fetches PR review events
* @param {string} owner - Repository owner
* @param {string} repo - Repository name
* @param {number} prNumber - PR number
* @returns {Array} Array of review events
*/
async fetchPRReviewEvents(owner, repo, prNumber) {
try {
logger.info(`Fetching review events for ${owner}/${repo}#${prNumber}`);
const response = await this.octokit.rest.pulls.listReviews({
owner,
repo,
pull_number: prNumber,
});
logger.info(
`Fetched ${response.data.length} review events for ${owner}/${repo}#${prNumber}`
);
return response.data;
} catch (err) {
logger.error(
`Error fetching review events for ${owner}/${repo}#${prNumber}`,
{},
err
);
// Implement basic retry for rate limiting
if (
err.status === 403 &&
err.response?.headers?.['x-ratelimit-remaining'] === '0'
) {
const resetTime =
parseInt(err.response.headers['x-ratelimit-reset'], 10) * 1000;
const waitTime = resetTime - Date.now();
if (waitTime > 0 && waitTime < 3600000) {
// Only retry if wait time is less than 1 hour
logger.info(
`Rate limit exceeded. Retrying in ${Math.ceil(
waitTime / 1000
)} seconds`
);
await new Promise((resolve) => setTimeout(resolve, waitTime + 1000));
return this.fetchPRReviewEvents(owner, repo, prNumber);
}
}
throw err;
}
}
/**
* Filters out bot reviews from review events
* @param {Array} reviewEvents - Array of review events
* @param {boolean} excludeBots - Whether to exclude bot reviews (default: false)
* @returns {Array} Filtered review events
*/
filterBotReviews(reviewEvents, excludeBots = false) {
if (!excludeBots) {
return reviewEvents;
}
const filteredReviews = reviewEvents.filter((review) => {
const botAnalysis = identifyBotUser(review.user);
if (botAnalysis.isBot) {
logger.debug(`Filtering out bot review from ${review.user.login}`, {
confidence: botAnalysis.confidence,
reasons: botAnalysis.reasons,
});
return false;
}
return true;
});
const botCount = reviewEvents.length - filteredReviews.length;
if (botCount > 0) {
logger.info(
`Filtered out ${botCount} bot reviews from ${reviewEvents.length} total reviews`
);
}
return filteredReviews;
}
/**
* Fetches PR timeline events
* @param {string} owner - Repository owner
* @param {string} repo - Repository name
* @param {number} prNumber - PR number
* @returns {Array} Array of timeline events
*/
async fetchPRTimelineEvents(owner, repo, prNumber) {
try {
logger.info(`Fetching timeline events for ${owner}/${repo}#${prNumber}`);
// GitHub API returns paginated results, so we need to fetch all pages
const timelineEvents = [];
let page = 1;
let hasMorePages = true;
while (hasMorePages) {
const response = await this.octokit.rest.issues.listEventsForTimeline({
owner,
repo,
issue_number: prNumber,
per_page: 100,
page,
});
if (response.data.length > 0) {
timelineEvents.push(...response.data);
page++;
} else {
hasMorePages = false;
}
// If we got fewer results than the page size, there are no more pages
if (response.data.length < 100) {
hasMorePages = false;
}
}
logger.info(
`Fetched ${timelineEvents.length} timeline events for ${owner}/${repo}#${prNumber}`
);
return timelineEvents;
} catch (err) {
logger.error(
`Error fetching timeline events for ${owner}/${repo}#${prNumber}`,
{},
err
);
// Implement basic retry for rate limiting
if (
err.status === 403 &&
err.response?.headers?.['x-ratelimit-remaining'] === '0'
) {
const resetTime =
parseInt(err.response.headers['x-ratelimit-reset'], 10) * 1000;
const waitTime = resetTime - Date.now();
if (waitTime > 0 && waitTime < 3600000) {
// Only retry if wait time is less than 1 hour
logger.info(
`Rate limit exceeded. Retrying in ${Math.ceil(
waitTime / 1000
)} seconds`
);
await new Promise((resolve) => setTimeout(resolve, waitTime + 1000));
return this.fetchPRTimelineEvents(owner, repo, prNumber);
}
}
throw err;
}
}
/**
* Calculates pickup time for a PR
* @param {Object} pr - Pull request object
* @param {Array} timelineEvents - PR timeline events
* @param {Array} reviewEvents - PR review events
* @returns {Object} Pickup time metrics
*/
calculatePickupTime(pr, timelineEvents, reviewEvents) {
try {
const result = this.getReadyAndFirstReview(
pr,
timelineEvents,
reviewEvents
);
if (!result || !result.firstReviewTime) {
return null;
}
const { relevantReadyEvent, firstReviewTime } = result;
const readyTime = relevantReadyEvent.time;
// Calculate pickup time excluding weekends
const pickupTimeSeconds = this.calculatePickupTimeExcludingWeekends(
readyTime,
firstReviewTime
);
// If pickup time is negative, something went wrong
if (pickupTimeSeconds < 0) {
logger.warn(`Negative pickup time for ${pr.html_url}`, {
readyTime,
firstReviewTime,
pickupTimeSeconds,
});
return null;
}
// Log which ready event was used
const readyEventType =
relevantReadyEvent.event.event === 'created_not_draft'
? 'PR creation (not draft)'
: 'ready_for_review event';
logger.info(`Calculated pickup time for ${pr.html_url}`, {
pickupTimeSeconds,
readyEventType,
readyTime: readyTime.toISOString(),
firstReviewTime: firstReviewTime.toISOString(),
});
// We already have readyEventType defined above, so we can use it here
return {
metricType: 'time_to_first_review',
repository: `${pr.base.repo.owner.login}/${pr.base.repo.name}`,
prNumber: pr.number,
prUrl: pr.html_url,
prCreator: pr.user.login,
targetBranch: pr.base.ref,
readyTime,
firstReviewTime,
reviewDate: firstReviewTime.toISOString().split('T')[0], // YYYY-MM-DD
pickupTimeSeconds,
readyEventType,
};
} catch (err) {
logger.error(`Error calculating pickup time for ${pr.html_url}`, {}, err);
return null;
}
}
/**
* Calculates pickup time for a PR
* @param {Object} pr - Pull request object
* @param {Array} timelineEvents - PR timeline events
* @param {Array} reviewEvents - PR review events
* @returns {Object} ready event and first review time
*/
getReadyAndFirstReview(pr, timelineEvents, reviewEvents) {
const mergeTime = pr.merged_at ? new Date(pr.merged_at) : null;
// Find all ready_for_review events that occurred before merge time (if merged)
const readyForReviewEvents = timelineEvents
.filter((event) => event.event === 'ready_for_review')
.map((event) => ({
time: new Date(event.created_at),
event,
}))
.filter((readyEvent) => !mergeTime || readyEvent.time <= mergeTime);
// Add PR creation time as a ready event if PR was not created as draft
if (!pr.draft) {
readyForReviewEvents.push({
time: new Date(pr.created_at),
event: { event: 'created_not_draft', created_at: pr.created_at },
});
}
// Sort ready events by time (ascending)
readyForReviewEvents.sort((a, b) => a.time - b.time);
// If we couldn't find any ready events, return null
if (readyForReviewEvents.length === 0) {
logger.warn(`No ready_for_review events found for ${pr.html_url}`);
return null;
}
// If there is no review events, the PR may have been merged without a review.
if (reviewEvents.length === 0) {
const relevantReadyEvent =
readyForReviewEvents[readyForReviewEvents.length - 1];
return {
relevantReadyEvent,
firstReviewTime: null,
};
}
// Sort review events by submitted_at (ascending)
const sortedReviewEvents = [...reviewEvents].sort(
(a, b) => new Date(a.submitted_at) - new Date(b.submitted_at)
);
const firstReview = sortedReviewEvents[0];
const firstReviewTime = new Date(firstReview.submitted_at);
// Find the most recent ready event that occurred before the first review
const relevantReadyEvent = readyForReviewEvents
.filter((readyEvent) => readyEvent.time < firstReviewTime)
.pop();
// If no ready event occurred before the first review, return null
if (!relevantReadyEvent) {
logger.warn(
`No ready_for_review event found before first review for ${pr.html_url}`
);
return null;
}
return {
relevantReadyEvent,
firstReviewTime,
};
}
/**
* Calculates pickup time excluding weekends
* @param {Date} readyTimeOrig - Time when PR was marked as ready for review
* @param {Date} reviewTimeOrig - Time when the first review occurred
* @returns {number} Pickup time in seconds, excluding weekends
*/
calculatePickupTimeExcludingWeekends(readyTimeOrig, reviewTimeOrig) {
const readyTime = new Date(readyTimeOrig);
const reviewTime = new Date(reviewTimeOrig);
// Get day of week (0 = Sunday, 1 = Monday, ..., 6 = Saturday)
const readyDay = readyTime.getUTCDay();
const reviewDay = reviewTime.getUTCDay();
// Case: Both ready time and review time are on the same weekend
if (
(readyDay === 0 || readyDay === 6) &&
(reviewDay === 0 || reviewDay === 6) &&
Math.floor(reviewTime / (24 * 60 * 60 * 1000)) -
Math.floor(readyTime / (24 * 60 * 60 * 1000)) <=
2
) {
// Return 0 seconds pickup time
return 0;
}
// Set to start of Monday if ready time is on weekend
if (readyDay === 0) {
// Sunday
readyTime.setUTCDate(readyTime.getUTCDate() + 1);
readyTime.setUTCHours(0, 0, 0, 0);
} else if (readyDay === 6) {
// Saturday
readyTime.setUTCDate(readyTime.getUTCDate() + 2);
readyTime.setUTCHours(0, 0, 0, 0);
}
// Set to start of Saturday if review time is on Sunday
if (reviewDay === 0) {
// Sunday
reviewTime.setUTCDate(reviewTime.getUTCDate() - 1);
reviewTime.setUTCHours(0, 0, 0, 0);
} else if (reviewDay === 6) {
// Saturday
reviewTime.setUTCHours(0, 0, 0, 0);
}
// Calculate raw time difference in milliseconds
const weekendDays = countWeekendDays(readyTime, reviewTime);
const diffMs = reviewTime - readyTime - weekendDays * 24 * 60 * 60 * 1000;
// Ensure we don't return negative values
return Math.max(0, Math.floor(diffMs / 1000));
}
/**
* Calculate time to merge metrics
* @param {Object} pr - Pull request object
* @param {Array} timelineEvents - Timeline events
* @param {Array} reviewEvents - PR review events
* @returns {Object|null} Time to merge metrics or null if not applicable
*/
calculateTimeToMerge(pr, timelineEvents, reviewEvents) {
try {
// Only process merged PRs
if (!pr.merged_at) {
return null;
}
// Find the ready time using the same algorithm as we use for Time to First Review
const result = this.getReadyAndFirstReview(
pr,
timelineEvents,
reviewEvents
);
if (!result) {
return null;
}
const relevantReadyEvent = result.relevantReadyEvent;
const readyTime = relevantReadyEvent.time;
const mergeTime = new Date(pr.merged_at);
// Calculate merge time excluding weekends
const mergeTimeSeconds = this.calculatePickupTimeExcludingWeekends(
readyTime,
mergeTime
);
// If merge time is negative, something went wrong
if (mergeTimeSeconds < 0) {
logger.warn(`Negative merge time for ${pr.html_url}`, {
readyTime,
mergeTime,
mergeTimeSeconds,
});
return null;
}
// Log which ready event was used
const readyEventType =
relevantReadyEvent.event.event === 'created_not_draft'
? 'PR creation (not draft)'
: 'ready_for_review event';
logger.info(`Calculated merge time for ${pr.html_url}`, {
mergeTimeSeconds,
readyEventType,
readyTime: readyTime.toISOString(),
mergeTime: mergeTime.toISOString(),
});
return {
metricType: 'time_to_merge',
repository: `${pr.base.repo.owner.login}/${pr.base.repo.name}`,
prNumber: pr.number,
prUrl: pr.html_url,
prCreator: pr.user.login,
targetBranch: pr.base.ref,
readyTime,
mergeTime,
mergeDate: mergeTime.toISOString().split('T')[0], // YYYY-MM-DD
mergeTimeSeconds,
readyEventType,
};
} catch (err) {
logger.error(`Error calculating merge time for ${pr.html_url}`, {}, err);
return null;
}
}
}
function countWeekendDays(startDate, endDate) {
// Make local copies of dates
startDate = new Date(startDate);
endDate = new Date(endDate);
// Ensure startDate is before endDate
if (startDate > endDate) {
[startDate, endDate] = [endDate, startDate];
}
// Make sure start dates and end dates are not on weekends. We just want to count the weekend days between them.
if (startDate.getUTCDay() === 0) {
startDate.setUTCDate(startDate.getUTCDate() + 1);
} else if (startDate.getUTCDay() === 6) {
startDate.setUTCDate(startDate.getUTCDate() + 2);
}
if (endDate.getUTCDay() === 0) {
endDate.setUTCDate(endDate.getUTCDate() - 2);
} else if (endDate.getUTCDay() === 6) {
endDate.setUTCDate(endDate.getUTCDate() - 1);
}
let count = 0;
const current = new Date(startDate);
while (current <= endDate) {
const day = current.getUTCDay();
if (day === 0 || day === 6) {
// Sunday (0) or Saturday (6)
count++;
}
current.setUTCDate(current.getUTCDate() + 1);
}
return count;
}
export default GitHubClient;

View file

@ -0,0 +1,109 @@
/**
* GitHub username validator
* Validates that extracted usernames are real GitHub accounts
*/
import { Octokit } from 'octokit';
import logger from './logger.js';
/**
* Validates a single GitHub username
* @param {Octokit} octokit - GitHub API client
* @param {string} username - GitHub username to validate
* @returns {Promise<boolean>} True if username exists, false otherwise
*/
const validateUsername = async (octokit, username) => {
try {
await octokit.rest.users.getByUsername({ username });
return true;
} catch (error) {
if (error.status === 404) {
logger.warn(`GitHub username not found: ${username}`);
return false;
}
// For other errors (rate limiting, network issues), log but assume valid
logger.warn(`Error validating username ${username}: ${error.message}`);
return true; // Assume valid to avoid false negatives
}
};
/**
* Validates multiple GitHub usernames
* @param {string} githubToken - GitHub API token
* @param {Array<string>} usernames - Array of usernames to validate
* @returns {Promise<Array<string>>} Array of valid usernames
*/
export const validateUsernames = async (githubToken, usernames) => {
if (!githubToken) {
throw new Error('GitHub token is required for username validation');
}
const octokit = new Octokit({ auth: githubToken });
const validUsernames = [];
const invalidUsernames = [];
logger.info(`Validating ${usernames.length} GitHub usernames...`);
// Process usernames with a small delay to respect rate limits
for (const username of usernames) {
const isValid = await validateUsername(octokit, username);
if (isValid) {
validUsernames.push(username);
} else {
invalidUsernames.push(username);
}
// Small delay to avoid hitting rate limits too aggressively.
// 2025/07/03: GitHub's authenticated rate limit is 5000 requests/hour (~1.4 requests/second). This could lead to rate limit errors with larger username lists.
await new Promise((resolve) => setTimeout(resolve, 100));
}
if (invalidUsernames.length > 0) {
logger.warn(
`Found ${
invalidUsernames.length
} invalid GitHub usernames: ${invalidUsernames.join(', ')}`
);
}
logger.info(
`Validated ${validUsernames.length} out of ${usernames.length} GitHub usernames`
);
return validUsernames;
};
/**
* Filters user groups to only include valid usernames
* @param {string} githubToken - GitHub API token
* @param {Array<{group: string, username: string}>} userGroups - Array of user group mappings
* @returns {Promise<Array<{group: string, username: string}>>} Array of user group mappings with valid usernames only
*/
export const filterValidUserGroups = async (githubToken, userGroups) => {
// Get unique usernames for validation
const uniqueUsernames = [...new Set(userGroups.map((ug) => ug.username))];
// Validate usernames
const validUsernames = await validateUsernames(githubToken, uniqueUsernames);
const validUsernameSet = new Set(validUsernames);
// Filter user groups to only include valid usernames
const validUserGroups = userGroups.filter((ug) =>
validUsernameSet.has(ug.username)
);
const removedCount = userGroups.length - validUserGroups.length;
if (removedCount > 0) {
logger.info(
`Removed ${removedCount} user group mappings due to invalid usernames`
);
}
return validUserGroups;
};
export default {
validateUsernames,
filterValidUserGroups,
};

74
.github/actions/eng-metrics/src/index.js vendored Executable file
View file

@ -0,0 +1,74 @@
#!/usr/bin/env node
/**
* Main entry point for engineering metrics collector
* Collects comprehensive GitHub engineering metrics including:
* - Time to First Review (currently implemented)
* - Time to Merge (planned)
* - Time to QA Ready (planned)
* - Time to Production Ready (planned)
*/
import { loadConfig } from './config.js';
import { MetricsCollector } from './metrics-collector.js';
import logger from './logger.js';
import { fileURLToPath } from 'url';
import { dirname, join, resolve } from 'path';
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
/**
* Main function
*/
async function main() {
try {
// Set the working directory to one level up from the current file location, which is the root of the action.
process.chdir(resolve(__dirname, '..'));
// Parse command line arguments
const args = process.argv.slice(2);
// Check for print-only flag in command line arguments
const printOnlyFlag = args.includes('--print-only');
// Get the configuration path from command line arguments
// Filter out the --print-only flag if present
const configPath =
args.filter((arg) => arg !== '--print-only')[0] ||
join(__dirname, '..', 'config.json');
// Load configuration
const config = loadConfig(configPath);
// Override printOnly setting if flag is provided
if (printOnlyFlag) {
config.printOnly = true;
}
// Create and run metrics collector
const metricsCollector = new MetricsCollector(config);
const metrics = await metricsCollector.run();
if (config.printOnly) {
logger.info(
`Successfully collected and printed ${metrics.length} engineering metrics`
);
} else {
logger.info(
`Successfully collected and uploaded ${metrics.length} engineering metrics to BigQuery`
);
}
// Exit with success
process.exit(0);
} catch (err) {
logger.error('Error running engineering metrics collector', {}, err);
// Exit with error
process.exit(1);
}
}
// Run the main function
await main();

View file

@ -0,0 +1,112 @@
/**
* Logger module for engineering metrics collector
* Provides structured JSON logging
*/
/**
* Log levels
*/
const LOG_LEVELS = {
DEBUG: 'debug',
INFO: 'info',
WARN: 'warn',
ERROR: 'error',
};
/**
* Creates a log entry with the specified level, message, and optional data
* @param {string} level - Log level
* @param {string} message - Log message
* @param {Object} [data] - Optional data to include in the log
* @param {Error} [error] - Optional error object
* @returns {Object} Log entry object
*/
const createLogEntry = (level, message, data = {}, error = null) => {
const logEntry = {
timestamp: new Date().toISOString(),
level,
message,
...(Object.keys(data).length > 0 && { data }),
};
if (error) {
logEntry.error = {
name: error.name,
message: error.message,
stack: error.stack,
};
}
return logEntry;
};
/**
* Logs a message at the specified level
* @param {string} level - Log level
* @param {string} message - Log message
* @param {Object} [data] - Optional data to include in the log
* @param {Error} [error] - Optional error object
*/
const log = (level, message, data = {}, error = null) => {
const logEntry = createLogEntry(level, message, data, error);
try {
console.log(JSON.stringify(logEntry));
} catch (serializationError) {
console.log(
JSON.stringify({
timestamp: new Date().toISOString(),
level: 'error',
message: 'Failed to serialize log entry',
originalMessage: message,
error: {
name: serializationError.name,
message: serializationError.message,
},
})
);
}
};
/**
* Logs a debug message
* @param {string} message - Log message
* @param {Object} [data] - Optional data to include in the log
*/
export const debug = (message, data = {}) => {
log(LOG_LEVELS.DEBUG, message, data);
};
/**
* Logs an info message
* @param {string} message - Log message
* @param {Object} [data] - Optional data to include in the log
*/
export const info = (message, data = {}) => {
log(LOG_LEVELS.INFO, message, data);
};
/**
* Logs a warning message
* @param {string} message - Log message
* @param {Object} [data] - Optional data to include in the log
*/
export const warn = (message, data = {}) => {
log(LOG_LEVELS.WARN, message, data);
};
/**
* Logs an error message
* @param {string} message - Log message
* @param {Object} [data] - Optional data to include in the log
* @param {Error} [error] - Optional error object
*/
export const error = (message, data = {}, error) => {
log(LOG_LEVELS.ERROR, message, data, error);
};
export default {
debug,
info,
warn,
error,
};

View file

@ -0,0 +1,156 @@
/**
* Markdown parser for extracting GitHub usernames from product groups
* Parses the product-groups.md file to extract developer usernames by group
*/
import fs from 'fs';
import path from 'path';
import logger from './logger.js';
/**
* Parses the product groups markdown file and extracts GitHub usernames
* @param {string} filePath - Path to the product-groups.md file
* @returns {Array<{group: string, username: string}>} Array of user group mappings
*/
export const parseProductGroups = (filePath) => {
try {
const resolvedPath = path.resolve(process.cwd(), filePath);
logger.info(`Parsing product groups from ${resolvedPath}`);
if (!fs.existsSync(resolvedPath)) {
logger.error(`Product groups file not found at ${resolvedPath}`);
return [];
}
const content = fs.readFileSync(resolvedPath, 'utf8');
return extractUsernamesFromMarkdown(content);
} catch (err) {
logger.error(`Error parsing product groups file: ${filePath}`, {}, err);
return [];
}
};
/**
* Extracts usernames from markdown content
* @param {string} content - Markdown content
* @returns {Array<{group: string, username: string}>} Array of user group mappings
*/
const extractUsernamesFromMarkdown = (content) => {
const userGroups = [];
// Define the groups we're looking for and their corresponding database group names
const groupMappings = {
'MDM group': 'mdm',
'Orchestration group': 'orchestration',
'Software group': 'software',
};
// For each group, find its section and extract usernames
for (const [sectionName, groupName] of Object.entries(groupMappings)) {
// Find the section for this group
const sectionRegex = new RegExp(
`### ${sectionName}([\\s\\S]*?)(?=### |$)`,
'i'
);
const sectionMatch = content.match(sectionRegex);
if (sectionMatch) {
const sectionContent = sectionMatch[1];
const usernames = extractUsernamesFromSection(sectionContent, groupName);
userGroups.push(...usernames);
} else {
logger.warn(`Section not found: ${sectionName}`);
}
}
logger.info(
`Extracted ${userGroups.length} user-group mappings from markdown`
);
return userGroups;
};
/**
* Extracts usernames from a specific section
* @param {string} sectionContent - Content of the section
* @param {string} groupName - Name of the group (mdm, orchestration, software)
* @returns {Array<{group: string, username: string}>} Array of user group mappings
*/
const extractUsernamesFromSection = (sectionContent, groupName) => {
const userGroups = [];
// Look for the Developer row in the table
// The pattern needs to handle multi-line content in the cell
const developerRowMatch = sectionContent.match(
/\|\s*Developer\s*\|\s*([\s\S]*?)(?=\n\||\n\n|$)/
);
if (!developerRowMatch) {
logger.warn(`No Developer row found in ${groupName} group section`);
return userGroups;
}
const developerCell = developerRowMatch[1];
// Extract GitHub usernames from the developer cell
// Look for patterns like [@username](https://github.com/username)
// Note: This match could fail with slight variations in formatting (extra spaces, different brackets, etc.).
const usernameMatches = developerCell.match(/\[@([a-zA-Z0-9-]+)]\([^)]+\)/g);
if (!usernameMatches) {
logger.warn(
`No GitHub usernames found in ${groupName} group Developer row`
);
return userGroups;
}
const usernames = usernameMatches
.map((match) => {
// Extract username from _([@username](url))_ format
const usernameMatch = match.match(/\[@([a-zA-Z0-9-]+)]/);
return usernameMatch ? usernameMatch[1] : null;
})
.filter(Boolean);
logger.info(
`Found ${usernames.length
} developers in ${groupName} group: ${usernames.join(', ')}`
);
// Create user group mappings for both the specific group and engineering
for (const username of usernames) {
// Add to specific group (mdm, orchestration, software)
userGroups.push({ group: groupName, username });
// Add to engineering group (all developers are in engineering)
userGroups.push({ group: 'engineering', username });
}
return userGroups;
};
/**
* Validates the structure of the markdown content
* @param {string} content - Markdown content to validate
* @returns {boolean} True if structure is valid, false otherwise
*/
export const validateMarkdownStructure = (content) => {
const requiredSections = [
'MDM group',
'Orchestration group',
'Software group',
];
for (const section of requiredSections) {
if (!content.includes(`### ${section}`)) {
logger.warn(`Missing required section: ${section}`);
return false;
}
}
return true;
};
export default {
parseProductGroups,
validateMarkdownStructure,
};

View file

@ -0,0 +1,494 @@
/**
* Engineering metrics collector module
* Orchestrates the collection and uploading of comprehensive GitHub engineering metrics:
* - Time to First Review (currently implemented)
* - Time to Merge (planned)
* - Time to QA Ready (planned)
* - Time to Production Ready (planned)
*/
import GitHubClient from './github-client.js';
import BigQueryClient from './bigquery-client.js';
import { UserGroupClient } from './user-group-client.js';
import { parseProductGroups } from './markdown-parser.js';
import { filterValidUserGroups } from './github-validator.js';
import logger from './logger.js';
/**
* Metrics collector class
*/
export class MetricsCollector {
/**
* Creates a new metrics collector
* @param {Object} config - Configuration object
*/
constructor(config) {
this.config = config;
this.githubClient = null;
this.bigqueryClient = null;
this.userGroupClient = null;
}
/**
* Initializes the metrics collector
*/
async initialize() {
try {
logger.info('Initializing metrics collector');
// Initialize GitHub client
this.githubClient = new GitHubClient(this.config.githubToken);
// Initialize BigQuery client only if not in print-only mode
if (!this.config.printOnly) {
this.bigqueryClient = new BigQueryClient(this.config.serviceAccountKeyPath);
} else {
logger.info('Running in print-only mode, BigQuery client not initialized');
}
// Initialize User Group client if user group processing is enabled
if (this.config.userGroupEnabled) {
if (!this.config.printOnly) {
// Get project ID from BigQuery client
const projectId = this.bigqueryClient.getProjectId();
this.userGroupClient = new UserGroupClient(
projectId,
this.config.bigQueryDatasetId,
this.config.serviceAccountKeyPath,
this.config.printOnly
);
} else {
// For print-only mode, we don't need a real project ID
this.userGroupClient = new UserGroupClient(
'print-only-project',
this.config.bigQueryDatasetId,
this.config.serviceAccountKeyPath,
this.config.printOnly
);
}
logger.info('User group client initialized');
}
logger.info('Metrics collector initialized');
} catch (err) {
logger.error('Failed to initialize metrics collector', {}, err);
throw err;
}
}
/**
* Collects metrics for a single repository
* @param {string} repository - Repository in the format owner/repo
* @returns {Array} Array of engineering metrics
*/
async collectRepositoryMetrics(repository) {
const [owner, repo] = repository.split('/');
if (!owner || !repo) {
const err = new Error(`Invalid repository format: ${repository}`);
logger.error(`Error collecting metrics for ${repository}`, {}, err);
return [];
}
logger.info(`Collecting metrics for ${repository}`);
try {
// Calculate the date to fetch PRs from (lookbackDays ago)
const since = new Date();
since.setDate(since.getDate() - this.config.lookbackDays);
// Fetch PRs updated since the lookback date
const pullRequests = await this.githubClient.fetchPullRequests(
owner,
repo,
'all',
since,
this.config.targetBranch
);
logger.info(`Found ${pullRequests.length} PRs for ${repository}`);
// Collect metrics for each PR
const metrics = [];
for (const pr of pullRequests) {
try {
// Fetch PR timeline events (shared for all metrics)
const timelineEvents = await this.githubClient.fetchPRTimelineEvents(
owner,
repo,
pr.number
);
// Fetch PR review events (needed for Time to First Review)
const rawReviewEvents = await this.githubClient.fetchPRReviewEvents(
owner,
repo,
pr.number
);
// Filter bot reviews if configured
const reviewEvents = this.githubClient.filterBotReviews(
rawReviewEvents,
this.config.excludeBotReviews
);
// Collect enabled metrics for this PR
const prMetrics = await this.collectPRMetrics(pr, timelineEvents, reviewEvents);
metrics.push(...prMetrics);
} catch (err) {
logger.error(`Error collecting metrics for PR ${repository}#${pr.number}`, {}, err);
}
}
logger.info(`Collected ${metrics.length} metrics for ${repository}`);
return metrics;
} catch (err) {
logger.error(`Error collecting metrics for ${repository}`, {}, err);
return [];
}
}
/**
* Collects enabled metrics for a single PR
* @param {Object} pr - Pull request object
* @param {Array} timelineEvents - PR timeline events
* @param {Array} reviewEvents - PR review events
* @returns {Array} Array of metrics for this PR
*/
async collectPRMetrics(pr, timelineEvents, reviewEvents) {
const metrics = [];
// Collect Time to First Review if enabled
if (this.config.metrics.timeToFirstReview.enabled) {
try {
const pickupTimeMetrics = this.githubClient.calculatePickupTime(
pr,
timelineEvents,
reviewEvents
);
if (pickupTimeMetrics) {
metrics.push(pickupTimeMetrics);
}
} catch (err) {
logger.error(`Error calculating Time to First Review for PR #${pr.number}`, {}, err);
}
}
// Collect Time to Merge if enabled
if (this.config.metrics.timeToMerge.enabled) {
try {
const mergeTimeMetrics = this.githubClient.calculateTimeToMerge(
pr,
timelineEvents,
reviewEvents
);
if (mergeTimeMetrics) {
metrics.push(mergeTimeMetrics);
}
} catch (err) {
logger.error(`Error calculating Time to Merge for PR #${pr.number}`, {}, err);
}
}
return metrics;
}
/**
* Collects metrics for all repositories
* @returns {Array} Array of engineering metrics
*/
async collectMetrics() {
try {
logger.info('Collecting metrics for all repositories');
const allMetrics = [];
// Collect metrics for each repository
for (const repository of this.config.repositories) {
const metrics = await this.collectRepositoryMetrics(repository);
allMetrics.push(...metrics);
}
logger.info(`Collected ${allMetrics.length} metrics in total`);
return allMetrics;
} catch (err) {
logger.error('Error collecting metrics', {}, err);
throw err;
}
}
/**
* Processes user groups from the markdown file
* @returns {Promise<void>}
*/
async processUserGroups() {
if (!this.config.userGroupEnabled) {
logger.info('User group processing is disabled');
return;
}
try {
logger.info('Processing user groups');
// Parse user groups from markdown file
const userGroups = parseProductGroups(this.config.userGroupFilepath);
if (userGroups.length === 0) {
logger.warn('No user groups found in markdown file');
return;
}
// Validate GitHub usernames
const validUserGroups = await filterValidUserGroups(
this.config.githubToken,
userGroups
);
if (validUserGroups.length === 0) {
logger.warn('No valid user groups found after validation');
return;
}
// Sync user groups to BigQuery
await this.userGroupClient.syncUserGroups(validUserGroups);
logger.info(`Successfully processed ${validUserGroups.length} user group mappings`);
} catch (err) {
logger.error('Error processing user groups', {}, err);
throw err;
}
}
/**
* Prints metrics to the console in a readable format
* @param {Array} metrics - Array of engineering metrics
*/
printMetrics(metrics) {
try {
if (!metrics || metrics.length === 0) {
logger.warn('No metrics to print');
return;
}
logger.info(`Printing ${metrics.length} metrics to console`);
// Group metrics by type for organized display
const metricsByType = this.groupMetricsByType(metrics);
console.log('\n=== Engineering Metrics ===\n');
// Print each metric type separately
for (const [metricType, typeMetrics] of Object.entries(metricsByType)) {
if (typeMetrics.length === 0) continue;
console.log(`--- ${this.getMetricTypeDisplayName(metricType)} (${typeMetrics.length} metrics) ---\n`);
// Sort metrics by time (descending)
const sortedMetrics = [...typeMetrics].sort((a, b) => {
const timeFieldA = this.getTimeFieldForMetricType(metricType, a);
const timeFieldB = this.getTimeFieldForMetricType(metricType, b);
return timeFieldB - timeFieldA;
});
// Print each metric
sortedMetrics.forEach((metric, index) => {
this.printSingleMetric(metric, index + 1);
});
// Print summary statistics for this metric type
this.printMetricTypeSummary(metricType, typeMetrics);
console.log('');
}
logger.info('Metrics printed successfully');
} catch (err) {
logger.error('Error printing metrics', {}, err);
throw err;
}
}
/**
* Prints a single metric to the console
* @param {Object} metric - Single metric object
* @param {number} index - Index for display
*/
printSingleMetric(metric, index) {
console.log(`[${index}] PR: ${metric.repository}#${metric.prNumber}`);
console.log(` URL: ${metric.prUrl}`);
console.log(` Creator: ${metric.prCreator}`);
console.log(` Ready Time: ${metric.readyTime.toISOString()}${metric.readyEventType ? ` (${metric.readyEventType})` : ''}`);
if (metric.metricType === 'time_to_first_review') {
const hours = Math.floor(metric.pickupTimeSeconds / 3600);
const minutes = Math.floor((metric.pickupTimeSeconds % 3600) / 60);
const seconds = metric.pickupTimeSeconds % 60;
console.log(` First Review Time: ${metric.firstReviewTime.toISOString()}`);
console.log(` Pickup Time: ${hours}h ${minutes}m ${seconds}s (${metric.pickupTimeSeconds} seconds)`);
} else if (metric.metricType === 'time_to_merge') {
const hours = Math.floor(metric.mergeTimeSeconds / 3600);
const minutes = Math.floor((metric.mergeTimeSeconds % 3600) / 60);
const seconds = metric.mergeTimeSeconds % 60;
console.log(` Merge Time: ${metric.mergeTime.toISOString()}`);
console.log(` Time to Merge: ${hours}h ${minutes}m ${seconds}s (${metric.mergeTimeSeconds} seconds)`);
}
console.log('');
}
/**
* Prints summary statistics for a metric type
* @param {string} metricType - Type of metric
* @param {Array} metrics - Array of metrics of this type
*/
printMetricTypeSummary(metricType, metrics) {
const timeField = metricType === 'time_to_first_review' ? 'pickupTimeSeconds' : 'mergeTimeSeconds';
const totalTime = metrics.reduce((sum, metric) => sum + metric[timeField], 0);
const avgTime = totalTime / metrics.length;
const avgHours = Math.floor(avgTime / 3600);
const avgMinutes = Math.floor((avgTime % 3600) / 60);
const avgSeconds = Math.floor(avgTime % 60);
console.log(`=== ${this.getMetricTypeDisplayName(metricType)} Summary ===`);
console.log(`Total PRs: ${metrics.length}`);
console.log(`Average Time: ${avgHours}h ${avgMinutes}m ${avgSeconds}s (${Math.floor(avgTime)} seconds)`);
}
/**
* Gets display name for metric type
* @param {string} metricType - Type of metric
* @returns {string} Display name
*/
getMetricTypeDisplayName(metricType) {
switch (metricType) {
case 'time_to_first_review':
return 'Time to First Review';
case 'time_to_merge':
return 'Time to Merge';
default:
return metricType;
}
}
/**
* Gets the time field value for sorting metrics
* @param {string} metricType - Type of metric
* @param {Object} metric - Metric object
* @returns {number} Time value in seconds
*/
getTimeFieldForMetricType(metricType, metric) {
switch (metricType) {
case 'time_to_first_review':
return metric.pickupTimeSeconds;
case 'time_to_merge':
return metric.mergeTimeSeconds;
default:
return 0;
}
}
/**
* Uploads metrics to BigQuery, grouped by metric type
* @param {Array} metrics - Array of engineering metrics
*/
async uploadMetrics(metrics) {
try {
if (!metrics || metrics.length === 0) {
logger.warn('No metrics to upload');
return;
}
logger.info(`Uploading ${metrics.length} metrics to BigQuery`);
// Group metrics by type
const metricsByType = this.groupMetricsByType(metrics);
// Upload each metric type to its respective table
for (const [metricType, typeMetrics] of Object.entries(metricsByType)) {
if (typeMetrics.length === 0) continue;
const tableName = this.getTableNameForMetricType(metricType);
logger.info(`Uploading ${typeMetrics.length} ${metricType} metrics to table ${tableName}`);
await this.bigqueryClient.uploadMetrics(
this.config.bigQueryDatasetId,
tableName,
typeMetrics
);
}
logger.info('All metrics uploaded successfully');
} catch (err) {
logger.error('Error uploading metrics to BigQuery', {}, err);
throw err;
}
}
/**
* Groups metrics by their type
* @param {Array} metrics - Array of metrics
* @returns {Object} Object with metric types as keys and arrays of metrics as values
*/
groupMetricsByType(metrics) {
return metrics.reduce((groups, metric) => {
const type = metric.metricType;
if (!groups[type]) {
groups[type] = [];
}
groups[type].push(metric);
return groups;
}, {});
}
/**
* Gets the table name for a specific metric type
* @param {string} metricType - Type of metric
* @returns {string} Table name
*/
getTableNameForMetricType(metricType) {
switch (metricType) {
case 'time_to_first_review':
return this.config.metrics.timeToFirstReview.tableName;
case 'time_to_merge':
return this.config.metrics.timeToMerge.tableName;
default:
throw new Error(`Unknown metric type: ${metricType}`);
}
}
/**
* Runs the metrics collection and upload process
*/
async run() {
try {
logger.info('Starting engineering metrics collection');
// Initialize the metrics collector
await this.initialize();
// Process user groups if enabled
await this.processUserGroups();
// Collect metrics
const metrics = await this.collectMetrics();
if (this.config.printOnly) {
// Print metrics to console
this.printMetrics(metrics);
} else {
// Upload metrics to BigQuery
await this.uploadMetrics(metrics);
}
logger.info('Engineering metrics collection completed successfully');
return metrics;
} catch (err) {
logger.error('Error running engineering metrics collection', {}, err);
throw err;
}
}
}
export default MetricsCollector;

View file

@ -0,0 +1,374 @@
/**
* User Group BigQuery Client
* Manages the user_group BigQuery table for team organization tracking
*/
import { BigQuery } from '@google-cloud/bigquery';
import logger from './logger.js';
/**
* User Group BigQuery Client class
*/
export class UserGroupClient {
/**
* Creates a new UserGroupClient instance
* @param {string} projectId - Google Cloud project ID
* @param {string} datasetId - BigQuery dataset ID
* @param {string} serviceAccountKeyPath - Path to service account key file
* @param {boolean} printOnly - If true, print operations instead of executing them
*/
constructor(projectId, datasetId, serviceAccountKeyPath, printOnly = false) {
this.projectId = projectId;
this.datasetId = datasetId;
this.tableId = 'user_group';
this.printOnly = printOnly;
if (!printOnly) {
this.bigquery = new BigQuery({
projectId,
keyFilename: serviceAccountKeyPath,
});
this.dataset = this.bigquery.dataset(datasetId);
this.table = this.dataset.table(this.tableId);
}
}
/**
* Creates the user_group table if it doesn't exist
* @returns {Promise<void>}
*/
async createUserGroupTable() {
if (this.printOnly) {
logger.info(
'[USER GROUPS] Print-only mode: would create user_group table with schema:'
);
logger.info('[USER GROUPS] - group (STRING, cluster key)');
logger.info('[USER GROUPS] - username (STRING)');
return;
}
try {
const [exists] = await this.table.exists();
if (exists) {
logger.info('user_group table already exists');
return;
}
const schema = [
{ name: 'group', type: 'STRING', mode: 'REQUIRED' },
{ name: 'username', type: 'STRING', mode: 'REQUIRED' },
];
const options = {
schema,
clustering: {
fields: ['group'],
},
};
await this.table.create(options);
logger.info('Created user_group table with clustering on group field');
} catch (error) {
logger.error('Error creating user_group table:', {}, error);
throw error;
}
}
/**
* Syncs user groups to the BigQuery table (replaces all data)
* @param {Array<{group: string, username: string}>} userGroups - Array of user group mappings
* @returns {Promise<void>}
*/
async syncUserGroups(userGroups) {
if (this.printOnly) {
this.printUserGroupsData(userGroups);
return;
}
try {
if (userGroups.length === 0) {
logger.warn('No user groups to sync');
return;
}
// Ensure table exists first
await this.createUserGroupTable();
// Small delay to ensure table is ready
await new Promise((resolve) => setTimeout(resolve, 1000));
// Refresh table reference
this.table = this.dataset.table(this.tableId);
// Get existing entries and perform differential sync
await this.syncUserGroupsDifferential(userGroups);
logger.info(
`Successfully synced ${userGroups.length} user group mappings to BigQuery`
);
} catch (error) {
logger.error('Error syncing user groups:', {}, error);
throw error;
}
}
/**
* Performs differential sync of user groups (only insert new, delete removed)
* @param {Array<{group: string, username: string}>} newUserGroups - New user group mappings
* @returns {Promise<void>}
*/
async syncUserGroupsDifferential(newUserGroups) {
if (this.printOnly) {
logger.info(
'[USER GROUPS] Print-only mode: would perform differential sync'
);
this.printDifferentialSync(newUserGroups);
return;
}
try {
// Get existing entries from the table
const existingUserGroups = await this.getExistingUserGroups();
// Create sets for comparison
const existingSet = new Set(
existingUserGroups.map((ug) => `${ug.group}:${ug.username}`)
);
const newSet = new Set(
newUserGroups.map((ug) => `${ug.group}:${ug.username}`)
);
// Find entries to insert (in new but not in existing)
const toInsert = newUserGroups.filter(
(ug) => !existingSet.has(`${ug.group}:${ug.username}`)
);
// Find entries to delete (in existing but not in new)
const toDelete = existingUserGroups.filter(
(ug) => !newSet.has(`${ug.group}:${ug.username}`)
);
logger.info(
`Differential sync: ${toInsert.length} to insert, ${toDelete.length} to delete`
);
// Handle deletions with streaming buffer awareness
if (toDelete.length > 0) {
try {
await this.deleteUserGroups(toDelete);
} catch (error) {
if (error.message?.includes('streaming buffer')) {
logger.warn(
'Streaming buffer prevents DELETE - this is expected during testing. In production (daily runs), this won\'t occur.'
);
logger.info(
'Skipping deletions for now. Data will be consistent on next daily run.'
);
} else {
throw error;
}
}
}
// Insert new entries using load job (no streaming buffer issues for inserts)
if (toInsert.length > 0) {
await this.insertUserGroups(toInsert);
}
if (toInsert.length === 0 && toDelete.length === 0) {
logger.info('No changes needed - user groups are already up to date');
}
} catch (error) {
logger.error('Error performing differential sync:', {}, error);
throw error;
}
}
/**
* Gets existing user groups from the BigQuery table
* @returns {Promise<Array<{group: string, username: string}>>}
*/
async getExistingUserGroups() {
try {
const query = `SELECT \`group\`, username FROM \`${this.projectId}.${this.datasetId}.${this.tableId}\``;
const [rows] = await this.bigquery.query({ query });
logger.info(`Found ${rows.length} existing user group entries`);
return rows.map((row) => ({
group: row.group,
username: row.username,
}));
} catch (error) {
if (error.message?.includes('not found')) {
logger.info('Table does not exist yet, no existing entries');
return [];
}
logger.error('Error getting existing user groups:', {}, error);
throw error;
}
}
/**
* Deletes specific user groups from the BigQuery table
* @param {Array<{group: string, username: string}>} userGroupsToDelete
* @returns {Promise<void>}
*/
async deleteUserGroups(userGroupsToDelete) {
try {
const groups = userGroupsToDelete.map((ug) => ug.group);
const usernames = userGroupsToDelete.map((ug) => ug.username);
const query = `
DELETE
FROM \`${this.projectId}.${this.datasetId}.${this.tableId}\`
WHERE EXISTS (SELECT 1
FROM UNNEST(@groups) AS g WITH OFFSET AS pos
JOIN UNNEST(@usernames) AS u WITH OFFSET AS upos
ON pos = upos
WHERE \`group\` = g AND username = u
)
`;
const options = {
query,
params: { groups, usernames },
};
await this.bigquery.query(options);
logger.info(`Deleted ${userGroupsToDelete.length} user group entries`);
} catch (error) {
logger.error('Error deleting user groups:', {}, error);
throw error;
}
}
/**
* Prints differential sync information for print-only mode
* @param {Array<{group: string, username: string}>} newUserGroups
*/
async printDifferentialSync(newUserGroups) {
logger.info(
'[USER GROUPS] Print-only mode: would perform differential sync'
);
logger.info('[USER GROUPS] Would check existing entries in table');
logger.info(
`[USER GROUPS] Would compare with ${newUserGroups.length} new entries`
);
logger.info(
'[USER GROUPS] Would insert only new entries and delete removed entries'
);
// Group users by their groups for better display
const groupedUsers = newUserGroups.reduce((acc, ug) => {
if (!acc[ug.group]) {
acc[ug.group] = [];
}
acc[ug.group].push(ug.username);
return acc;
}, {});
logger.info('[USER GROUPS] New user groups to sync:');
for (const [group, usernames] of Object.entries(groupedUsers)) {
logger.info(` ${group}: ${usernames.join(', ')}`);
}
}
/**
* Inserts user groups into the BigQuery table using load job (avoids streaming buffer)
* @param {Array<{group: string, username: string}>} userGroups - Array of user group mappings
* @returns {Promise<void>}
*/
async insertUserGroups(userGroups) {
if (this.printOnly) {
logger.info(
`[USER GROUPS] Print-only mode: would insert ${userGroups.length} records`
);
return;
}
try {
// Transform data for BigQuery streaming insert
const rows = userGroups.map((ug) => ({
group: ug.group,
username: ug.username,
}));
// Use streaming insert - simpler and appropriate for small user group datasets
// Streaming buffer issues only occur during rapid testing, not daily production runs
await this.table.insert(rows);
logger.info(
`Inserted ${rows.length} user group mappings into BigQuery using streaming insert`
);
} catch (error) {
logger.error('Error inserting user groups:', {}, error);
throw error;
}
}
/**
* Prints user groups data in a readable format (for print-only mode)
* @param {Array<{group: string, username: string}>} userGroups - Array of user group mappings
*/
printUserGroupsData(userGroups) {
logger.info(
'[USER GROUPS] Print-only mode enabled - no BigQuery updates will be performed'
);
logger.info('');
if (userGroups.length === 0) {
logger.info('[USER GROUPS] No user groups found to process');
return;
}
// Group users by their groups for better display
const groupedUsers = userGroups.reduce((acc, ug) => {
if (!acc[ug.group]) {
acc[ug.group] = [];
}
acc[ug.group].push(ug.username);
return acc;
}, {});
logger.info('[USER GROUPS] Extracted user groups:');
for (const [group, usernames] of Object.entries(groupedUsers)) {
logger.info(` ${group}: ${usernames.join(', ')}`);
}
logger.info('');
logger.info(
`[USER GROUPS] Would insert ${userGroups.length} records into user_group table:`
);
// Count users per group
const groupCounts = Object.entries(groupedUsers).map(
([group, usernames]) => ` ${group}: ${usernames.length} users`
);
logger.info(groupCounts.join('\n'));
logger.info('');
logger.info('[USER GROUPS] Sample records that would be inserted:');
logger.info('| group | username |');
logger.info('|---------------|-----------------|');
// Show first few records from each group
const sampleRecords = [];
for (const [group, usernames] of Object.entries(groupedUsers)) {
const sampleUser = usernames[0];
sampleRecords.push(`| ${group.padEnd(13)} | ${sampleUser.padEnd(15)} |`);
if (sampleRecords.length >= 6) break; // Limit sample output
}
sampleRecords.forEach((record) => logger.info(record));
if (userGroups.length > sampleRecords.length) {
logger.info(
`... and ${userGroups.length - sampleRecords.length} more records`
);
}
}
}
export default UserGroupClient;

View file

@ -0,0 +1,275 @@
/**
* Tests for BigQuery client module
*/
import { jest } from '@jest/globals';
import { BigQueryClient } from '../src/bigquery-client.js';
// Mock the logger
jest.mock('../src/logger.js', () => ({
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn()
}));
// Mock fs
jest.mock('fs', () => ({
existsSync: jest.fn(() => true),
readFileSync: jest.fn(() => JSON.stringify({
project_id: 'test-project-id',
type: 'service_account',
private_key_id: 'key-id',
private_key: '-----BEGIN PRIVATE KEY-----\ntest-key\n-----END PRIVATE KEY-----\n',
client_email: 'test@test-project-id.iam.gserviceaccount.com',
client_id: '123456789',
auth_uri: 'https://accounts.google.com/o/oauth2/auth',
token_uri: 'https://oauth2.googleapis.com/token'
}))
}));
// Mock @google-cloud/bigquery
jest.mock('@google-cloud/bigquery', () => ({
BigQuery: jest.fn(() => ({
dataset: jest.fn(),
query: jest.fn()
}))
}));
describe('BigQueryClient', () => {
let bigqueryClient;
let mockBigQuery;
let mockDataset;
let mockTable;
beforeEach(() => {
mockTable = {
exists: jest.fn(() => [true]),
create: jest.fn(),
insert: jest.fn(() => [{}])
};
mockDataset = {
exists: jest.fn(() => [true]),
create: jest.fn(),
table: jest.fn(() => mockTable)
};
mockBigQuery = {
dataset: jest.fn(() => mockDataset),
query: jest.fn(() => [[]])
};
// Create client without calling constructor to avoid file check
bigqueryClient = Object.create(BigQueryClient.prototype);
bigqueryClient.bigquery = mockBigQuery;
bigqueryClient.projectId = 'test-project-id';
});
describe('getProjectId', () => {
test('should return the extracted project ID', () => {
expect(bigqueryClient.getProjectId()).toBe('test-project-id');
});
});
describe('getSchemaForMetricType', () => {
test('should return first_review table schema', () => {
const schema = bigqueryClient.getSchemaForMetricType('time_to_first_review');
expect(schema.fields).toBeTruthy();
});
test('should return pr_merge table schema', () => {
const schema = bigqueryClient.getSchemaForMetricType('time_to_merge');
expect(schema.fields).toBeTruthy();
});
test('should throw error for unknown metric type', () => {
expect(() => {
bigqueryClient.getSchemaForMetricType('unknown_metric');
}).toThrow('Unknown metric type: unknown_metric');
});
});
describe('getConfigurationForMetricType', () => {
test('should return first_review table configuration', () => {
const config = bigqueryClient.getConfigurationForMetricType('time_to_first_review');
expect(config).toBeTruthy();
});
test('should return pr_merge table configuration', () => {
const config = bigqueryClient.getConfigurationForMetricType('time_to_merge');
expect(config).toBeTruthy();
});
test('should throw error for unknown metric type configuration', () => {
expect(() => {
bigqueryClient.getConfigurationForMetricType('unknown_metric');
}).toThrow('Unknown metric type for table configuration: unknown_metric');
});
});
describe('transformMetricsToRow', () => {
test('should transform time_to_first_review metrics', () => {
const metrics = {
metricType: 'time_to_first_review',
reviewDate: '2023-06-15',
prCreator: 'testuser',
prUrl: 'https://github.com/owner/repo/pull/123',
pickupTimeSeconds: 7200,
repository: 'owner/repo',
prNumber: 123,
targetBranch: 'main',
readyTime: new Date('2023-06-15T10:00:00Z'),
firstReviewTime: new Date('2023-06-15T12:00:00Z')
};
const row = bigqueryClient.transformMetricsToRow(metrics);
expect(row).toEqual({
review_date: '2023-06-15',
pr_creator: 'testuser',
pr_url: 'https://github.com/owner/repo/pull/123',
pickup_time_seconds: 7200,
repository: 'owner/repo',
pr_number: 123,
target_branch: 'main',
ready_time: '2023-06-15T10:00:00.000Z',
first_review_time: '2023-06-15T12:00:00.000Z'
});
});
test('should transform time_to_merge metrics', () => {
const metrics = {
metricType: 'time_to_merge',
mergeDate: '2023-06-15',
prCreator: 'testuser',
prUrl: 'https://github.com/owner/repo/pull/123',
mergeTimeSeconds: 16200,
repository: 'owner/repo',
prNumber: 123,
targetBranch: 'main',
readyTime: new Date('2023-06-15T10:00:00Z'),
mergeTime: new Date('2023-06-15T14:30:00Z')
};
const row = bigqueryClient.transformMetricsToRow(metrics);
expect(row).toEqual({
merge_date: '2023-06-15',
pr_creator: 'testuser',
pr_url: 'https://github.com/owner/repo/pull/123',
merge_time_seconds: 16200,
repository: 'owner/repo',
pr_number: 123,
target_branch: 'main',
ready_time: '2023-06-15T10:00:00.000Z',
merge_time: '2023-06-15T14:30:00.000Z'
});
});
test('should throw error for unknown metric type', () => {
const metrics = {
metricType: 'unknown_type'
};
expect(() => {
bigqueryClient.transformMetricsToRow(metrics);
}).toThrow('Unknown metric type: unknown_type');
});
});
describe('createTableIfNotExists', () => {
test('should create table with correct configuration for first_review', async () => {
mockTable.exists.mockResolvedValue([false]);
const schema = { fields: [] };
await bigqueryClient.createTableIfNotExists('test_dataset', 'first_review', schema, 'time_to_first_review');
expect(mockTable.create).toHaveBeenCalled();
});
test('should create table with correct configuration for pr_merge', async () => {
mockTable.exists.mockResolvedValue([false]);
const schema = { fields: [] };
await bigqueryClient.createTableIfNotExists('test_dataset', 'pr_merge', schema, 'time_to_merge');
expect(mockTable.create).toHaveBeenCalled();
});
test('should not create table if it already exists', async () => {
mockTable.exists.mockResolvedValue([true]);
const schema = { fields: [] };
await bigqueryClient.createTableIfNotExists('test_dataset', 'first_review', schema, 'time_to_first_review');
expect(mockTable.create).not.toHaveBeenCalled();
});
});
describe('uploadMetrics', () => {
test('should upload metrics with correct schema', async () => {
const metrics = [
{
metricType: 'time_to_first_review',
prNumber: 123,
reviewDate: '2023-06-15',
prCreator: 'testuser',
prUrl: 'https://github.com/owner/repo/pull/123',
pickupTimeSeconds: 7200,
repository: 'owner/repo',
targetBranch: 'main',
readyTime: new Date('2023-06-15T10:00:00Z'),
firstReviewTime: new Date('2023-06-15T12:00:00Z')
}
];
await bigqueryClient.uploadMetrics('test_dataset', 'pr_first_review', metrics);
expect(mockTable.insert).toHaveBeenCalledWith([
{
review_date: '2023-06-15',
pr_creator: 'testuser',
pr_url: 'https://github.com/owner/repo/pull/123',
pickup_time_seconds: 7200,
repository: 'owner/repo',
pr_number: 123,
target_branch: 'main',
ready_time: '2023-06-15T10:00:00.000Z',
first_review_time: '2023-06-15T12:00:00.000Z'
}
]);
});
test('should handle empty metrics array', async () => {
await bigqueryClient.uploadMetrics('test_dataset', 'pr_first_review', []);
expect(mockTable.insert).not.toHaveBeenCalled();
});
test('should filter out existing metrics', async () => {
const metrics = [
{
metricType: 'time_to_first_review',
prNumber: 123,
reviewDate: '2023-06-15',
prCreator: 'testuser',
prUrl: 'https://github.com/owner/repo/pull/123',
pickupTimeSeconds: 7200,
repository: 'owner/repo',
targetBranch: 'main',
readyTime: new Date('2023-06-15T10:00:00Z'),
firstReviewTime: new Date('2023-06-15T12:00:00Z')
}
];
// Mock existing metrics check to return that PR 123 already exists
mockBigQuery.query.mockResolvedValue([[{ pr_number: 123 }]]);
await bigqueryClient.uploadMetrics('test_dataset', 'pr_first_review', metrics);
expect(mockTable.insert).not.toHaveBeenCalled();
});
});
});

View file

@ -0,0 +1,210 @@
/**
* Tests for bot detection functionality
*/
import { jest } from '@jest/globals';
// Mock the logger
const mockLogger = {
info: jest.fn(),
debug: jest.fn(),
warn: jest.fn(),
error: jest.fn()
};
jest.unstable_mockModule('../src/logger.js', () => ({
default: mockLogger
}));
const { GitHubClient } = await import('../src/github-client.js');
describe('Bot Detection', () => {
let githubClient;
beforeEach(() => {
githubClient = new GitHubClient('fake-token');
jest.clearAllMocks();
});
describe('identifyBotUser', () => {
test('should identify GitHub API Bot type', () => {
const botUser = {
login: 'coderabbitai[bot]',
type: 'Bot',
name: 'CodeRabbit AI',
bio: 'AI-powered code review assistant'
};
// Access the function through the module (it's not exported as a method)
// We'll test it indirectly through filterBotReviews
const reviews = [
{
user: botUser,
state: 'COMMENTED',
submitted_at: '2023-06-15T12:00:00Z'
}
];
const filtered = githubClient.filterBotReviews(reviews, true);
expect(filtered).toHaveLength(0);
});
test('should identify bot by username patterns', () => {
const testCases = [
{ login: 'dependabot[bot]', type: 'User' },
{ login: 'renovate[bot]', type: 'User' },
{ login: 'github-actions[bot]', type: 'User' },
{ login: 'codecov-commenter', type: 'User' },
{ login: 'coderabbitai', type: 'User' },
{ login: 'sonarcloud[bot]', type: 'User' },
{ login: 'snyk-bot', type: 'User' },
{ login: 'greenkeeper[bot]', type: 'User' },
{ login: 'semantic-release-bot', type: 'User' },
{ login: 'stale[bot]', type: 'User' },
{ login: 'imgbot[bot]', type: 'User' },
{ login: 'allcontributors[bot]', type: 'User' },
{ login: 'whitesource-bolt', type: 'User' },
{ login: 'deepsource-autofix[bot]', type: 'User' }
];
testCases.forEach(({ login, type }) => {
const reviews = [
{
user: { login, type },
state: 'COMMENTED',
submitted_at: '2023-06-15T12:00:00Z'
}
];
const filtered = githubClient.filterBotReviews(reviews, true);
expect(filtered).toHaveLength(0);
});
});
test('should not filter human users', () => {
const humanUser = {
login: 'johndoe',
type: 'User',
name: 'John Doe',
bio: 'Software engineer'
};
const reviews = [
{
user: humanUser,
state: 'APPROVED',
submitted_at: '2023-06-15T12:00:00Z'
}
];
const filtered = githubClient.filterBotReviews(reviews, true);
expect(filtered).toHaveLength(1);
expect(filtered[0].user.login).toBe('johndoe');
});
test('should not filter when excludeBots is false', () => {
const botUser = {
login: 'coderabbitai[bot]',
type: 'Bot'
};
const humanUser = {
login: 'johndoe',
type: 'User'
};
const reviews = [
{
user: botUser,
state: 'COMMENTED',
submitted_at: '2023-06-15T12:00:00Z'
},
{
user: humanUser,
state: 'APPROVED',
submitted_at: '2023-06-15T13:00:00Z'
}
];
const filtered = githubClient.filterBotReviews(reviews, false);
expect(filtered).toHaveLength(2);
});
test('should handle mixed bot and human reviews', () => {
const reviews = [
{
user: { login: 'coderabbitai[bot]', type: 'Bot' },
state: 'COMMENTED',
submitted_at: '2023-06-15T12:00:00Z'
},
{
user: { login: 'johndoe', type: 'User' },
state: 'APPROVED',
submitted_at: '2023-06-15T13:00:00Z'
},
{
user: { login: 'dependabot[bot]', type: 'User' },
state: 'COMMENTED',
submitted_at: '2023-06-15T14:00:00Z'
},
{
user: { login: 'janedoe', type: 'User' },
state: 'CHANGES_REQUESTED',
submitted_at: '2023-06-15T15:00:00Z'
}
];
const filtered = githubClient.filterBotReviews(reviews, true);
expect(filtered).toHaveLength(2);
expect(filtered[0].user.login).toBe('johndoe');
expect(filtered[1].user.login).toBe('janedoe');
});
test('should log filtering activity', () => {
const reviews = [
{
user: { login: 'coderabbitai[bot]', type: 'Bot' },
state: 'COMMENTED',
submitted_at: '2023-06-15T12:00:00Z'
},
{
user: { login: 'johndoe', type: 'User' },
state: 'APPROVED',
submitted_at: '2023-06-15T13:00:00Z'
}
];
githubClient.filterBotReviews(reviews, true);
expect(mockLogger.info).toHaveBeenCalledWith(
'Filtered out 1 bot reviews from 2 total reviews'
);
});
test('should handle empty review array', () => {
const filtered = githubClient.filterBotReviews([], true);
expect(filtered).toHaveLength(0);
});
test('should handle reviews with no bots', () => {
const reviews = [
{
user: { login: 'johndoe', type: 'User' },
state: 'APPROVED',
submitted_at: '2023-06-15T13:00:00Z'
},
{
user: { login: 'janedoe', type: 'User' },
state: 'CHANGES_REQUESTED',
submitted_at: '2023-06-15T15:00:00Z'
}
];
const filtered = githubClient.filterBotReviews(reviews, true);
expect(filtered).toHaveLength(2);
expect(mockLogger.info).not.toHaveBeenCalledWith(
expect.stringContaining('Filtered out')
);
});
});
});

View file

@ -0,0 +1,421 @@
/**
* Tests for configuration module
*/
import { jest } from "@jest/globals";
// Mock the logger
const mockLogger = {
default: {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
},
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
};
// Mock fs
const mockFs = {
default: {
existsSync: jest.fn(),
readFileSync: jest.fn(),
},
existsSync: jest.fn(),
readFileSync: jest.fn(),
};
// Mock dotenv
const mockDotenv = {
default: {
config: jest.fn(),
},
config: jest.fn(),
};
// Mock modules before importing
jest.unstable_mockModule("../src/logger.js", () => mockLogger);
jest.unstable_mockModule("fs", () => mockFs);
jest.unstable_mockModule("dotenv", () => mockDotenv);
// Now import the module under test
const { loadConfig, validateConfig } = await import("../src/config.js");
describe("Config", () => {
let originalEnv;
beforeEach(() => {
// Save original environment
originalEnv = { ...process.env };
// Clear environment variables. This is the recommended approach to prevent performance hit.
Reflect.deleteProperty(process.env, "GITHUB_TOKEN");
Reflect.deleteProperty(process.env, "SERVICE_ACCOUNT_KEY_PATH");
Reflect.deleteProperty(process.env, "BIGQUERY_DATASET_ID");
Reflect.deleteProperty(process.env, "REPOSITORIES");
Reflect.deleteProperty(process.env, "LOOKBACK_DAYS");
Reflect.deleteProperty(process.env, "TARGET_BRANCH");
Reflect.deleteProperty(process.env, "PRINT_ONLY");
Reflect.deleteProperty(process.env, "ENABLED_METRICS");
Reflect.deleteProperty(process.env, "TIME_TO_FIRST_REVIEW_TABLE");
Reflect.deleteProperty(process.env, "TIME_TO_MERGE_TABLE");
// Reset all mocks
jest.clearAllMocks();
// Mock fs.existsSync to return true by default (for config.json and user group file)
mockFs.existsSync.mockReturnValue(true);
mockFs.default.existsSync.mockReturnValue(true);
// Mock fs.readFileSync to return a default config
mockFs.readFileSync.mockReturnValue(
JSON.stringify({
repositories: ["owner/repo1", "owner/repo2"],
targetBranch: "main",
bigQueryDatasetId: "test_dataset",
lookbackDays: 5,
serviceAccountKeyPath: "./service-account-key.json",
printOnly: false,
userGroupEnabled: true,
userGroupFilepath: "../../../handbook/company/product-groups.md",
})
);
});
afterEach(() => {
// Restore original environment
process.env = originalEnv;
});
describe("loadConfig", () => {
test("should load default configuration", () => {
process.env.GITHUB_TOKEN = "test-token";
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
process.env.BIGQUERY_DATASET_ID = "test_dataset";
process.env.REPOSITORIES = "owner/repo1,owner/repo2";
const config = loadConfig();
expect(config).toEqual({
githubToken: "test-token",
serviceAccountKeyPath: "/path/to/key.json",
bigQueryDatasetId: "test_dataset",
repositories: ["owner/repo1", "owner/repo2"],
lookbackDays: 5,
targetBranch: "main",
printOnly: false,
userGroupEnabled: true,
userGroupFilepath: "../../../handbook/company/product-groups.md",
excludeBotReviews: true,
metrics: {
timeToFirstReview: {
enabled: true,
tableName: "pr_first_review",
},
timeToMerge: {
enabled: true,
tableName: "pr_merge",
},
},
});
});
test("should override defaults with environment variables", () => {
process.env.GITHUB_TOKEN = "test-token";
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
process.env.BIGQUERY_DATASET_ID = "test_dataset";
process.env.REPOSITORIES = "owner/repo";
process.env.LOOKBACK_DAYS = "14";
process.env.TARGET_BRANCH = "develop";
process.env.PRINT_ONLY = "true";
process.env.ENABLED_METRICS = "time_to_first_review";
process.env.TIME_TO_FIRST_REVIEW_TABLE = "custom_first_review";
process.env.TIME_TO_MERGE_TABLE = "custom_pr_merge";
const config = loadConfig();
expect(config).toEqual({
githubToken: "test-token",
serviceAccountKeyPath: "/path/to/key.json",
bigQueryDatasetId: "test_dataset",
repositories: ["owner/repo"],
lookbackDays: 5,
targetBranch: "develop",
printOnly: true,
userGroupEnabled: true,
userGroupFilepath: "../../../handbook/company/product-groups.md",
excludeBotReviews: true,
metrics: {
timeToFirstReview: {
enabled: true,
tableName: "custom_first_review",
},
timeToMerge: {
enabled: false,
tableName: "custom_pr_merge",
},
},
});
});
test("should handle multiple enabled metrics", () => {
process.env.GITHUB_TOKEN = "test-token";
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
process.env.BIGQUERY_DATASET_ID = "test_dataset";
process.env.REPOSITORIES = "owner/repo";
process.env.ENABLED_METRICS = "time_to_first_review,time_to_merge";
const config = loadConfig();
expect(config.metrics).toEqual({
timeToFirstReview: {
enabled: true,
tableName: "pr_first_review",
},
timeToMerge: {
enabled: true,
tableName: "pr_merge",
},
});
});
test("should handle only time_to_merge enabled", () => {
process.env.GITHUB_TOKEN = "test-token";
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
process.env.BIGQUERY_DATASET_ID = "test_dataset";
process.env.REPOSITORIES = "owner/repo";
process.env.ENABLED_METRICS = "time_to_merge";
const config = loadConfig();
expect(config.metrics).toEqual({
timeToFirstReview: {
enabled: false,
tableName: "pr_first_review",
},
timeToMerge: {
enabled: true,
tableName: "pr_merge",
},
});
});
test("should trim whitespace from repositories", () => {
process.env.GITHUB_TOKEN = "test-token";
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
process.env.BIGQUERY_DATASET_ID = "test_dataset";
process.env.REPOSITORIES = " owner/repo1 , owner/repo2 , owner/repo3 ";
const config = loadConfig();
expect(config.repositories).toEqual([
"owner/repo1",
"owner/repo2",
"owner/repo3",
]);
});
test("should trim whitespace from enabled metrics", () => {
process.env.GITHUB_TOKEN = "test-token";
process.env.SERVICE_ACCOUNT_KEY_PATH = "/path/to/key.json";
process.env.BIGQUERY_DATASET_ID = "test_dataset";
process.env.REPOSITORIES = "owner/repo";
process.env.ENABLED_METRICS = " time_to_first_review , time_to_merge ";
const config = loadConfig();
expect(config.metrics).toEqual({
timeToFirstReview: {
enabled: true,
tableName: "pr_first_review",
},
timeToMerge: {
enabled: true,
tableName: "pr_merge",
},
});
});
});
describe("validateConfig", () => {
const baseValidConfig = {
githubToken: "test-token",
serviceAccountKeyPath: "/path/to/key.json",
bigQueryDatasetId: "test_dataset",
repositories: ["owner/repo"],
lookbackDays: 30,
targetBranch: "main",
printOnly: false,
userGroupEnabled: true,
userGroupFilepath: "../../../handbook/company/product-groups.md",
excludeBotReviews: true,
metrics: {
timeToFirstReview: {
enabled: true,
tableName: "pr_first_review",
},
timeToMerge: {
enabled: true,
tableName: "pr_merge",
},
},
};
test("should validate correct configuration", () => {
expect(validateConfig(baseValidConfig)).toBe(true);
});
test("should return false for missing GitHub token", () => {
const config = { ...baseValidConfig, githubToken: "" };
expect(validateConfig(config)).toBe(false);
});
test("should return false for missing service account key path in non-print mode", () => {
const config = { ...baseValidConfig, serviceAccountKeyPath: "" };
expect(validateConfig(config)).toBe(false);
});
test("should not require service account key path in print-only mode", () => {
const config = {
...baseValidConfig,
serviceAccountKeyPath: "",
printOnly: true,
};
expect(validateConfig(config)).toBe(true);
});
test("should not require BigQuery dataset ID in print-only mode", () => {
const config = {
...baseValidConfig,
bigQueryDatasetId: "",
printOnly: true,
};
expect(validateConfig(config)).toBe(true);
});
test("should return false for empty repositories array", () => {
const config = { ...baseValidConfig, repositories: [] };
expect(validateConfig(config)).toBe(false);
});
test("should return false for invalid repository format", () => {
const config = { ...baseValidConfig, repositories: ["invalid-repo"] };
expect(validateConfig(config)).toBe(false);
});
test("should validate lookback days correctly", () => {
const config = { ...baseValidConfig, lookbackDays: 5 };
expect(validateConfig(config)).toBe(true);
});
test("should return false when no metrics are enabled", () => {
const config = {
...baseValidConfig,
metrics: {
timeToFirstReview: { enabled: false, tableName: "pr_first_review" },
timeToMerge: { enabled: false, tableName: "pr_merge" },
},
};
expect(validateConfig(config)).toBe(false);
});
test("should return false for missing table name when metric is enabled", () => {
const config = {
...baseValidConfig,
metrics: {
timeToFirstReview: { enabled: true, tableName: "" },
timeToMerge: { enabled: false, tableName: "pr_merge" },
},
};
expect(validateConfig(config)).toBe(false);
});
test("should allow missing table name when metric is disabled", () => {
const config = {
...baseValidConfig,
metrics: {
timeToFirstReview: { enabled: false, tableName: "" },
timeToMerge: { enabled: true, tableName: "pr_merge" },
},
};
expect(validateConfig(config)).toBe(true);
});
test("should validate multiple valid repositories", () => {
const config = {
...baseValidConfig,
repositories: ["owner1/repo1", "owner2/repo2", "owner3/repo3"],
};
expect(validateConfig(config)).toBe(true);
});
test("should throw error for mixed valid and invalid repositories", () => {
const config = {
...baseValidConfig,
repositories: ["owner1/repo1", "invalid-repo", "owner3/repo3"],
};
expect(validateConfig(config)).toBe(false);
});
test("should validate when userGroupEnabled is true and file exists", () => {
const config = {
...baseValidConfig,
userGroupEnabled: true,
userGroupFilepath: "../../../handbook/company/product-groups.md",
};
// Mock file exists - need to reset and set up the mock properly
mockFs.existsSync.mockReset();
mockFs.default.existsSync.mockReset();
mockFs.existsSync.mockReturnValue(true);
mockFs.default.existsSync.mockReturnValue(true);
expect(validateConfig(config)).toBe(true);
});
test("should return false when userGroupEnabled is true but file does not exist", () => {
const config = {
...baseValidConfig,
userGroupEnabled: true,
userGroupFilepath: "../../../handbook/company/product-groups.md",
};
// Mock file does not exist
mockFs.existsSync.mockReset();
mockFs.default.existsSync.mockReset();
mockFs.existsSync.mockReturnValue(false);
mockFs.default.existsSync.mockReturnValue(false);
expect(validateConfig(config)).toBe(false);
});
test("should validate when userGroupEnabled is false regardless of file existence", () => {
const config = {
...baseValidConfig,
userGroupEnabled: false,
userGroupFilepath: "../../../handbook/company/product-groups.md",
};
// Mock file does not exist
mockFs.existsSync.mockReset();
mockFs.default.existsSync.mockReset();
mockFs.existsSync.mockReturnValue(false);
mockFs.default.existsSync.mockReturnValue(false);
expect(validateConfig(config)).toBe(true);
});
test("should return false when userGroupEnabled is true but userGroupFilepath is not set", () => {
const config = {
...baseValidConfig,
userGroupEnabled: true,
userGroupFilepath: undefined,
};
expect(validateConfig(config)).toBe(false);
});
});
});

View file

@ -0,0 +1,232 @@
/**
* Tests the complete workflow from configuration to metric collection
*/
import { jest } from '@jest/globals';
import { loadConfig } from '../src/config.js';
import { MetricsCollector } from '../src/metrics-collector.js';
// Mock the logger
jest.mock('../src/logger.js', () => ({
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn()
}));
// Mock GitHubClient
jest.mock('../src/github-client.js', () => {
return jest.fn().mockImplementation(() => ({
fetchPullRequests: jest.fn(),
fetchPRTimelineEvents: jest.fn(),
fetchPRReviewEvents: jest.fn(),
filterBotReviews: jest.fn((reviews, excludeBots) => excludeBots ? [] : reviews),
calculatePickupTime: jest.fn(),
calculateTimeToMerge: jest.fn()
}));
});
// Mock BigQueryClient
jest.mock('../src/bigquery-client.js', () => ({
BigQueryClient: jest.fn().mockImplementation(() => ({
uploadMetrics: jest.fn()
}))
}));
describe('End-to-End Time to Merge Workflow', () => {
beforeEach(() => {
// Set environment variables for testing
process.env.GITHUB_TOKEN = 'test-token';
process.env.SERVICE_ACCOUNT_KEY_PATH = '/fake/path';
process.env.BIGQUERY_DATASET_ID = 'test_dataset';
process.env.REPOSITORIES = 'owner/repo';
process.env.PRINT_ONLY = 'true';
});
test('should collect both Time to First Review and Time to Merge metrics', async () => {
// Load configuration
const config = loadConfig();
// Verify both metrics are enabled
expect(config.metrics.timeToFirstReview.enabled).toBe(true);
expect(config.metrics.timeToMerge.enabled).toBe(true);
// Create metrics collector
const metricsCollector = new MetricsCollector(config);
// Mock the GitHub client methods
metricsCollector.githubClient = {
fetchPullRequests: jest.fn().mockResolvedValue([
{
number: 123,
html_url: 'https://github.com/owner/repo/pull/123',
user: { login: 'testuser' },
base: {
ref: 'main',
repo: {
owner: { login: 'owner' },
name: 'repo'
}
},
head: { repo: { full_name: 'owner/repo' } },
state: 'closed',
merged_at: '2023-06-15T14:30:00Z'
}
]),
fetchPRTimelineEvents: jest.fn().mockResolvedValue([
{
event: 'ready_for_review',
created_at: '2023-06-15T10:00:00Z'
}
]),
fetchPRReviewEvents: jest.fn().mockResolvedValue([
{
submitted_at: '2023-06-15T12:00:00Z',
state: 'approved',
user: { login: 'reviewer1' }
}
]),
calculatePickupTime: jest.fn().mockReturnValue({
metricType: 'time_to_first_review',
reviewDate: '2023-06-15',
prCreator: 'testuser',
prUrl: 'https://github.com/owner/repo/pull/123',
pickupTimeSeconds: 7200,
repository: 'owner/repo',
prNumber: 123,
targetBranch: 'main',
readyTime: new Date('2023-06-15T10:00:00Z'),
firstReviewTime: new Date('2023-06-15T12:00:00Z')
}),
calculateTimeToMerge: jest.fn().mockReturnValue({
metricType: 'time_to_merge',
mergeDate: '2023-06-15',
prCreator: 'testuser',
prUrl: 'https://github.com/owner/repo/pull/123',
mergeTimeSeconds: 16200,
repository: 'owner/repo',
prNumber: 123,
targetBranch: 'main',
readyTime: new Date('2023-06-15T10:00:00Z'),
mergeTime: new Date('2023-06-15T14:30:00Z')
}),
filterBotReviews: jest.fn((reviews, excludeBots) => excludeBots ? [] : reviews)
};
// Collect metrics for a single repository
const metrics = await metricsCollector.collectRepositoryMetrics('owner/repo');
// Verify that both metrics were collected
expect(metrics).toHaveLength(2);
const firstReviewMetric = metrics.find(m => m.metricType === 'time_to_first_review');
const mergeMetric = metrics.find(m => m.metricType === 'time_to_merge');
expect(firstReviewMetric).toBeDefined();
expect(firstReviewMetric.pickupTimeSeconds).toBe(7200);
expect(firstReviewMetric.prNumber).toBe(123);
expect(mergeMetric).toBeDefined();
expect(mergeMetric.mergeTimeSeconds).toBe(16200);
expect(mergeMetric.prNumber).toBe(123);
// Verify that both calculation methods were called
expect(metricsCollector.githubClient.calculatePickupTime).toHaveBeenCalled();
expect(metricsCollector.githubClient.calculateTimeToMerge).toHaveBeenCalled();
});
test('should handle configuration with only Time to Merge enabled', async () => {
// Override environment to enable only Time to Merge
process.env.ENABLED_METRICS = 'time_to_merge';
const config = loadConfig();
expect(config.metrics.timeToFirstReview.enabled).toBe(false);
expect(config.metrics.timeToMerge.enabled).toBe(true);
const metricsCollector = new MetricsCollector(config);
// Mock the GitHub client
metricsCollector.githubClient = {
calculateTimeToMerge: jest.fn().mockReturnValue({
metricType: 'time_to_merge',
mergeDate: '2023-06-15',
prCreator: 'testuser',
prUrl: 'https://github.com/owner/repo/pull/123',
mergeTimeSeconds: 16200,
repository: 'owner/repo',
prNumber: 123,
targetBranch: 'main',
readyTime: new Date('2023-06-15T10:00:00Z'),
mergeTime: new Date('2023-06-15T14:30:00Z')
}),
calculatePickupTime: jest.fn().mockReturnValue(null) // Should not be called
};
// Mock PR data
const mockPR = {
number: 123,
html_url: 'https://github.com/owner/repo/pull/123',
user: { login: 'testuser' },
base: {
ref: 'main',
repo: {
owner: { login: 'owner' },
name: 'repo'
}
},
head: { repo: { full_name: 'owner/repo' } },
state: 'closed',
merged_at: '2023-06-15T14:30:00Z'
};
const mockTimelineEvents = [
{
event: 'ready_for_review',
created_at: '2023-06-15T10:00:00Z'
}
];
const mockReviewEvents = [];
// Test collectPRMetrics with only merge enabled
const metrics = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
// Should only collect Time to Merge metric
expect(metrics).toHaveLength(1);
expect(metrics[0].metricType).toBe('time_to_merge');
});
test('should group and route metrics to correct tables', () => {
const config = loadConfig();
const metricsCollector = new MetricsCollector(config);
const mixedMetrics = [
{
metricType: 'time_to_first_review',
prNumber: 123,
pickupTimeSeconds: 7200
},
{
metricType: 'time_to_merge',
prNumber: 123,
mergeTimeSeconds: 16200
},
{
metricType: 'time_to_first_review',
prNumber: 124,
pickupTimeSeconds: 3600
}
];
// Test grouping
const grouped = metricsCollector.groupMetricsByType(mixedMetrics);
expect(grouped.time_to_first_review).toHaveLength(2);
expect(grouped.time_to_merge).toHaveLength(1);
// Test table name mapping
expect(metricsCollector.getTableNameForMetricType('time_to_first_review')).toBe('pr_first_review');
expect(metricsCollector.getTableNameForMetricType('time_to_merge')).toBe('pr_merge');
});
});

View file

@ -0,0 +1,547 @@
import { jest } from '@jest/globals';
import GitHubClient from '../src/github-client.js';
// Mock the logger to avoid console output during tests
jest.mock('../src/logger.js', () => ({
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
default: {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
}
}));
describe('GitHubClient - Time to first review (pickup time)', () => {
let githubClient;
beforeEach(() => {
// Create a new instance of GitHubClient for each test
githubClient = new GitHubClient('fake-token');
// Mock the Octokit instance
githubClient.octokit = {
rest: {
pulls: {
list: jest.fn(),
listReviews: jest.fn(),
},
issues: {
listEventsForTimeline: jest.fn(),
}
}
};
});
describe('calculatePickupTime', () => {
// Table-driven test cases for calculatePickupTime
const testCases = [
{
name: 'PR created as non-draft with one review',
pr: {
number: 123,
html_url: 'https://github.com/owner/repo/pull/123',
draft: false,
created_at: '2023-05-10T10:00:00Z',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-10T11:30:00Z' }
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 123,
prUrl: 'https://github.com/owner/repo/pull/123',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-10T10:00:00Z'),
firstReviewTime: new Date('2023-05-10T11:30:00Z'),
reviewDate: '2023-05-10',
pickupTimeSeconds: 5400, // 1.5 hours = 5400 seconds
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR created as draft, then marked as ready for review, then reviewed',
pr: {
number: 124,
html_url: 'https://github.com/owner/repo/pull/124',
draft: true,
created_at: '2023-05-11T09:00:00Z',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [
{
event: 'ready_for_review',
created_at: '2023-05-11T10:00:00Z'
}
],
reviewEvents: [
{ submitted_at: '2023-05-11T11:00:00Z' }
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 124,
prUrl: 'https://github.com/owner/repo/pull/124',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-11T10:00:00Z'),
firstReviewTime: new Date('2023-05-11T11:00:00Z'),
reviewDate: '2023-05-11',
pickupTimeSeconds: 3600, // 1 hour = 3600 seconds
readyEventType: 'ready_for_review event'
}
},
{
name: 'PR with multiple ready_for_review events - should use the latest one',
pr: {
number: 125,
html_url: 'https://github.com/owner/repo/pull/125',
draft: true,
created_at: '2023-05-12T09:00:00Z',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [
{
event: 'ready_for_review',
created_at: '2023-05-12T10:00:00Z'
},
{
event: 'convert_to_draft',
created_at: '2023-05-12T11:00:00Z'
},
{
event: 'ready_for_review',
created_at: '2023-05-12T12:00:00Z'
}
],
reviewEvents: [
{ submitted_at: '2023-05-12T13:00:00Z' }
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 125,
prUrl: 'https://github.com/owner/repo/pull/125',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-12T12:00:00Z'),
firstReviewTime: new Date('2023-05-12T13:00:00Z'),
reviewDate: '2023-05-12',
pickupTimeSeconds: 3600, // 1 hour = 3600 seconds
readyEventType: 'ready_for_review event'
}
},
{
name: 'PR with ready_for_review event after the first review',
pr: {
number: 126,
html_url: 'https://github.com/owner/repo/pull/126',
draft: false,
created_at: '2023-05-16T09:00:00Z',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [
{
event: 'convert_to_draft',
created_at: '2023-05-16T10:00:00Z'
},
{
event: 'ready_for_review',
created_at: '2023-05-16T12:00:00Z'
}
],
reviewEvents: [
{ submitted_at: '2023-05-16T11:00:00Z' }
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 126,
prUrl: 'https://github.com/owner/repo/pull/126',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-16T09:00:00Z'),
firstReviewTime: new Date('2023-05-16T11:00:00Z'),
reviewDate: '2023-05-16',
pickupTimeSeconds: 7200, // 2 hours = 7200 seconds
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR with no ready_for_review events and created as draft',
pr: {
number: 127,
html_url: 'https://github.com/owner/repo/pull/127',
draft: true,
created_at: '2023-05-14T09:00:00Z',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-14T11:00:00Z' }
],
expected: null // Should return null because no ready event was found
},
{
name: 'PR with no reviews',
pr: {
number: 128,
html_url: 'https://github.com/owner/repo/pull/128',
draft: false,
created_at: '2023-05-15T09:00:00Z',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: null // Should return null because no reviews were found
},
{
name: 'PR with multiple reviews - only first one should be counted',
pr: {
number: 129,
html_url: 'https://github.com/owner/repo/pull/129',
draft: false,
created_at: '2023-05-16T09:00:00Z',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-16T10:00:00Z' }, // First review - should be used
{ submitted_at: '2023-05-16T11:00:00Z' }, // Second review - should be ignored
{ submitted_at: '2023-05-16T12:00:00Z' } // Third review - should be ignored
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 129,
prUrl: 'https://github.com/owner/repo/pull/129',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-16T09:00:00Z'),
firstReviewTime: new Date('2023-05-16T10:00:00Z'),
reviewDate: '2023-05-16',
pickupTimeSeconds: 3600, // 1 hour = 3600 seconds
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Saturday, reviewed on Sunday 3 weeks later (should exclude weekend days)',
pr: {
number: 136,
html_url: 'https://github.com/owner/repo/pull/136',
draft: false,
created_at: '2023-05-20T14:00:00Z', // Saturday
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-06-11T14:00:00Z' } // Sunday, 3 weeks later
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 136,
prUrl: 'https://github.com/owner/repo/pull/136',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-20T14:00:00Z'),
firstReviewTime: new Date('2023-06-11T14:00:00Z'),
reviewDate: '2023-06-11',
pickupTimeSeconds: 1296000, // 15 days = 1296000 seconds (3 weeks of 5 working days)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Sunday, reviewed on Monday (should use end of Sunday as ready time)',
pr: {
number: 134,
html_url: 'https://github.com/owner/repo/pull/134',
draft: false,
created_at: '2023-05-21T14:00:00Z', // Sunday
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-22T10:00:00Z' } // Monday
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 134,
prUrl: 'https://github.com/owner/repo/pull/134',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-21T14:00:00Z'),
firstReviewTime: new Date('2023-05-22T10:00:00Z'),
reviewDate: '2023-05-22',
pickupTimeSeconds: 36000, // 10 hours = 36000 seconds (from end of Sunday to Monday 10am)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Sunday, reviewed on next Saturday (should exclude weekend days)',
pr: {
number: 135,
html_url: 'https://github.com/owner/repo/pull/135',
draft: false,
created_at: '2023-05-21T14:00:00Z', // Sunday
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-27T14:00:00Z' } // Saturday
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 135,
prUrl: 'https://github.com/owner/repo/pull/135',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-21T14:00:00Z'),
firstReviewTime: new Date('2023-05-27T14:00:00Z'),
reviewDate: '2023-05-27',
pickupTimeSeconds: 432000, // 5 days = 432000 seconds (6 days - 1 weekend day)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Friday, reviewed on Monday (should subtract weekend days)',
pr: {
number: 130,
html_url: 'https://github.com/owner/repo/pull/130',
draft: false,
created_at: '2023-05-19T14:00:00Z', // Friday
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-22T10:00:00Z' } // Monday
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 130,
prUrl: 'https://github.com/owner/repo/pull/130',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-19T14:00:00Z'),
firstReviewTime: new Date('2023-05-22T10:00:00Z'),
reviewDate: '2023-05-22',
pickupTimeSeconds: 72000, // 20 hours = 72000 seconds (3 days - 2 weekend days)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Saturday, reviewed on Monday (should use end of Sunday as ready time)',
pr: {
number: 131,
html_url: 'https://github.com/owner/repo/pull/131',
draft: false,
created_at: '2023-05-20T14:00:00Z', // Saturday
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-22T10:00:00Z' } // Monday
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 131,
prUrl: 'https://github.com/owner/repo/pull/131',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-20T14:00:00Z'),
firstReviewTime: new Date('2023-05-22T10:00:00Z'),
reviewDate: '2023-05-22',
pickupTimeSeconds: 36000, // 10 hours = 36000 seconds (from end of Sunday to Monday 10am)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Saturday, reviewed on Sunday (should have 0 pickup time)',
pr: {
number: 132,
html_url: 'https://github.com/owner/repo/pull/132',
draft: false,
created_at: '2023-05-20T14:00:00Z', // Saturday
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-21T14:00:00Z' } // Sunday
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 132,
prUrl: 'https://github.com/owner/repo/pull/132',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-20T14:00:00Z'),
firstReviewTime: new Date('2023-05-21T14:00:00Z'),
reviewDate: '2023-05-21',
pickupTimeSeconds: 0, // 0 seconds (both on weekend)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on weekday, reviewed after multiple weekends (should subtract weekend days)',
pr: {
number: 133,
html_url: 'https://github.com/owner/repo/pull/133',
draft: false,
created_at: '2023-05-17T14:00:00Z', // Wednesday
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-29T14:00:00Z' } // Monday, 12 days later
],
expected: {
metricType: 'time_to_first_review',
repository: 'owner/repo',
prNumber: 133,
prUrl: 'https://github.com/owner/repo/pull/133',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-17T14:00:00Z'),
firstReviewTime: new Date('2023-05-29T14:00:00Z'),
reviewDate: '2023-05-29',
pickupTimeSeconds: 691200, // 8 days = 691200 seconds (12 days - 4 weekend days)
readyEventType: 'PR creation (not draft)'
}
}
];
// Run each test case
test.each(testCases)('$name', ({ pr, timelineEvents, reviewEvents, expected }) => {
const result = githubClient.calculatePickupTime(pr, timelineEvents, reviewEvents);
if (expected === null) {
expect(result).toBeNull();
} else {
// Compare date objects separately
expect(result.readyTime).toEqual(expected.readyTime);
expect(result.firstReviewTime).toEqual(expected.firstReviewTime);
// Compare the rest of the properties
expect({
...result,
readyTime: undefined,
firstReviewTime: undefined
}).toEqual({
...expected,
readyTime: undefined,
firstReviewTime: undefined
});
}
});
});
});

View file

@ -0,0 +1,136 @@
/**
* General tests for GitHub client functionality
*/
import { jest } from '@jest/globals';
import GitHubClient from '../src/github-client.js';
// Mock the logger
jest.mock('../src/logger.js', () => ({
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn()
}));
describe('GitHubClient - General Functionality', () => {
let githubClient;
let mockOctokit;
beforeEach(() => {
// Mock Octokit
mockOctokit = {
rest: {
pulls: {
list: jest.fn(),
listReviews: jest.fn()
},
issues: {
listEventsForTimeline: jest.fn()
}
}
};
githubClient = new GitHubClient('fake-token');
githubClient.octokit = mockOctokit;
});
describe('constructor', () => {
test('should initialize with token', () => {
const client = new GitHubClient('test-token');
expect(client).toBeInstanceOf(GitHubClient);
});
test('should throw error without token', () => {
expect(() => new GitHubClient()).toThrow('GitHub token is required');
});
});
describe('fetchPullRequests', () => {
test('should fetch pull requests successfully', async () => {
const mockResponse = {
data: [
{
number: 1,
title: 'Test PR',
state: 'closed',
merged_at: '2023-06-15T14:30:00Z',
updated_at: '2023-06-15T14:30:00Z',
base: { ref: 'main' }
}
]
};
mockOctokit.rest.pulls.list.mockResolvedValue(mockResponse);
const result = await githubClient.fetchPullRequests('owner', 'repo', 'all', new Date('2023-01-01'));
expect(result).toHaveLength(1);
expect(result[0].number).toBe(1);
});
test('should handle API errors gracefully', async () => {
const mockError = new Error('API Error');
mockOctokit.rest.pulls.list.mockRejectedValue(mockError);
await expect(githubClient.fetchPullRequests('owner', 'repo', 'all', new Date('2023-01-01')))
.rejects.toThrow('API Error');
});
});
describe('fetchPRTimelineEvents', () => {
test('should fetch timeline events successfully', async () => {
const mockResponse = {
data: [
{
event: 'ready_for_review',
created_at: '2023-06-15T10:00:00Z'
}
]
};
mockOctokit.rest.issues.listEventsForTimeline.mockResolvedValue(mockResponse);
const result = await githubClient.fetchPRTimelineEvents('owner', 'repo', 123);
expect(result).toHaveLength(1);
expect(result[0].event).toBe('ready_for_review');
});
test('should handle API errors gracefully', async () => {
const mockError = new Error('Timeline API Error');
mockOctokit.rest.issues.listEventsForTimeline.mockRejectedValue(mockError);
await expect(githubClient.fetchPRTimelineEvents('owner', 'repo', 123))
.rejects.toThrow('Timeline API Error');
});
});
describe('fetchPRReviewEvents', () => {
test('should fetch reviews successfully', async () => {
const mockResponse = {
data: [
{
id: 1,
state: 'APPROVED',
submitted_at: '2023-06-15T12:00:00Z',
user: { login: 'reviewer1' }
}
]
};
mockOctokit.rest.pulls.listReviews.mockResolvedValue(mockResponse);
const result = await githubClient.fetchPRReviewEvents('owner', 'repo', 123);
expect(result).toHaveLength(1);
expect(result[0].state).toBe('APPROVED');
});
test('should handle API errors gracefully', async () => {
const mockError = new Error('Reviews API Error');
mockOctokit.rest.pulls.listReviews.mockRejectedValue(mockError);
await expect(githubClient.fetchPRReviewEvents('owner', 'repo', 123))
.rejects.toThrow('Reviews API Error');
});
});
});

View file

@ -0,0 +1,547 @@
import { jest } from '@jest/globals';
import GitHubClient from '../src/github-client.js';
// Mock the logger to avoid console output during tests
jest.mock('../src/logger.js', () => ({
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
default: {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn(),
}
}));
describe('GitHubClient - Time to Merge', () => {
let githubClient;
beforeEach(() => {
// Create a new instance of GitHubClient for each test
githubClient = new GitHubClient('fake-token');
// Mock the Octokit instance
githubClient.octokit = {
rest: {
pulls: {
list: jest.fn(),
listReviews: jest.fn(),
},
issues: {
listEventsForTimeline: jest.fn(),
}
}
};
});
describe('calculateTimeToMerge', () => {
// Table-driven test cases for calculateTimeToMerge
const testCases = [
{
name: 'PR created as non-draft and merged',
pr: {
number: 123,
html_url: 'https://github.com/owner/repo/pull/123',
draft: false,
created_at: '2023-05-10T10:00:00Z',
merged_at: '2023-05-10T11:30:00Z',
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-10T11:30:00Z' }
],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 123,
prUrl: 'https://github.com/owner/repo/pull/123',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-10T10:00:00Z'),
mergeTime: new Date('2023-05-10T11:30:00Z'),
mergeDate: '2023-05-10',
mergeTimeSeconds: 5400, // 1.5 hours = 5400 seconds
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR created as draft, then marked as ready for review, then merged',
pr: {
number: 124,
html_url: 'https://github.com/owner/repo/pull/124',
draft: true,
created_at: '2023-05-11T09:00:00Z',
merged_at: '2023-05-11T11:00:00Z',
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [
{
event: 'ready_for_review',
created_at: '2023-05-11T10:00:00Z'
}
],
reviewEvents: [
{ submitted_at: '2023-05-11T11:00:00Z' }
],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 124,
prUrl: 'https://github.com/owner/repo/pull/124',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-11T10:00:00Z'),
mergeTime: new Date('2023-05-11T11:00:00Z'),
mergeDate: '2023-05-11',
mergeTimeSeconds: 3600, // 1 hour = 3600 seconds
readyEventType: 'ready_for_review event'
}
},
{
name: 'PR with multiple ready_for_review events - should use the latest one',
pr: {
number: 125,
html_url: 'https://github.com/owner/repo/pull/125',
draft: true,
created_at: '2023-05-12T09:00:00Z',
merged_at: '2023-05-12T13:00:00Z',
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [
{
event: 'ready_for_review',
created_at: '2023-05-12T10:00:00Z'
},
{
event: 'convert_to_draft',
created_at: '2023-05-12T11:00:00Z'
},
{
event: 'ready_for_review',
created_at: '2023-05-12T12:00:00Z'
}
],
reviewEvents: [
{ submitted_at: '2023-05-12T13:00:00Z' }
],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 125,
prUrl: 'https://github.com/owner/repo/pull/125',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-12T12:00:00Z'),
mergeTime: new Date('2023-05-12T13:00:00Z'),
mergeDate: '2023-05-12',
mergeTimeSeconds: 3600, // 1 hour = 3600 seconds
readyEventType: 'ready_for_review event'
}
},
{
name: 'PR with ready_for_review event after merge time - should use PR creation',
pr: {
number: 126,
html_url: 'https://github.com/owner/repo/pull/126',
draft: false,
created_at: '2023-05-16T09:00:00Z',
merged_at: '2023-05-16T11:00:00Z',
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [
{
event: 'convert_to_draft',
created_at: '2023-05-16T10:00:00Z'
},
{
event: 'ready_for_review',
created_at: '2023-05-16T12:00:00Z' // After merge
}
],
reviewEvents: [
{ submitted_at: '2023-05-16T11:00:00Z' }
],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 126,
prUrl: 'https://github.com/owner/repo/pull/126',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-16T09:00:00Z'),
mergeTime: new Date('2023-05-16T11:00:00Z'),
mergeDate: '2023-05-16',
mergeTimeSeconds: 7200, // 2 hours = 7200 seconds
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR with no ready_for_review events and created as draft',
pr: {
number: 127,
html_url: 'https://github.com/owner/repo/pull/127',
draft: true,
created_at: '2023-05-14T09:00:00Z',
merged_at: '2023-05-14T11:00:00Z',
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-05-14T11:00:00Z' }
],
expected: null // Should return null because no ready event was found
},
{
name: 'PR that was never merged',
pr: {
number: 128,
html_url: 'https://github.com/owner/repo/pull/128',
draft: false,
created_at: '2023-05-15T09:00:00Z',
merged_at: null,
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: null // Should return null because PR was not merged
},
{
name: 'PR that is still open',
pr: {
number: 129,
html_url: 'https://github.com/owner/repo/pull/129',
draft: false,
created_at: '2023-05-16T09:00:00Z',
merged_at: null,
state: 'open',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: null // Should return null because PR is not merged
},
{
name: 'PR ready on Saturday, merged on Sunday 3 weeks later (should exclude weekend days)',
pr: {
number: 136,
html_url: 'https://github.com/owner/repo/pull/136',
draft: false,
created_at: '2023-05-20T14:00:00Z', // Saturday
merged_at: '2023-06-11T14:00:00Z', // Sunday, 3 weeks later
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [
{ submitted_at: '2023-06-11T13:00:00Z' } // Sunday, 3 weeks later
],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 136,
prUrl: 'https://github.com/owner/repo/pull/136',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-20T14:00:00Z'),
mergeTime: new Date('2023-06-11T14:00:00Z'),
mergeDate: '2023-06-11',
mergeTimeSeconds: 1296000, // 15 days = 1296000 seconds (3 weeks of 5 working days)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Sunday, merged on Monday (should use end of Sunday as ready time)',
pr: {
number: 134,
html_url: 'https://github.com/owner/repo/pull/134',
draft: false,
created_at: '2023-05-21T14:00:00Z', // Sunday
merged_at: '2023-05-22T10:00:00Z', // Monday
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 134,
prUrl: 'https://github.com/owner/repo/pull/134',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-21T14:00:00Z'),
mergeTime: new Date('2023-05-22T10:00:00Z'),
mergeDate: '2023-05-22',
mergeTimeSeconds: 36000, // 10 hours = 36000 seconds (from end of Sunday to Monday 10am)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Sunday, merged on next Saturday (should exclude weekend days)',
pr: {
number: 135,
html_url: 'https://github.com/owner/repo/pull/135',
draft: false,
created_at: '2023-05-21T14:00:00Z', // Sunday
merged_at: '2023-05-27T14:00:00Z', // Saturday
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 135,
prUrl: 'https://github.com/owner/repo/pull/135',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-21T14:00:00Z'),
mergeTime: new Date('2023-05-27T14:00:00Z'),
mergeDate: '2023-05-27',
mergeTimeSeconds: 432000, // 5 days = 432000 seconds (6 days - 1 weekend day)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Friday, merged on Monday (should subtract weekend days)',
pr: {
number: 130,
html_url: 'https://github.com/owner/repo/pull/130',
draft: false,
created_at: '2023-05-19T14:00:00Z', // Friday
merged_at: '2023-05-22T10:00:00Z', // Monday
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 130,
prUrl: 'https://github.com/owner/repo/pull/130',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-19T14:00:00Z'),
mergeTime: new Date('2023-05-22T10:00:00Z'),
mergeDate: '2023-05-22',
mergeTimeSeconds: 72000, // 20 hours = 72000 seconds (3 days - 2 weekend days)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Saturday, merged on Monday (should use end of Sunday as ready time)',
pr: {
number: 131,
html_url: 'https://github.com/owner/repo/pull/131',
draft: false,
created_at: '2023-05-20T14:00:00Z', // Saturday
merged_at: '2023-05-22T10:00:00Z', // Monday
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 131,
prUrl: 'https://github.com/owner/repo/pull/131',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-20T14:00:00Z'),
mergeTime: new Date('2023-05-22T10:00:00Z'),
mergeDate: '2023-05-22',
mergeTimeSeconds: 36000, // 10 hours = 36000 seconds (from end of Sunday to Monday 10am)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on Saturday, merged on Sunday (should have 0 merge time)',
pr: {
number: 132,
html_url: 'https://github.com/owner/repo/pull/132',
draft: false,
created_at: '2023-05-20T14:00:00Z', // Saturday
merged_at: '2023-05-21T14:00:00Z', // Sunday
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 132,
prUrl: 'https://github.com/owner/repo/pull/132',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-20T14:00:00Z'),
mergeTime: new Date('2023-05-21T14:00:00Z'),
mergeDate: '2023-05-21',
mergeTimeSeconds: 0, // 0 seconds (both on weekend)
readyEventType: 'PR creation (not draft)'
}
},
{
name: 'PR ready on weekday, merged after multiple weekends (should subtract weekend days)',
pr: {
number: 133,
html_url: 'https://github.com/owner/repo/pull/133',
draft: false,
created_at: '2023-05-17T14:00:00Z', // Wednesday
merged_at: '2023-05-29T14:00:00Z', // Monday, 12 days later
state: 'closed',
user: { login: 'author' },
base: {
ref: 'main',
repo: {
name: 'repo',
owner: { login: 'owner' }
}
}
},
timelineEvents: [],
reviewEvents: [],
expected: {
metricType: 'time_to_merge',
repository: 'owner/repo',
prNumber: 133,
prUrl: 'https://github.com/owner/repo/pull/133',
prCreator: 'author',
targetBranch: 'main',
readyTime: new Date('2023-05-17T14:00:00Z'),
mergeTime: new Date('2023-05-29T14:00:00Z'),
mergeDate: '2023-05-29',
mergeTimeSeconds: 691200, // 8 days = 691200 seconds (12 days - 4 weekend days)
readyEventType: 'PR creation (not draft)'
}
}
];
// Run each test case
test.each(testCases)('$name', ({ pr, timelineEvents, reviewEvents, expected }) => {
const result = githubClient.calculateTimeToMerge(pr, timelineEvents, reviewEvents);
if (expected === null) {
expect(result).toBeNull();
} else {
// Compare date objects separately
expect(result.readyTime).toEqual(expected.readyTime);
expect(result.mergeTime).toEqual(expected.mergeTime);
// Compare the rest of the properties
expect({
...result,
readyTime: undefined,
mergeTime: undefined
}).toEqual({
...expected,
readyTime: undefined,
mergeTime: undefined
});
}
});
});
});

View file

@ -0,0 +1,354 @@
/**
* Tests for markdown parser module
*/
import { jest } from '@jest/globals';
// Mock the logger
const mockLogger = {
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn()
};
// Mock fs
const mockFs = {
existsSync: jest.fn(),
readFileSync: jest.fn()
};
// Mock path
const mockPath = {
resolve: jest.fn()
};
// Set up module mocks
jest.unstable_mockModule('../src/logger.js', () => ({
default: mockLogger,
...mockLogger
}));
jest.unstable_mockModule('fs', () => ({
default: mockFs,
...mockFs
}));
jest.unstable_mockModule('path', () => ({
default: mockPath,
...mockPath
}));
// Import the module after mocking
const { parseProductGroups, validateMarkdownStructure } = await import('../src/markdown-parser.js');
describe('MarkdownParser', () => {
beforeEach(() => {
jest.clearAllMocks();
mockPath.resolve.mockImplementation((cwd, filePath) => `/resolved/${filePath}`);
});
describe('parseProductGroups', () => {
it('should parse valid markdown with all groups correctly', () => {
const mockMarkdown = `
# Product Groups
## Some other content
### MDM group
| Role | Contributor |
|------|-------------|
| Product Manager | _([@testpm1](https://github.com/testpm1))_ |
| Developer | _([@testdev1](https://github.com/testdev1))_, _([@testdev2](https://github.com/testdev2))_, _([@testdev3](https://github.com/testdev3))_ |
| Quality Assurance | _([@testqa1](https://github.com/testqa1))_ |
### Orchestration group
| Role | Contributor |
|------|-------------|
| Product Manager | _([@testpm2](https://github.com/testpm2))_ |
| Developer | _([@orchdev1](https://github.com/orchdev1))_, _([@orchdev2](https://github.com/orchdev2))_, _([@orchdev3](https://github.com/orchdev3))_ |
### Software group
| Role | Contributor |
|------|-------------|
| Developer | _([@softdev1](https://github.com/softdev1))_, _([@softdev2](https://github.com/softdev2))_ |
| Quality Assurance | _([@testqa2](https://github.com/testqa2))_ |
`;
mockFs.existsSync.mockReturnValue(true);
mockFs.readFileSync.mockReturnValue(mockMarkdown);
const result = parseProductGroups('test-file.md');
expect(mockFs.existsSync).toHaveBeenCalledWith('/resolved/test-file.md');
expect(mockFs.readFileSync).toHaveBeenCalledWith('/resolved/test-file.md', 'utf8');
// Should extract usernames and create dual group membership
expect(result).toEqual([
// MDM group users
{ group: 'mdm', username: 'testdev1' },
{ group: 'engineering', username: 'testdev1' },
{ group: 'mdm', username: 'testdev2' },
{ group: 'engineering', username: 'testdev2' },
{ group: 'mdm', username: 'testdev3' },
{ group: 'engineering', username: 'testdev3' },
// Orchestration group users
{ group: 'orchestration', username: 'orchdev1' },
{ group: 'engineering', username: 'orchdev1' },
{ group: 'orchestration', username: 'orchdev2' },
{ group: 'engineering', username: 'orchdev2' },
{ group: 'orchestration', username: 'orchdev3' },
{ group: 'engineering', username: 'orchdev3' },
// Software group users
{ group: 'software', username: 'softdev1' },
{ group: 'engineering', username: 'softdev1' },
{ group: 'software', username: 'softdev2' },
{ group: 'engineering', username: 'softdev2' }
]);
expect(mockLogger.info).toHaveBeenCalledWith('Parsing product groups from /resolved/test-file.md');
expect(mockLogger.info).toHaveBeenCalledWith('Found 3 developers in mdm group: testdev1, testdev2, testdev3');
expect(mockLogger.info).toHaveBeenCalledWith('Found 3 developers in orchestration group: orchdev1, orchdev2, orchdev3');
expect(mockLogger.info).toHaveBeenCalledWith('Found 2 developers in software group: softdev1, softdev2');
expect(mockLogger.info).toHaveBeenCalledWith('Extracted 16 user-group mappings from markdown');
});
it('should handle missing sections gracefully', () => {
const mockMarkdown = `
# Product Groups
### MDM group
| Role | Contributor |
|------|-------------|
| Developer | _([@testdev1](https://github.com/testdev1))_ |
### Software group
| Role | Contributor |
|------|-------------|
| Developer | _([@softdev1](https://github.com/softdev1))_ |
`;
mockFs.existsSync.mockReturnValue(true);
mockFs.readFileSync.mockReturnValue(mockMarkdown);
const result = parseProductGroups('test-file.md');
expect(result).toEqual([
{ group: 'mdm', username: 'testdev1' },
{ group: 'engineering', username: 'testdev1' },
{ group: 'software', username: 'softdev1' },
{ group: 'engineering', username: 'softdev1' }
]);
expect(mockLogger.warn).toHaveBeenCalledWith('Section not found: Orchestration group');
});
it('should handle missing Developer rows', () => {
const mockMarkdown = `
# Product Groups
### MDM group
| Role | Contributor |
|------|-------------|
| Product Manager | _([@testpm1](https://github.com/testpm1))_ |
| Quality Assurance | _([@testqa1](https://github.com/testqa1))_ |
### Orchestration group
| Role | Contributor |
|------|-------------|
| Developer | _([@orchdev1](https://github.com/orchdev1))_ |
`;
mockFs.existsSync.mockReturnValue(true);
mockFs.readFileSync.mockReturnValue(mockMarkdown);
const result = parseProductGroups('test-file.md');
expect(result).toEqual([
{ group: 'orchestration', username: 'orchdev1' },
{ group: 'engineering', username: 'orchdev1' }
]);
expect(mockLogger.warn).toHaveBeenCalledWith('No Developer row found in mdm group section');
});
it('should handle malformed GitHub username patterns', () => {
const mockMarkdown = `
# Product Groups
### MDM group
| Role | Contributor |
|------|-------------|
| Developer | Some text without proper format, _([@testvaliduser](https://github.com/testvaliduser))_, invalid format here |
### Orchestration group
| Role | Contributor |
|------|-------------|
| Developer | No valid usernames here at all |
`;
mockFs.existsSync.mockReturnValue(true);
mockFs.readFileSync.mockReturnValue(mockMarkdown);
const result = parseProductGroups('test-file.md');
expect(result).toEqual([
{ group: 'mdm', username: 'testvaliduser' },
{ group: 'engineering', username: 'testvaliduser' }
]);
expect(mockLogger.warn).toHaveBeenCalledWith('No GitHub usernames found in orchestration group Developer row');
});
it('should handle multi-line developer cells', () => {
const mockMarkdown = `
# Product Groups
### MDM group
| Role | Contributor |
|------|-------------|
| Developer | _([@testuser1](https://github.com/testuser1))_,
_([@testuser2](https://github.com/testuser2))_,
_([@testuser3](https://github.com/testuser3))_ |
`;
mockFs.existsSync.mockReturnValue(true);
mockFs.readFileSync.mockReturnValue(mockMarkdown);
const result = parseProductGroups('test-file.md');
expect(result).toEqual([
{ group: 'mdm', username: 'testuser1' },
{ group: 'engineering', username: 'testuser1' },
{ group: 'mdm', username: 'testuser2' },
{ group: 'engineering', username: 'testuser2' },
{ group: 'mdm', username: 'testuser3' },
{ group: 'engineering', username: 'testuser3' }
]);
});
it('should handle file not found', () => {
mockFs.existsSync.mockReturnValue(false);
const result = parseProductGroups('nonexistent-file.md');
expect(result).toEqual([]);
expect(mockLogger.error).toHaveBeenCalledWith('Product groups file not found at /resolved/nonexistent-file.md');
});
it('should handle file read errors', () => {
mockFs.existsSync.mockReturnValue(true);
mockFs.readFileSync.mockImplementation(() => {
throw new Error('Permission denied');
});
const result = parseProductGroups('error-file.md');
expect(result).toEqual([]);
expect(mockLogger.error).toHaveBeenCalledWith(
'Error parsing product groups file: error-file.md',
{},
expect.any(Error)
);
});
it('should handle usernames with hyphens and numbers', () => {
const mockMarkdown = `
# Product Groups
### MDM group
| Role | Contributor |
|------|-------------|
| Developer | _([@testuser-123](https://github.com/testuser-123))_, _([@fakeuser-456](https://github.com/fakeuser-456))_ |
`;
mockFs.existsSync.mockReturnValue(true);
mockFs.readFileSync.mockReturnValue(mockMarkdown);
const result = parseProductGroups('test-file.md');
expect(result).toEqual([
{ group: 'mdm', username: 'testuser-123' },
{ group: 'engineering', username: 'testuser-123' },
{ group: 'mdm', username: 'fakeuser-456' },
{ group: 'engineering', username: 'fakeuser-456' }
]);
});
});
describe('validateMarkdownStructure', () => {
it('should return true for valid markdown with all required sections', () => {
const validMarkdown = `
# Product Groups
### MDM group
Some content
### Orchestration group
Some content
### Software group
Some content
`;
const result = validateMarkdownStructure(validMarkdown);
expect(result).toBe(true);
});
it('should return false and warn for missing sections', () => {
const invalidMarkdown = `
# Product Groups
### MDM group
Some content
### Software group
Some content
`;
const result = validateMarkdownStructure(invalidMarkdown);
expect(result).toBe(false);
expect(mockLogger.warn).toHaveBeenCalledWith('Missing required section: Orchestration group');
});
it('should return false for completely empty content', () => {
const result = validateMarkdownStructure('');
expect(result).toBe(false);
expect(mockLogger.warn).toHaveBeenCalledWith('Missing required section: MDM group');
// The function returns false on first missing section, so other warnings may not be called
expect(mockLogger.warn).toHaveBeenCalledTimes(1);
});
it('should handle case-sensitive section matching', () => {
const invalidMarkdown = `
# Product Groups
### mdm group
Some content
### orchestration group
Some content
### software group
Some content
`;
const result = validateMarkdownStructure(invalidMarkdown);
expect(result).toBe(false);
expect(mockLogger.warn).toHaveBeenCalledWith('Missing required section: MDM group');
// The function returns false on first missing section, so other warnings may not be called
expect(mockLogger.warn).toHaveBeenCalledTimes(1);
});
});
});

View file

@ -0,0 +1,321 @@
/**
* Tests for metrics collector module
*/
import { jest } from '@jest/globals';
import { MetricsCollector } from '../src/metrics-collector.js';
// Mock the logger
jest.mock('../src/logger.js', () => ({
info: jest.fn(),
warn: jest.fn(),
error: jest.fn(),
debug: jest.fn()
}));
// Mock GitHubClient
jest.mock('../src/github-client.js', () => {
return jest.fn().mockImplementation(() => ({
fetchPullRequests: jest.fn(),
fetchPRTimelineEvents: jest.fn(),
fetchPRReviewEvents: jest.fn(),
calculatePickupTime: jest.fn(),
calculateTimeToMerge: jest.fn()
}));
});
// Mock BigQueryClient
jest.mock('../src/bigquery-client.js', () => ({
BigQueryClient: jest.fn().mockImplementation(() => ({
uploadMetrics: jest.fn()
}))
}));
describe('MetricsCollector', () => {
let metricsCollector;
let mockConfig;
let mockGitHubClient;
let mockBigQueryClient;
beforeEach(() => {
mockConfig = {
githubToken: 'fake-token',
serviceAccountKeyPath: '/fake/path',
repositories: ['owner/repo'],
lookbackDays: 7,
targetBranch: 'main',
bigQueryDatasetId: 'test_dataset',
printOnly: false,
metrics: {
timeToFirstReview: {
enabled: true,
tableName: 'pr_first_review'
},
timeToMerge: {
enabled: true,
tableName: 'pr_merge'
}
}
};
metricsCollector = new MetricsCollector(mockConfig);
// Mock the clients
mockGitHubClient = {
fetchPullRequests: jest.fn(),
fetchPRTimelineEvents: jest.fn(),
fetchPRReviewEvents: jest.fn(),
calculatePickupTime: jest.fn(),
calculateTimeToMerge: jest.fn()
};
mockBigQueryClient = {
uploadMetrics: jest.fn()
};
metricsCollector.githubClient = mockGitHubClient;
metricsCollector.bigqueryClient = mockBigQueryClient;
});
describe('collectPRMetrics', () => {
const mockPR = {
number: 123,
html_url: 'https://github.com/owner/repo/pull/123',
user: { login: 'testuser' },
base: { ref: 'main' },
head: { repo: { full_name: 'owner/repo' } }
};
const mockTimelineEvents = [
{ event: 'ready_for_review', created_at: '2023-06-15T10:00:00Z' }
];
const mockReviewEvents = [
{ submitted_at: '2023-06-15T12:00:00Z', state: 'approved', user: { login: 'reviewer1' } }
];
test('should collect both metrics when both are enabled', async () => {
const firstReviewMetric = {
metricType: 'time_to_first_review',
prNumber: 123,
pickupTimeSeconds: 7200
};
const mergeMetric = {
metricType: 'time_to_merge',
prNumber: 123,
mergeTimeSeconds: 16200
};
mockGitHubClient.calculatePickupTime.mockReturnValue(firstReviewMetric);
mockGitHubClient.calculateTimeToMerge.mockReturnValue(mergeMetric);
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
expect(result).toEqual([firstReviewMetric, mergeMetric]);
expect(mockGitHubClient.calculatePickupTime).toHaveBeenCalledWith(mockPR, mockTimelineEvents, mockReviewEvents);
expect(mockGitHubClient.calculateTimeToMerge).toHaveBeenCalledWith(mockPR, mockTimelineEvents, mockReviewEvents);
});
test('should only collect first review metric when merge is disabled', async () => {
metricsCollector.config.metrics.timeToMerge.enabled = false;
const firstReviewMetric = {
metricType: 'time_to_first_review',
prNumber: 123,
pickupTimeSeconds: 7200
};
mockGitHubClient.calculatePickupTime.mockReturnValue(firstReviewMetric);
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
expect(result).toEqual([firstReviewMetric]);
expect(mockGitHubClient.calculatePickupTime).toHaveBeenCalled();
expect(mockGitHubClient.calculateTimeToMerge).not.toHaveBeenCalled();
});
test('should only collect merge metric when first review is disabled', async () => {
metricsCollector.config.metrics.timeToFirstReview.enabled = false;
const mergeMetric = {
metricType: 'time_to_merge',
prNumber: 123,
mergeTimeSeconds: 16200
};
mockGitHubClient.calculateTimeToMerge.mockReturnValue(mergeMetric);
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
expect(result).toEqual([mergeMetric]);
expect(mockGitHubClient.calculatePickupTime).not.toHaveBeenCalled();
expect(mockGitHubClient.calculateTimeToMerge).toHaveBeenCalled();
});
test('should handle null metrics gracefully', async () => {
mockGitHubClient.calculatePickupTime.mockReturnValue(null);
mockGitHubClient.calculateTimeToMerge.mockReturnValue(null);
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
expect(result).toEqual([]);
});
test('should handle errors in metric calculation', async () => {
mockGitHubClient.calculatePickupTime.mockImplementation(() => {
throw new Error('Calculation error');
});
const mergeMetric = {
metricType: 'time_to_merge',
prNumber: 123,
mergeTimeSeconds: 16200
};
mockGitHubClient.calculateTimeToMerge.mockReturnValue(mergeMetric);
const result = await metricsCollector.collectPRMetrics(mockPR, mockTimelineEvents, mockReviewEvents);
expect(result).toEqual([mergeMetric]);
});
});
describe('groupMetricsByType', () => {
test('should group metrics by type correctly', () => {
const metrics = [
{ metricType: 'time_to_first_review', prNumber: 123 },
{ metricType: 'time_to_merge', prNumber: 123 },
{ metricType: 'time_to_first_review', prNumber: 124 },
{ metricType: 'time_to_merge', prNumber: 124 }
];
const grouped = metricsCollector.groupMetricsByType(metrics);
expect(grouped).toEqual({
time_to_first_review: [
{ metricType: 'time_to_first_review', prNumber: 123 },
{ metricType: 'time_to_first_review', prNumber: 124 }
],
time_to_merge: [
{ metricType: 'time_to_merge', prNumber: 123 },
{ metricType: 'time_to_merge', prNumber: 124 }
]
});
});
test('should handle empty metrics array', () => {
const grouped = metricsCollector.groupMetricsByType([]);
expect(grouped).toEqual({});
});
});
describe('getTableNameForMetricType', () => {
test('should return correct table name for time_to_first_review', () => {
const tableName = metricsCollector.getTableNameForMetricType('time_to_first_review');
expect(tableName).toBe('pr_first_review');
});
test('should return correct table name for time_to_merge', () => {
const tableName = metricsCollector.getTableNameForMetricType('time_to_merge');
expect(tableName).toBe('pr_merge');
});
test('should throw error for unknown metric type', () => {
expect(() => {
metricsCollector.getTableNameForMetricType('unknown_type');
}).toThrow('Unknown metric type: unknown_type');
});
});
describe('uploadMetrics', () => {
test('should upload metrics to correct tables', async () => {
const metrics = [
{
metricType: 'time_to_first_review',
prNumber: 123,
pickupTimeSeconds: 7200
},
{
metricType: 'time_to_merge',
prNumber: 123,
mergeTimeSeconds: 16200
},
{
metricType: 'time_to_first_review',
prNumber: 124,
pickupTimeSeconds: 3600
}
];
await metricsCollector.uploadMetrics(metrics);
expect(mockBigQueryClient.uploadMetrics).toHaveBeenCalledTimes(2);
// Check first call (time_to_first_review)
expect(mockBigQueryClient.uploadMetrics).toHaveBeenNthCalledWith(
1,
'test_dataset',
'pr_first_review',
[
{ metricType: 'time_to_first_review', prNumber: 123, pickupTimeSeconds: 7200 },
{ metricType: 'time_to_first_review', prNumber: 124, pickupTimeSeconds: 3600 }
]
);
// Check second call (time_to_merge)
expect(mockBigQueryClient.uploadMetrics).toHaveBeenNthCalledWith(
2,
'test_dataset',
'pr_merge',
[
{ metricType: 'time_to_merge', prNumber: 123, mergeTimeSeconds: 16200 }
]
);
});
test('should handle empty metrics array', async () => {
await metricsCollector.uploadMetrics([]);
expect(mockBigQueryClient.uploadMetrics).not.toHaveBeenCalled();
});
test('should skip empty metric groups', async () => {
const metrics = [
{
metricType: 'time_to_first_review',
prNumber: 123,
pickupTimeSeconds: 7200
}
];
await metricsCollector.uploadMetrics(metrics);
expect(mockBigQueryClient.uploadMetrics).toHaveBeenCalledTimes(1);
expect(mockBigQueryClient.uploadMetrics).toHaveBeenCalledWith(
'test_dataset',
'pr_first_review',
[{ metricType: 'time_to_first_review', prNumber: 123, pickupTimeSeconds: 7200 }]
);
});
});
describe('getMetricTypeDisplayName', () => {
test('should return correct display names', () => {
expect(metricsCollector.getMetricTypeDisplayName('time_to_first_review')).toBe('Time to First Review');
expect(metricsCollector.getMetricTypeDisplayName('time_to_merge')).toBe('Time to Merge');
expect(metricsCollector.getMetricTypeDisplayName('unknown_type')).toBe('unknown_type');
});
});
describe('getTimeFieldForMetricType', () => {
test('should return correct time fields', () => {
const firstReviewMetric = { pickupTimeSeconds: 7200, mergeTimeSeconds: 16200 };
const mergeMetric = { pickupTimeSeconds: 7200, mergeTimeSeconds: 16200 };
expect(metricsCollector.getTimeFieldForMetricType('time_to_first_review', firstReviewMetric)).toBe(7200);
expect(metricsCollector.getTimeFieldForMetricType('time_to_merge', mergeMetric)).toBe(16200);
expect(metricsCollector.getTimeFieldForMetricType('unknown_type', {})).toBe(0);
});
});
});

View file

@ -0,0 +1,16 @@
/**
* Jest setup file for engineering metrics tests
*/
import { jest } from "@jest/globals";
// Set test environment variables
process.env.NODE_ENV = "test";
// Mock Date.now for consistent testing
beforeAll(() => {
jest.useFakeTimers().setSystemTime(new Date("2023-06-15T12:00:00Z"));
});
afterAll(() => {
jest.useRealTimers();
});

View file

@ -0,0 +1,57 @@
name: Collect engineering metrics test
on:
push:
branches:
- main
- patch-*
- prepare-*
paths:
- '.github/actions/eng-metrics/**'
pull_request:
paths:
- '.github/actions/eng-metrics/**'
workflow_dispatch:
# This allows a subsequently queued workflow run to interrupt previous runs
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id}}
cancel-in-progress: true
permissions:
contents: read
jobs:
test:
name: Test Engineering Metrics Action
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # 4.2.2
- name: Setup Node.js 20
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # 4.4.0
with:
node-version: 20
cache: 'npm'
cache-dependency-path: '.github/actions/eng-metrics/package-lock.json'
- name: Install dependencies
run: npm ci
working-directory: .github/actions/eng-metrics
- name: Run linting
run: npm run lint
working-directory: .github/actions/eng-metrics
- name: Run tests
run: npm test
working-directory: .github/actions/eng-metrics
env:
NODE_ENV: test

View file

@ -0,0 +1,55 @@
name: Collect engineering metrics
on:
schedule:
- cron: '0 9 * * *' # Run at 4am CDT (9am UTC)
workflow_dispatch: # Allow manual triggering
# This allows a subsequently queued workflow run to interrupt previous runs
concurrency:
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id}}
cancel-in-progress: true
permissions:
contents: read # fetch repo metadata
pull-requests: read # read PR timelines
jobs:
collect-metrics:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@63c24ba6bd7ba022e95695ff85de572c04a18142 # v2.7.0
with:
egress-policy: audit
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # 4.2.2
- name: Set up Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # 4.4.0
with:
node-version: '20.18.1'
cache: 'npm'
cache-dependency-path: .github/actions/eng-metrics/package-lock.json
- name: Install dependencies
working-directory: .github/actions/eng-metrics
run: npm ci
- name: Create service account key file
working-directory: .github/actions/eng-metrics
run: |
echo '${{ secrets.ENG_METRICS_GCP_SERVICE_ACCOUNT_KEY }}' > service-account-key.json
# Verify the file is valid JSON and is a top-level object
if ! jq 'type == "object"' service-account-key.json | grep -q true; then
echo "Error: service-account-key.json is either invalid JSON or not a JSON object"
echo "Is ENG_METRICS_GCP_SERVICE_ACCOUNT_KEY secret properly set?"
exit 1
fi
- name: Collect and upload engineering metrics
uses: ./.github/actions/eng-metrics
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SERVICE_ACCOUNT_KEY_PATH: './service-account-key.json'