n8n/packages/testing/janitor

@n8n/playwright-janitor

Static analysis and architecture enforcement for Playwright test suites.

Why?

Playwright tests are easy to write but hard to maintain at scale. Without guardrails, test code accumulates problems:

• Selector duplication - Same getByTestId('button') scattered across files
• Leaky abstractions - Tests directly manipulating the DOM instead of using page objects
• Dead code - Unused page object methods nobody deletes
• Architecture drift - Flows importing pages, pages importing tests, layers bleeding together
• Orphaned test data - Workflow files nobody references anymore

The janitor catches these problems through static analysis, enforcing your architecture before bad patterns spread.

Architecture Model

The janitor enforces a layered architecture for Playwright test suites:

┌─────────────────────────────────────────────────────────┐
│                        Tests                            │
│   test('user can login', async ({ app }) => { ... })    │
└──────────────────────────┬──────────────────────────────┘
                           │ uses
                           ▼
┌─────────────────────────────────────────────────────────┐
│                   Flows / Composables                   │
│   await app.workflows.createAndRun('my-workflow')       │
└──────────────────────────┬──────────────────────────────┘
                           │ orchestrates
                           ▼
┌─────────────────────────────────────────────────────────┐
│                      Page Objects                       │
│   await this.canvas.addNode('HTTP Request')             │
└──────────────────────────┬──────────────────────────────┘
                           │ encapsulates
                           ▼
┌─────────────────────────────────────────────────────────┐
│                       Components                        │
│   await this.nodePanel.selectNode('Webhook')            │
└──────────────────────────┬──────────────────────────────┘
                           │ wraps
                           ▼
┌─────────────────────────────────────────────────────────┐
│                    Playwright API                       │
│   page.getByTestId(), page.locator(), page.click()      │
└─────────────────────────────────────────────────────────┘

Key principles:

1. Dependencies flow downward - Tests depend on flows, flows on pages, pages on components
2. No skipping layers - Tests should use flows, not reach directly into page internals
3. Selectors belong in page objects - Raw getByTestId() calls don't belong in tests or flows
4. One home per selector - Each test ID should be defined in exactly one page object

Quick Start

Installation

pnpm add -D @n8n/playwright-janitor

Configuration

Create a janitor.config.js in your Playwright test root:

import { defineConfig } from '@n8n/playwright-janitor';

export default defineConfig({
  rootDir: __dirname,

  // Where your different artifact types live
  patterns: {
    pages: ['pages/**/*.ts'],
    components: ['pages/components/**/*.ts'],
    flows: ['composables/**/*.ts'],       // or 'actions/**/*.ts', 'scenarios/**/*.ts'
    tests: ['tests/**/*.spec.ts'],
    services: ['services/**/*.ts'],
    fixtures: ['fixtures/**/*.ts'],
    helpers: ['helpers/**/*.ts'],
    factories: ['factories/**/*.ts'],
    testData: ['workflows/**/*'],          // Static JSON/fixtures
  },

  // The main page object facade that exposes sub-pages
  facade: {
    file: 'pages/AppPage.ts',
    className: 'AppPage',
    excludeTypes: ['Page', 'APIRequestContext'],
  },

  // What you call the fixture in your tests
  fixtureObjectName: 'app',  // test('...', async ({ app }) => ...)
});

Run Analysis

import { runAnalysis } from '@n8n/playwright-janitor';
import config from './janitor.config.js';

const report = runAnalysis(config);
console.log(`Found ${report.summary.totalViolations} violations`);

Or create a script:

// scripts/run-janitor.ts
import { runAnalysis, toConsole } from '@n8n/playwright-janitor';
import config from '../janitor.config.js';

const report = runAnalysis(config);
toConsole(report);
process.exit(report.summary.totalViolations > 0 ? 1 : 0);

Baseline (Incremental Cleanup)

For existing codebases with many violations, use a baseline to enable incremental cleanup:

# Create baseline of current violations
playwright-janitor baseline

# Commit the baseline
git add .janitor-baseline.json
git commit -m "chore: add janitor baseline"

Once a baseline exists, janitor and TCR only fail on new violations. Pre-existing violations are tracked but don't block commits.

Safeguard: TCR blocks commits that modify .janitor-baseline.json. This prevents accidentally "fixing" violations by updating the baseline instead of the actual code. Baseline updates must always be done manually.

# This now passes (only checks for NEW violations)
playwright-janitor tcr --execute -m="Add new feature"

# As you fix violations, update the baseline (manual commit required - TCR won't commit baseline changes)
playwright-janitor baseline
git add .janitor-baseline.json
git commit -m "chore: update baseline after cleanup"

Baseline file format: .janitor-baseline.json - tracks violations by file and content hash, so line number shifts don't cause false positives.

List Rules

View all available rules with their descriptions:

# Human-readable list
playwright-janitor rules

# JSON output (for AI agents/automation)
playwright-janitor rules --json

# Verbose (includes target globs)
playwright-janitor rules --verbose

The JSON output is useful for AI agents that need to understand the rules before writing code.

Test Discovery & Orchestration

Discover test specs via AST analysis and distribute them across CI shards:

# Discover specs and capabilities (JSON output)
playwright-janitor discover

# Distribute specs across shards (JSON output)
playwright-janitor orchestrate --shards=14

# Get specs for a single shard (0-indexed)
playwright-janitor orchestrate --shards=14 --shard-index=0

# Only include specs affected by git changes
playwright-janitor orchestrate --shards=14 --impact

Discovery detects test.fixme() and test.skip() via AST and excludes them automatically. Capability tags (@capability:proxy) are extracted for grouping.

Rules

Architecture Rules

boundary-protection

Severity: error

Prevents pages from importing other pages directly. Each page should be independent; if you need to compose pages, that's what the facade/flows layer is for.

// Bad - WorkflowPage importing SettingsPage
import { SettingsPage } from './SettingsPage';

export class WorkflowPage {
  async openSettings() {
    await this.settingsPage.open(); // Coupling between pages
  }
}

// Good - Pages are independent, composition happens in flows
export class WorkflowPage {
  async getWorkflowName() {
    return this.header.getByTestId('workflow-name').textContent();
  }
}

scope-lockdown

Severity: error

Enforces explicit architectural intent for page objects. Each page must either:

1. Have a container getter (scoped component - must use container for all locators)
2. Have a navigation method (standalone top-level page - can use this.page directly)

This prevents ambiguous page objects and ensures consistent patterns.

// Bad - Ambiguous page (neither container nor navigation method)
export class SettingsPage {
  async toggleOption() {
    await this.page.getByTestId('toggle').click(); // Is this a page or component?
  }
}

// Good - Standalone page with navigation method
export class SettingsPage {
  async goto() {
    await this.page.goto('/settings');
  }

  async toggleOption() {
    await this.page.getByTestId('toggle').click(); // OK - explicit standalone page
  }
}

// Good - Scoped component with container
export class NodePanel {
  get container() { return this.page.locator('.node-panel'); }

  async selectNode(name: string) {
    await this.container.getByTestId('node-item').click(); // Scoped to container
  }
}

// Bad - Component with container using unscoped locators
export class NodePanel {
  get container() { return this.page.locator('.node-panel'); }

  async selectNode(name: string) {
    await this.page.getByTestId('node-item').click(); // Escapes container!
  }
}

Configuration:

rules: {
  'scope-lockdown': {
    enabled: true,
    severity: 'error',
    // Customize which method names indicate a standalone page
    navigationMethods: ['goto', 'navigate', 'visit', 'open'],
  },
}

selector-purity

Severity: error

Raw Playwright locators (getByTestId, locator, etc.) should only appear in page objects, not in tests or flows.

Catches:

• Direct page locator calls: page.getByTestId(), app.page.locator()
• Chained locator calls on variables: someLocator.locator(), category.getByText()

Note: Selectors inside expect() calls are allowed by default (allowInExpect: true). This recognizes that assertions often need to check specific elements.

// Bad - Direct page locator in test
test('creates workflow', async ({ app }) => {
  await app.page.getByTestId('new-workflow-btn').click(); // Leaked selector
});

// Bad - Chained locator on returned Locator
test('finds links', async ({ app }) => {
  const category = app.settings.getCategory('nodes');
  const links = category.locator('a[href*="/workflow/"]'); // Leaked selector
});

// Good - Selector encapsulated in page object
test('creates workflow', async ({ app }) => {
  await app.workflows.create(); // Implementation hidden
});

// Good - Page object returns the specific element
test('finds links', async ({ app }) => {
  const links = app.settings.getWorkflowLinks('nodes'); // Selector in page object
});

no-page-in-flow

Severity: warning

Flows/composables shouldn't access page directly. They should work through page objects.

// Bad - Flow reaching into page internals
export class WorkflowComposer {
  async createAndRun() {
    await this.app.page.getByTestId('run-btn').click(); // Direct page access
  }
}

// Good - Flow uses page objects
export class WorkflowComposer {
  async createAndRun() {
    await this.app.canvas.runWorkflow(); // Through page object
  }
}

Certain page-level operations are allowed (configurable via allowPatterns):

• page.keyboard.* - Keyboard shortcuts
• page.evaluate() - JavaScript execution
• page.waitForLoadState() - Navigation waits
• page.waitForURL() - URL assertions
• page.reload() - Page refresh

api-purity

Severity: warning

Raw HTTP calls (request.get(), fetch()) should go through API service classes, not appear directly in tests.

// Bad - Raw HTTP in test
test('gets workflows', async ({ request }) => {
  const response = await request.get('/api/workflows');
});

// Good - Through API service
test('gets workflows', async ({ api }) => {
  const workflows = await api.workflows.list();
});

Code Quality Rules

dead-code

Severity: warning | Fixable: yes

Detects unused public methods and properties in page objects. If nothing references a method, it's probably dead code.

export class WorkflowPage {
  async usedMethod() { /* called from tests */ }
  async unusedMethod() { /* nobody calls this */ } // Violation
}

deduplication

Severity: warning

Detects the same getByTestId() value used in multiple page object files. Each test ID should have one authoritative home.

// pages/WorkflowPage.ts
this.page.getByTestId('save-button'); // Duplicate

// pages/SettingsPage.ts
this.page.getByTestId('save-button'); // Duplicate

Note: Same ID within a single file is allowed (e.g., helper methods).

test-data-hygiene

Severity: warning

Detects:

• Orphaned test data - Workflow/expectation files not referenced by any test
• Generic names - Files named test.json, data.json, workflow_1.json
• Ticket-only names - Files named just CAT-123.json without description

workflows/
  webhook-with-retry.json     Good - Descriptive
  test.json                   Bad - Generic
  CAT-123.json                Bad - Ticket-only
  unused-workflow.json        Bad - Orphaned (if not referenced)

duplicate-logic

Severity: warning

Detects duplicate code using AST structural fingerprinting. Finds copy-paste patterns across tests, pages, flows, and helpers by normalizing code structure (ignoring variable names and literal values).

Catches:

• Duplicate methods - Same logic in multiple page objects
• Duplicate tests - Copy-pasted test bodies across files
• Tests duplicating methods - Test code that reimplements existing page object methods

// pages/WorkflowPage.ts
async saveWorkflow() {
  await this.page.click('#save');
  await this.page.fill('#name', 'workflow');
  await this.page.waitForSelector('.saved');
}

// pages/CredentialPage.ts - Violation: duplicates WorkflowPage.saveWorkflow()
async saveCredential() {
  await this.page.click('#save');
  await this.page.fill('#name', 'credential');
  await this.page.waitForSelector('.saved');
}

Threshold: Methods/tests with fewer than 2 statements are ignored (configurable via minStatements).

Configuration Reference

interface JanitorConfig {
  /** Root directory for the Playwright test suite (absolute path) */
  rootDir: string;

  /** Directory patterns for different artifact types */
  patterns: {
    pages: string[];
    components: string[];
    flows: string[];
    tests: string[];
    services: string[];
    fixtures: string[];
    helpers: string[];
    factories: string[];
    testData: string[];
  };

  /** Files to exclude from page analysis (facades, base classes) */
  excludeFromPages: string[];

  /** Facade configuration - the main aggregator that exposes page objects */
  facade: {
    file: string;       // Path relative to rootDir
    className: string;  // e.g., 'AppPage'
    excludeTypes: string[]; // Types to exclude from mapping
  };

  /** The fixture object name used in tests */
  fixtureObjectName: string;  // e.g., 'app', 'po', 'n8n'

  /** The API fixture/helper object name */
  apiFixtureName: string;  // e.g., 'api'

  /** Patterns indicating raw API calls */
  rawApiPatterns: RegExp[];

  /** What you call the middle layer */
  flowLayerName: string;  // e.g., 'Composable', 'Action', 'Flow'

  /** Rule-specific configuration */
  rules: {
    [ruleId: string]: {
      enabled?: boolean;
      severity?: 'error' | 'warning' | 'off';
      allowPatterns?: RegExp[];
    };
  };

  /** Tags that exclude specs from discovery (e.g., ['@wip', '@local-only']) */
  skipTags: string[];

  /** Prefix for extracting capabilities from tags (default: '@capability:') */
  capabilityPrefix: string;

  /** Orchestration configuration for distributing specs across shards */
  orchestration: {
    /** Path to metrics JSON file (relative to rootDir) */
    metricsPath?: string;
    /** Default duration for specs without metrics (ms, default: 60000) */
    defaultDuration: number;
    /** Max group duration before splitting (ms, default: 300000) */
    maxGroupDuration: number;
    /** Only include specs matching this path prefix */
    specFilter?: string;
  };

  /** TCR configuration */
  tcr: {
    testCommand: string;  // Default: 'npx playwright test'
    workerCount?: number; // Default: 1
    allowedTestCommands?: string[]; // Restrict --test-command to these
  };
}

Disabling Rules

Globally

// janitor.config.js
export default defineConfig({
  // ...
  rules: {
    'dead-code': { enabled: false },
    'deduplication': { severity: 'off' },
  },
});

Per-Rule Allow Patterns

rules: {
  'no-page-in-flow': {
    allowPatterns: [
      /\.page\.keyboard/,     // Allow keyboard shortcuts
      /\.page\.evaluate/,     // Allow JS execution
    ],
  },
}

Programmatic API

import {
  defineConfig,
  runAnalysis,
  createDefaultRunner,
  RuleRunner,
  BaseRule,
  toJSON,
  toConsole,
} from '@n8n/playwright-janitor';

// Simple usage
const report = runAnalysis(config);

// Custom runner with specific rules
const runner = new RuleRunner();
runner.registerRule(new BoundaryProtectionRule());
runner.registerRule(new SelectorPurityRule());

const { project, root } = createProject(config.rootDir);
const report = runner.run(project, root);

// Output
toConsole(report);           // Human-readable
const json = toJSON(report); // Machine-readable

Writing Custom Rules

Extend BaseRule to create custom rules:

import { SyntaxKind } from 'ts-morph';
import { BaseRule } from '@n8n/playwright-janitor';
import type { Project, SourceFile, Violation } from '@n8n/playwright-janitor';

export class NoHardcodedUrlsRule extends BaseRule {
  readonly id = 'no-hardcoded-urls';
  readonly name = 'No Hardcoded URLs';
  readonly description = 'URLs should come from configuration';
  readonly severity = 'warning' as const;

  getTargetGlobs(): string[] {
    return ['**/*.ts']; // Analyze all TypeScript files
  }

  analyze(project: Project, files: SourceFile[]): Violation[] {
    const violations: Violation[] = [];

    for (const file of files) {
      // Use ts-morph to analyze the AST
      const stringLiterals = file.getDescendantsOfKind(SyntaxKind.StringLiteral);

      for (const literal of stringLiterals) {
        const value = literal.getLiteralText();
        if (value.startsWith('http://') || value.startsWith('https://')) {
          violations.push(
            this.createViolation(
              file,
              literal.getStartLineNumber(),
              literal.getStart() - literal.getStartLinePos(),
              `Hardcoded URL found: ${value}`,
              'Move URL to configuration or environment variable',
            ),
          );
        }
      }
    }

    return violations;
  }
}

// Register with runner
const runner = createDefaultRunner();
runner.registerRule(new NoHardcodedUrlsRule());

Integration with CI

Add to your CI pipeline:

# .github/workflows/test.yml
- name: Run Janitor
  run: pnpm janitor

The janitor exits with code 1 if violations are found, failing the build.

Philosophy

The janitor embodies these principles:

1. Catch problems early - Static analysis finds issues without running tests
2. Enforce by default - Good architecture should be the path of least resistance
3. Configurable, not prescriptive - Your naming conventions, your layers, your rules
4. Fixable where possible - Dead code removal shouldn't require manual work
5. AI-friendly guardrails - When AI generates test code, the janitor keeps it clean

TCR (Test && Commit || Revert)

The janitor includes tools for TCR-style development workflows, where changes are automatically committed if tests pass, or reverted if they fail.

Recommended Workflow:

1. Run pnpm janitor to identify violations
2. Fix violations in your code
3. Debug and verify - Run affected tests manually, check behavior
4. TCR as the last step - Once confident, use TCR to safely commit

Why TCR last? Running TCR immediately after fixing violations doesn't give you time to debug if something breaks. The revert happens automatically, and you lose your work. Fix → verify → TCR ensures you only commit working code.

Impact Analysis

Determine which tests are affected by file changes:

import { createProject, ImpactAnalyzer, formatImpactConsole } from '@n8n/playwright-janitor';

const { project } = createProject('./');
const analyzer = new ImpactAnalyzer(project);

// Analyze impact of changed files
const result = analyzer.analyze(['pages/CanvasPage.ts', 'pages/WorkflowPage.ts']);

console.log(`Affected tests: ${result.affectedTests.length}`);
result.affectedTests.forEach(t => console.log(`  - ${t}`));

// Or use the formatter
formatImpactConsole(result, true); // verbose mode

Method-Level Impact

Track which tests use specific page object methods:

import { createProject, MethodUsageAnalyzer } from '@n8n/playwright-janitor';

const { project } = createProject('./');
const analyzer = new MethodUsageAnalyzer(project);

// Build a complete index of method usages
const index = analyzer.buildIndex();
console.log(`Tracked ${Object.keys(index.methods).length} methods`);

// Find tests affected by a specific method change
const impact = analyzer.getMethodImpact('CanvasPage.addNode');
console.log(`Tests using CanvasPage.addNode():`);
impact.affectedTestFiles.forEach(t => console.log(`  - ${t}`));

AST Diff Analysis

Detect which methods changed in a file compared to git HEAD:

import { diffFileMethods, formatDiffConsole } from '@n8n/playwright-janitor';

const result = diffFileMethods('pages/CanvasPage.ts', 'HEAD');

console.log(`Changed methods:`);
for (const change of result.changedMethods) {
  const symbol = change.changeType === 'added' ? '+'
    : change.changeType === 'removed' ? '-' : '~';
  console.log(`  ${symbol} ${change.className}.${change.methodName}`);
}

TCR Executor

Run the full TCR workflow:

import { TcrExecutor } from '@n8n/playwright-janitor';

const tcr = new TcrExecutor();

// Dry run - analyze but don't commit/revert
const result = await tcr.run({ verbose: true });

console.log(`Changed files: ${result.changedFiles.length}`);
console.log(`Changed methods: ${result.changedMethods.length}`);
console.log(`Affected tests: ${result.affectedTests.length}`);
console.log(`Tests passed: ${result.testsPassed}`);

// Execute TCR - commit on success, revert on failure
const executed = await tcr.run({
  execute: true,
  commitMessage: 'feat: Add new workflow feature'
});

console.log(`Action taken: ${executed.action}`); // 'commit' | 'revert' | 'dry-run'

Codebase Inventory

Generate a complete inventory of your test codebase:

import { createProject, InventoryAnalyzer, formatInventoryJSON } from '@n8n/playwright-janitor';

const { project } = createProject('./');
const analyzer = new InventoryAnalyzer(project);

const inventory = analyzer.generate();

console.log(`Pages: ${inventory.summary.totalPages}`);
console.log(`Components: ${inventory.summary.totalComponents}`);
console.log(`Flows: ${inventory.summary.totalFlows}`);
console.log(`Test files: ${inventory.summary.totalTestFiles}`);
console.log(`Total tests: ${inventory.summary.totalTests}`);
console.log(`Total methods: ${inventory.summary.totalMethods}`);

// JSON output
const json = formatInventoryJSON(inventory);

TCR Types

interface TcrOptions {
  /** Git ref to compare against (default: HEAD) */
  baseRef?: string;
  /** Whether to actually commit/revert (false = dry run) */
  execute?: boolean;
  /** Custom commit message */
  commitMessage?: string;
  /** Verbose output */
  verbose?: boolean;
  /** Override test command (default: from config or 'npx playwright test') */
  testCommand?: string;
}

interface TcrResult {
  changedFiles: string[];
  changedMethods: MethodChange[];
  affectedTests: string[];
  testsRun: string[];
  testsPassed: boolean;
  action: 'commit' | 'revert' | 'dry-run';
  durationMs: number;
}

interface MethodChange {
  className: string;
  methodName: string;
  changeType: 'added' | 'removed' | 'modified';
}

interface ImpactResult {
  changedFiles: string[];
  affectedFiles: string[];
  affectedTests: string[];
  graph: Record<string, string[]>;
}

Development

pnpm install
pnpm build
pnpm test

License

MIT