mirror of
https://github.com/podman-desktop/podman-desktop
synced 2026-04-21 09:37:22 +00:00
chore(test): update testing skill to better conform to project standards and requirements (#16793)
Signed-off-by: Vladimir Lazar <vlazar@redhat.com>
This commit is contained in:
Vladimir Lazar
GitHub
parent
ebe39f26a0
commit
26cf08ca12
5 changed files with 961 additions and 1004 deletions
|
|
@ -1,370 +1,373 @@
|
|||
---
|
||||
name: playwright-testing
|
||||
description: >-
|
||||
Guide for writing, organizing, and maintaining Playwright end-to-end tests
|
||||
using the Page Object Model pattern. Use when creating new Playwright tests,
|
||||
building page objects, debugging test failures, configuring Playwright, or
|
||||
when the user asks about E2E testing, test automation, or Playwright.
|
||||
Guide for writing, updating, and maintaining Playwright end-to-end tests for
|
||||
Podman Desktop using the project's Electron runner, custom fixtures, and Page
|
||||
Object Model hierarchy. Use when creating new E2E spec files, building or
|
||||
modifying page objects, updating the test framework or utilities, debugging
|
||||
test failures, adding smoke tests, or when the user asks about Playwright
|
||||
tests, test automation, spec files, page models, or the E2E test structure.
|
||||
---
|
||||
|
||||
# Playwright Test Automation
|
||||
# Playwright E2E Testing for Podman Desktop
|
||||
|
||||
## Core Principles
|
||||
|
||||
- **Page Object Model (POM)**: Every page/component gets its own class. Tests never touch raw locators directly.
|
||||
- **Resilient locators**: Prefer accessible selectors (`getByRole`, `getByLabel`, `getByText`) over CSS/XPath.
|
||||
- **Explicit waits over sleeps**: Use `expect` with auto-retry, `waitForSelector`, or polling — never arbitrary `waitForTimeout` in production tests.
|
||||
- **Serial vs parallel**: Use `test.describe.serial()` only when tests share state or depend on ordering. Default to parallel.
|
||||
- **Fail fast, report clearly**: Every assertion should produce a readable error. Prefer `toBeVisible`, `toHaveText`, `toContainText` over generic `toBeTruthy`.
|
||||
This skill covers the Podman Desktop-specific Playwright framework. It is an
|
||||
Electron desktop app — tests launch the app via `Runner`, not a browser URL.
|
||||
|
||||
## Project Structure
|
||||
|
||||
Organize test code with clear separation of concerns:
|
||||
|
||||
```
|
||||
playwright.config.ts # At repo root, not under tests/
|
||||
tests/playwright/
|
||||
├── playwright.config.ts # Runner configuration
|
||||
├── package.json # Dependencies and scripts
|
||||
├── tsconfig.json # TypeScript config
|
||||
├── package.json # @podman-desktop/tests-playwright
|
||||
├── tsconfig.json # Path alias: /@/ → src/
|
||||
├── vite.config.js # Library build config
|
||||
├── src/
|
||||
│ ├── *.spec.ts # Test spec files
|
||||
│ └── model/ # Page Object Models
|
||||
│ ├── *-page.ts # Full-page POM classes
|
||||
│ └── *-component.ts # Reusable component POMs
|
||||
├── resources/ # Test fixtures and data files
|
||||
└── output/ # Traces, videos, reports (gitignored)
|
||||
│ ├── specs/ # Test spec files
|
||||
│ │ ├── *-smoke.spec.ts # Smoke test suites
|
||||
│ │ ├── *.spec.ts # Other specs
|
||||
│ │ └── z-*.spec.ts # Ordered-last suites (Podman machine)
|
||||
│ ├── special-specs/ # Isolated/focused suites
|
||||
│ │ ├── installation/
|
||||
│ │ ├── managed-configuration/
|
||||
│ │ ├── podman-remote/
|
||||
│ │ └── ui-stress/
|
||||
│ ├── model/ # Page Object Models
|
||||
│ │ ├── pages/ # Page POMs (base-page, main-page, details-page, ...)
|
||||
│ │ │ ├── base-page.ts # Abstract base: holds readonly page
|
||||
│ │ │ ├── main-page.ts # Abstract: list pages (Images, Containers, Volumes, Pods)
|
||||
│ │ │ ├── details-page.ts # Abstract: resource detail views
|
||||
│ │ │ ├── *-page.ts # Concrete page POMs
|
||||
│ │ │ ├── forms/ # Form-specific POMs
|
||||
│ │ │ └── compose-onboarding/ # Compose onboarding flow POMs
|
||||
│ │ ├── workbench/ # App shell POMs
|
||||
│ │ │ ├── navigation.ts # NavigationBar — sidebar nav, returns page POMs
|
||||
│ │ │ └── status-bar.ts # StatusBar
|
||||
│ │ ├── components/ # Reusable widget POMs
|
||||
│ │ └── core/ # Enums, types, states, settings helpers
|
||||
│ ├── runner/ # Electron app launcher
|
||||
│ │ ├── podman-desktop-runner.ts # Runner singleton
|
||||
│ │ └── runner-options.ts # RunnerOptions config class
|
||||
│ ├── utility/ # Shared helpers
|
||||
│ │ ├── fixtures.ts # Custom Playwright test + fixtures
|
||||
│ │ ├── operations.ts # UI workflow helpers
|
||||
│ │ ├── wait.ts # waitUntil, waitWhile, waitForPodmanMachineStartup
|
||||
│ │ ├── kubernetes.ts # K8s helpers
|
||||
│ │ ├── cluster-operations.ts # Kind cluster helpers
|
||||
│ │ ├── platform.ts # isLinux, isMac, isWindows, isCI
|
||||
│ │ └── auth-utils.ts # Browser-based auth flows (Chromium)
|
||||
│ ├── setupFiles/ # Feature gate helpers
|
||||
│ └── globalSetup/ # Setup/teardown (exported, not in config)
|
||||
├── resources/ # Containerfiles, YAML, fixtures
|
||||
└── output/ # Traces, videos, reports (gitignored)
|
||||
```
|
||||
|
||||
## Writing Page Object Models
|
||||
## Imports and Fixtures
|
||||
|
||||
### Structure
|
||||
|
||||
Each POM class encapsulates a page or distinct UI region:
|
||||
**Always** import `test` and `expect` from the project fixtures, not from `@playwright/test`:
|
||||
|
||||
```typescript
|
||||
import type { Locator, Page } from '@playwright/test';
|
||||
import { expect as playExpect } from '@playwright/test';
|
||||
import { expect as playExpect, test } from '/@/utility/fixtures';
|
||||
```
|
||||
|
||||
export class ExamplePage {
|
||||
All source imports use the `/@/` path alias, which Vite resolves to `src/`.
|
||||
|
||||
### Available Test Fixtures
|
||||
|
||||
The custom `test` provides these fixtures:
|
||||
|
||||
| Fixture | Type | Description |
|
||||
| --------------- | --------------- | ----------------------------------------------------------- |
|
||||
| `runner` | `Runner` | Electron app lifecycle (singleton via `Runner.getInstance`) |
|
||||
| `page` | `Page` | The Electron renderer window (`runner.getPage()`) |
|
||||
| `navigationBar` | `NavigationBar` | Sidebar navigation POM |
|
||||
| `welcomePage` | `WelcomePage` | Welcome/onboarding page POM |
|
||||
| `statusBar` | `StatusBar` | Bottom status bar POM |
|
||||
| `runnerOptions` | `RunnerOptions` | Configurable option (override with `test.use`) |
|
||||
|
||||
Destructure these directly in test hooks and test functions:
|
||||
|
||||
```typescript
|
||||
test.beforeAll(async ({ runner, welcomePage, page }) => { ... });
|
||||
test('my test', async ({ navigationBar }) => { ... });
|
||||
```
|
||||
|
||||
## Page Object Model Hierarchy
|
||||
|
||||
### Three-Level Inheritance
|
||||
|
||||
```
|
||||
BasePage (abstract)
|
||||
├── MainPage (abstract) — list pages with tables (Images, Containers, Volumes, Pods)
|
||||
│ ├── ImagesPage
|
||||
│ ├── ContainersPage
|
||||
│ ├── VolumesPage
|
||||
│ └── PodsPage
|
||||
├── DetailsPage (abstract) — resource detail views with tabs
|
||||
│ ├── ImageDetailsPage
|
||||
│ ├── ContainerDetailsPage
|
||||
│ └── ...
|
||||
└── Other concrete pages (WelcomePage, DashboardPage, ...)
|
||||
|
||||
Workbench classes (not BasePage subclasses):
|
||||
├── NavigationBar
|
||||
└── StatusBar
|
||||
```
|
||||
|
||||
### BasePage
|
||||
|
||||
All page POMs extend `BasePage`:
|
||||
|
||||
```typescript
|
||||
import type { Page } from '@playwright/test';
|
||||
|
||||
export abstract class BasePage {
|
||||
readonly page: Page;
|
||||
readonly heading: Locator;
|
||||
readonly submitButton: Locator;
|
||||
readonly nameInput: Locator;
|
||||
|
||||
constructor(page: Page) {
|
||||
this.page = page;
|
||||
this.heading = page.getByRole('heading', { name: 'Example' });
|
||||
this.submitButton = page.getByRole('button', { name: 'Submit' });
|
||||
this.nameInput = page.getByLabel('Name');
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
async fillAndSubmit(name: string): Promise<void> {
|
||||
await this.nameInput.fill(name);
|
||||
await playExpect(this.submitButton).toBeEnabled();
|
||||
await this.submitButton.click();
|
||||
### MainPage
|
||||
|
||||
For list pages with header, search, content regions, and table rows:
|
||||
|
||||
```typescript
|
||||
export abstract class MainPage extends BasePage {
|
||||
readonly title: string;
|
||||
readonly mainPage: Locator;
|
||||
readonly header: Locator;
|
||||
readonly search: Locator;
|
||||
readonly content: Locator;
|
||||
readonly additionalActions: Locator;
|
||||
readonly heading: Locator;
|
||||
|
||||
constructor(page: Page, title: string) {
|
||||
super(page);
|
||||
this.title = title;
|
||||
this.mainPage = page.getByRole('region', { name: this.title });
|
||||
this.header = this.mainPage.getByRole('region', { name: 'header' });
|
||||
this.search = this.mainPage.getByRole('region', { name: 'search' });
|
||||
this.content = this.mainPage.getByRole('region', { name: 'content' });
|
||||
this.additionalActions = this.header.getByRole('group', { name: 'additionalActions' });
|
||||
this.heading = this.header.getByRole('heading', { name: this.title });
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Concrete pages call `super(page, 'images')`, `super(page, 'containers')`, etc.
|
||||
|
||||
### DetailsPage
|
||||
|
||||
For resource detail views with tabs, breadcrumb, and control actions:
|
||||
|
||||
```typescript
|
||||
export abstract class DetailsPage extends BasePage {
|
||||
readonly header: Locator;
|
||||
readonly tabs: Locator;
|
||||
readonly tabContent: Locator;
|
||||
readonly closeButton: Locator;
|
||||
readonly backLink: Locator;
|
||||
readonly heading: Locator;
|
||||
|
||||
constructor(page: Page, resourceName: string) {
|
||||
super(page);
|
||||
this.tabContent = page.getByRole('region', { name: 'Tab Content' });
|
||||
this.header = page.getByRole('region', { name: 'Header' });
|
||||
this.tabs = page.getByRole('region', { name: 'Tabs' });
|
||||
this.heading = this.header.getByRole('heading', { name: resourceName });
|
||||
// ... breadcrumb, close, back locators
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### POM Rules
|
||||
|
||||
1. **Declare all locators as `readonly` properties** in the constructor. This makes them discoverable and reusable.
|
||||
2. **Methods return other POM instances** when navigation occurs (e.g., `openSettings()` returns `SettingsPage`).
|
||||
3. **Keep assertions in tests**, not in POM methods — unless the method explicitly validates state (e.g., `waitUntilBuildFinished`).
|
||||
4. **Use descriptive method names**: `fillAndSubmit`, `selectArchitecture`, `waitForImageReady` — not `doAction` or `step2`.
|
||||
|
||||
### Navigation POMs
|
||||
|
||||
For apps with sidebar/navbar navigation, create a navigation POM that returns page POMs:
|
||||
1. **Extend the correct base class**: `MainPage` for list pages, `DetailsPage` for detail views, `BasePage` for other pages
|
||||
2. **Declare all locators as `readonly` in the constructor** — eager `Locator` chains, not lazy getters
|
||||
3. **Wrap every method body in `test.step()`** for trace readability:
|
||||
|
||||
```typescript
|
||||
export class NavigationBar {
|
||||
readonly page: Page;
|
||||
readonly dashboardLink: Locator;
|
||||
readonly settingsLink: Locator;
|
||||
|
||||
constructor(page: Page) {
|
||||
this.page = page;
|
||||
this.dashboardLink = page.getByRole('link', { name: 'Dashboard' });
|
||||
this.settingsLink = page.getByRole('link', { name: 'Settings' });
|
||||
}
|
||||
|
||||
async openDashboard(): Promise<DashboardPage> {
|
||||
await this.dashboardLink.click();
|
||||
const dashboard = new DashboardPage(this.page);
|
||||
await playExpect(dashboard.heading).toBeVisible();
|
||||
return dashboard;
|
||||
}
|
||||
async pullImage(image: string): Promise<ImagesPage> {
|
||||
return test.step(`Pull image: ${image}`, async () => {
|
||||
const pullImagePage = await this.openPullImage();
|
||||
await playExpect(pullImagePage.heading).toBeVisible();
|
||||
return await pullImagePage.pullImage(image);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## Writing Tests
|
||||
4. **Navigation methods return POM instances** — e.g. `openPullImage()` returns `PullImagePage`
|
||||
5. **Use `playExpect`** (aliased from `@playwright/test`) inside POM files for assertions
|
||||
6. **Import `test` from `@playwright/test`** in POM files (for `test.step`), but from `/@/utility/fixtures` in spec files
|
||||
|
||||
### Test File Template
|
||||
### NavigationBar
|
||||
|
||||
Returns page POMs from sidebar navigation. Each method wraps in `test.step()`:
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
import { ExamplePage } from './model/example-page';
|
||||
async openImages(): Promise<ImagesPage> {
|
||||
return test.step('Open Images page', async () => {
|
||||
await playExpect(this.imagesLink).toBeVisible({ timeout: 10_000 });
|
||||
await this.imagesLink.click({ force: true });
|
||||
return new ImagesPage(this.page);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
test.describe('Feature Name', () => {
|
||||
test('should do the expected thing', async ({ page }) => {
|
||||
const examplePage = new ExamplePage(page);
|
||||
await examplePage.fillAndSubmit('test-name');
|
||||
await expect(page.getByText('Success')).toBeVisible();
|
||||
## Writing Spec Files
|
||||
|
||||
### Template
|
||||
|
||||
```typescript
|
||||
import { RunnerOptions } from '/@/runner/runner-options';
|
||||
import { expect as playExpect, test } from '/@/utility/fixtures';
|
||||
import { waitForPodmanMachineStartup } from '/@/utility/wait';
|
||||
|
||||
// Optional: override runner options for isolated profile
|
||||
test.use({ runnerOptions: new RunnerOptions({ customFolder: 'my-feature' }) });
|
||||
|
||||
test.beforeAll(async ({ runner, welcomePage, page }) => {
|
||||
runner.setVideoAndTraceName('my-feature-e2e');
|
||||
await welcomePage.handleWelcomePage(true);
|
||||
await waitForPodmanMachineStartup(page);
|
||||
});
|
||||
|
||||
test.afterAll(async ({ runner }) => {
|
||||
await runner.close();
|
||||
});
|
||||
|
||||
test.describe.serial('Feature name', { tag: '@smoke' }, () => {
|
||||
test.describe.configure({ retries: 1 });
|
||||
|
||||
test('first test', async ({ navigationBar }) => {
|
||||
const imagesPage = await navigationBar.openImages();
|
||||
await playExpect(imagesPage.heading).toBeVisible();
|
||||
// ... test body
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Lifecycle Hooks
|
||||
### Key Patterns
|
||||
|
||||
- **Serial suites**: Use `test.describe.serial()` — most Podman Desktop E2E tests share Electron state
|
||||
- **Tags**: `{ tag: '@smoke' }`, `{ tag: '@k8s_e2e' }`, `{ tag: ['@smoke', '@windows_sanity'] }`
|
||||
- **Retries**: `test.describe.configure({ retries: 1 })` inside the describe block
|
||||
- **Timeouts**: `test.setTimeout(180_000)` per test or in `beforeAll`
|
||||
- **Conditional skip**: `test.skip(isLinux, 'Not supported on Linux')`
|
||||
- **Runner options**: `test.use({ runnerOptions: new RunnerOptions({ ... }) })` for custom profiles
|
||||
- **Cleanup in afterAll**: Always wrap in `try/finally` with `runner.close()` in `finally`
|
||||
|
||||
### Naming Conventions
|
||||
|
||||
- Spec files: `kebab-case-smoke.spec.ts` (use `-smoke` suffix for smoke tests)
|
||||
- Prefix with `z-` for suites that must run last (e.g. `z-podman-machine-tests.spec.ts`)
|
||||
- POM files: `feature-page.ts` in `model/pages/`, `feature-component.ts` in `model/components/`
|
||||
|
||||
## Runner and RunnerOptions
|
||||
|
||||
### Runner Lifecycle
|
||||
|
||||
`Runner` is a singleton that launches the Electron app:
|
||||
|
||||
1. `Runner.getInstance({ runnerOptions })` — creates/reuses the singleton, calls `electron.launch()`
|
||||
2. `runner.getPage()` — returns the `Page` from `firstWindow()`
|
||||
3. `runner.setVideoAndTraceName('name')` — sets artifact naming (call in `beforeAll`)
|
||||
4. `runner.close()` — stops tracing, closes app, saves artifacts
|
||||
|
||||
### RunnerOptions
|
||||
|
||||
Configure with `test.use()`:
|
||||
|
||||
```typescript
|
||||
test.beforeAll(async ({ browser }) => {
|
||||
// One-time setup: seed data, start services
|
||||
});
|
||||
|
||||
test.afterAll(async () => {
|
||||
// Cleanup: remove test data, close connections
|
||||
});
|
||||
|
||||
test.beforeEach(async ({ page }) => {
|
||||
// Per-test setup: navigate to starting page
|
||||
test.use({
|
||||
runnerOptions: new RunnerOptions({
|
||||
customFolder: 'my-test-profile', // isolated profile directory
|
||||
extensionsDisabled: ['podman'], // disable specific extensions
|
||||
autoUpdate: false, // disable auto-update checks
|
||||
saveTracesOnPass: true, // keep traces even on pass
|
||||
customSettings: { key: 'value' }, // inject settings.json values
|
||||
}),
|
||||
});
|
||||
```
|
||||
|
||||
### Serial Test Suites
|
||||
### Environment Variables
|
||||
|
||||
Use `test.describe.serial()` when tests must run in order and share state:
|
||||
| Variable | Purpose |
|
||||
| ----------------------- | ----------------------------------------------------------------------- |
|
||||
| `PODMAN_DESKTOP_BINARY` | Path to packaged binary (mutually exclusive with `PODMAN_DESKTOP_ARGS`) |
|
||||
| `PODMAN_DESKTOP_ARGS` | Path to repo for dev mode |
|
||||
| `KEEP_TRACES_ON_PASS` | Retain traces on passing tests |
|
||||
| `KEEP_VIDEOS_ON_PASS` | Retain videos on passing tests |
|
||||
|
||||
## Wait Utilities
|
||||
|
||||
Use the project's wait helpers from `/@/utility/wait`, not custom polling:
|
||||
|
||||
```typescript
|
||||
let imageBuilt = false;
|
||||
import { waitUntil, waitWhile, waitForPodmanMachineStartup } from '/@/utility/wait';
|
||||
|
||||
test.describe.serial('Image Build Pipeline', () => {
|
||||
test('build image', async ({ page }) => {
|
||||
// ...build logic...
|
||||
imageBuilt = true;
|
||||
});
|
||||
|
||||
test('verify image exists', async ({ page }) => {
|
||||
test.skip(!imageBuilt, 'Build failed, skipping verification');
|
||||
// ...verification...
|
||||
});
|
||||
});
|
||||
await waitUntil(() => someCondition(), { timeout: 10_000, diff: 500, message: 'Condition not met' });
|
||||
await waitWhile(() => dialogIsOpen(), { timeout: 5_000 });
|
||||
await waitForPodmanMachineStartup(page);
|
||||
```
|
||||
|
||||
### Conditional Skipping
|
||||
Parameters: `timeout` (ms, default 5000), `diff` (polling interval ms, default 500), `sendError` (throw on timeout, default true), `message` (error text).
|
||||
|
||||
## Locator Strategy
|
||||
|
||||
1. **`getByRole`** — primary choice, use `exact: true` when needed
|
||||
2. **`getByLabel`** — form inputs and ARIA-labeled elements
|
||||
3. **`getByText`** — visible text
|
||||
4. **`getByTestId`** — when no semantic option exists
|
||||
5. **CSS locators** — last resort
|
||||
|
||||
Scope locators to parent regions when possible:
|
||||
|
||||
```typescript
|
||||
test.skip(condition, 'reason for skipping');
|
||||
test.skip(os.platform() === 'linux', 'Not supported on Linux');
|
||||
test.skip(!!process.env.SKIP_FEATURE, 'Feature disabled via env');
|
||||
this.pullImageButton = this.additionalActions.getByRole('button', { name: 'Pull', exact: true });
|
||||
```
|
||||
|
||||
### Timeouts
|
||||
## Running Tests
|
||||
|
||||
Set timeouts explicitly for long-running operations:
|
||||
|
||||
```typescript
|
||||
test('long operation', async ({ page }) => {
|
||||
test.setTimeout(300_000); // 5 minutes
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
Use numeric separators for readability: `1_200_000` not `1200000`.
|
||||
|
||||
## Locator Strategy (Priority Order)
|
||||
|
||||
1. **`getByRole`** — best for accessibility and resilience
|
||||
2. **`getByLabel`** — form inputs and labeled elements
|
||||
3. **`getByText`** — visible text content
|
||||
4. **`getByTestId`** — when no semantic alternative exists
|
||||
5. **`locator('css')`** — last resort
|
||||
|
||||
### Locator Examples
|
||||
|
||||
```typescript
|
||||
page.getByRole('button', { name: 'Submit' });
|
||||
page.getByRole('heading', { name: 'Dashboard' });
|
||||
page.getByLabel('Email address');
|
||||
page.getByText('Welcome back');
|
||||
page.getByTestId('user-avatar');
|
||||
page.getByRole('link', { name: /settings/i });
|
||||
```
|
||||
|
||||
### Avoid
|
||||
|
||||
- **`page.locator('#id')`** — fragile, breaks on refactor
|
||||
- **`page.locator('.className')`** — fragile, implementation detail
|
||||
- **`page.locator('xpath=...')`** — brittle and unreadable
|
||||
|
||||
## Assertions
|
||||
|
||||
### Auto-Retrying Assertions (Preferred)
|
||||
|
||||
```typescript
|
||||
await expect(locator).toBeVisible();
|
||||
await expect(locator).toBeEnabled();
|
||||
await expect(locator).toHaveText('Expected');
|
||||
await expect(locator).toContainText('partial');
|
||||
await expect(locator).toHaveValue('input-value');
|
||||
await expect(locator).toBeChecked();
|
||||
await expect(locator).not.toBeVisible();
|
||||
```
|
||||
|
||||
### Polling Assertions
|
||||
|
||||
For conditions that need periodic re-evaluation:
|
||||
|
||||
```typescript
|
||||
await expect.poll(async () => await checkSomeCondition(), { timeout: 30_000 }).toBeTruthy();
|
||||
```
|
||||
|
||||
### Soft Assertions
|
||||
|
||||
Continue test execution after failure (for gathering multiple failures):
|
||||
|
||||
```typescript
|
||||
await expect.soft(locator).toBeVisible();
|
||||
await expect.soft(locator).toHaveText('expected');
|
||||
```
|
||||
|
||||
## Waiting Patterns
|
||||
|
||||
### Prefer Built-in Auto-Wait
|
||||
|
||||
Playwright actions (`click`, `fill`, `check`) auto-wait for actionability. Don't add explicit waits before them unless there's a specific timing issue.
|
||||
|
||||
### When You Need Explicit Waits
|
||||
|
||||
```typescript
|
||||
await page.waitForLoadState('networkidle');
|
||||
await page.waitForURL('**/dashboard');
|
||||
await locator.waitFor({ state: 'visible', timeout: 10_000 });
|
||||
```
|
||||
|
||||
### Polling Helper Pattern
|
||||
|
||||
For custom conditions without built-in waits:
|
||||
|
||||
```typescript
|
||||
async function waitUntil(
|
||||
fn: () => Promise<boolean>,
|
||||
opts: { timeout: number; interval?: number; message?: string },
|
||||
): Promise<void> {
|
||||
const { timeout, interval = 1000, message = 'Condition not met' } = opts;
|
||||
const start = Date.now();
|
||||
while (Date.now() - start < timeout) {
|
||||
if (await fn()) return;
|
||||
await new Promise(r => setTimeout(r, interval));
|
||||
}
|
||||
throw new Error(`${message} (after ${timeout}ms)`);
|
||||
}
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### `playwright.config.ts` Template
|
||||
|
||||
```typescript
|
||||
import { defineConfig, devices } from '@playwright/test';
|
||||
|
||||
export default defineConfig({
|
||||
outputDir: './output/',
|
||||
workers: 1,
|
||||
retries: process.env.CI ? 2 : 0,
|
||||
timeout: 60_000,
|
||||
expect: { timeout: 10_000 },
|
||||
|
||||
reporter: [
|
||||
['list'],
|
||||
['junit', { outputFile: './tests/output/junit-results.xml' }],
|
||||
['json', { outputFile: './tests/output/json-results.json' }],
|
||||
['html', { open: 'never', outputFolder: './tests/output/html-results/' }],
|
||||
],
|
||||
|
||||
use: {
|
||||
trace: 'on-first-retry',
|
||||
video: 'retain-on-failure',
|
||||
screenshot: 'only-on-failure',
|
||||
},
|
||||
|
||||
projects: [
|
||||
{
|
||||
name: 'chromium',
|
||||
use: { ...devices['Desktop Chrome'] },
|
||||
},
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
## Debugging Tests
|
||||
|
||||
### Interactive Mode
|
||||
Tests execute from the **repo root** (not from `tests/playwright/`):
|
||||
|
||||
```bash
|
||||
npx playwright test --ui
|
||||
# All E2E tests (excluding k8s)
|
||||
npx playwright test tests/playwright/src/specs/ --grep-invert @k8s_e2e
|
||||
|
||||
# Smoke tests only
|
||||
npx playwright test tests/playwright/src/specs/ --grep @smoke
|
||||
|
||||
# Single spec file
|
||||
npx playwright test tests/playwright/src/specs/image-smoke.spec.ts
|
||||
|
||||
# View report
|
||||
pnpm exec playwright show-report tests/playwright/output/html-results
|
||||
```
|
||||
|
||||
### Headed Mode
|
||||
## Troubleshooting
|
||||
|
||||
```bash
|
||||
npx playwright test --headed
|
||||
```
|
||||
### Podman machine stuck in STARTING
|
||||
|
||||
### Debug Mode (Step Through)
|
||||
The `waitForPodmanMachineStartup` utility handles this by resetting via CLI. If tests time out waiting for RUNNING state, check that Podman is installed and the machine provider is available.
|
||||
|
||||
```bash
|
||||
npx playwright test --debug
|
||||
```
|
||||
### No Container Engine
|
||||
|
||||
### Trace Viewer
|
||||
Some POM methods (e.g. `openPullImage`) use `waitWhile(() => this.noContainerEngine())` to gate on engine availability. If tests fail with "No Container Engine", the Podman machine likely didn't start.
|
||||
|
||||
```bash
|
||||
npx playwright show-trace output/trace.zip
|
||||
```
|
||||
### Platform-specific skips
|
||||
|
||||
### Within Tests
|
||||
Use helpers from `/@/utility/platform`:
|
||||
|
||||
```typescript
|
||||
await page.pause(); // Opens inspector
|
||||
console.log(await locator.innerHTML());
|
||||
await page.screenshot({ path: 'debug.png' });
|
||||
import { isLinux, isMac, isWindows, isCI } from '/@/utility/platform';
|
||||
test.skip(isLinux, 'Not supported on Linux');
|
||||
```
|
||||
|
||||
## Webview / Electron Testing
|
||||
|
||||
When testing Electron apps or extensions with webviews:
|
||||
|
||||
```typescript
|
||||
async function handleWebview(runner: Runner): Promise<[Page, Page]> {
|
||||
const page = runner.getPage();
|
||||
// Navigate to the extension's entry point
|
||||
const extensionButton = page.getByRole('link', { name: 'My Extension' });
|
||||
await expect(extensionButton).toBeEnabled();
|
||||
await extensionButton.click();
|
||||
|
||||
// Wait for the webview to load
|
||||
const webView = page.getByRole('document', { name: 'Webview Label' });
|
||||
await expect(webView).toBeVisible();
|
||||
|
||||
// Access webview's separate page context
|
||||
const [mainPage, webViewPage] = runner.getElectronApp().windows();
|
||||
return [mainPage, webViewPage];
|
||||
}
|
||||
```
|
||||
|
||||
POMs that operate on webviews accept both `page` and `webview` parameters and use `webview` for locators inside the webview content.
|
||||
|
||||
## CI/CD Considerations
|
||||
|
||||
- Run headless with `xvfb-maybe` on Linux: `xvfb-maybe --auto-servernum -- npx playwright test`
|
||||
- Set `SKIP_INSTALLATION` or similar env vars to control test scope in CI
|
||||
- Use `test.skip(!!isCI && condition)` for platform-specific CI skips
|
||||
- Configure retries in CI: `retries: process.env.CI ? 2 : 0`
|
||||
- Archive `output/` directory for traces, screenshots, and video on failure
|
||||
- Use JUnit reporter for CI test result integration
|
||||
|
||||
## Additional Resources
|
||||
|
||||
- For detailed API patterns and advanced topics, see [reference.md](reference.md)
|
||||
- For concrete test examples and patterns, see [examples.md](examples.md)
|
||||
- For the project's wait utilities, operation helpers, and framework API, see [references/reference.md](references/reference.md)
|
||||
- For concrete examples from actual spec files, see [references/examples.md](references/examples.md)
|
||||
|
|
|
|||
|
|
@ -1,362 +0,0 @@
|
|||
# Playwright Test Examples
|
||||
|
||||
## Example 1: Basic Page Object + Test
|
||||
|
||||
### Page Object
|
||||
|
||||
```typescript
|
||||
import type { Locator, Page } from '@playwright/test';
|
||||
import { expect as playExpect } from '@playwright/test';
|
||||
|
||||
export class LoginPage {
|
||||
readonly page: Page;
|
||||
readonly heading: Locator;
|
||||
readonly emailInput: Locator;
|
||||
readonly passwordInput: Locator;
|
||||
readonly loginButton: Locator;
|
||||
readonly errorMessage: Locator;
|
||||
|
||||
constructor(page: Page) {
|
||||
this.page = page;
|
||||
this.heading = page.getByRole('heading', { name: 'Sign In' });
|
||||
this.emailInput = page.getByLabel('Email');
|
||||
this.passwordInput = page.getByLabel('Password');
|
||||
this.loginButton = page.getByRole('button', { name: 'Sign In' });
|
||||
this.errorMessage = page.getByRole('alert');
|
||||
}
|
||||
|
||||
async login(email: string, password: string): Promise<void> {
|
||||
await this.emailInput.fill(email);
|
||||
await this.passwordInput.fill(password);
|
||||
await this.loginButton.click();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Test
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
import { LoginPage } from './model/login-page';
|
||||
|
||||
test.describe('Authentication', () => {
|
||||
test('successful login redirects to dashboard', async ({ page }) => {
|
||||
await page.goto('/login');
|
||||
const loginPage = new LoginPage(page);
|
||||
await expect(loginPage.heading).toBeVisible();
|
||||
|
||||
await loginPage.login('user@example.com', 'password123');
|
||||
await expect(page).toHaveURL(/.*dashboard/);
|
||||
});
|
||||
|
||||
test('invalid credentials show error', async ({ page }) => {
|
||||
await page.goto('/login');
|
||||
const loginPage = new LoginPage(page);
|
||||
|
||||
await loginPage.login('wrong@example.com', 'badpassword');
|
||||
await expect(loginPage.errorMessage).toBeVisible();
|
||||
await expect(loginPage.errorMessage).toContainText('Invalid credentials');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## Example 2: Serial Suite with Shared State
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
let itemCreated = false;
|
||||
|
||||
test.describe.serial('CRUD Operations', () => {
|
||||
test('create item', async ({ page }) => {
|
||||
test.setTimeout(60_000);
|
||||
await page.goto('/items/new');
|
||||
|
||||
await page.getByLabel('Name').fill('Test Item');
|
||||
await page.getByLabel('Description').fill('A test item');
|
||||
await page.getByRole('button', { name: 'Create' }).click();
|
||||
|
||||
await expect(page.getByText('Item created successfully')).toBeVisible();
|
||||
itemCreated = true;
|
||||
});
|
||||
|
||||
test('read item', async ({ page }) => {
|
||||
test.skip(!itemCreated, 'Create failed, skipping read');
|
||||
await page.goto('/items');
|
||||
|
||||
const row = page.getByRole('row').filter({ hasText: 'Test Item' });
|
||||
await expect(row).toBeVisible();
|
||||
await expect(row.getByRole('cell').nth(1)).toHaveText('A test item');
|
||||
});
|
||||
|
||||
test('update item', async ({ page }) => {
|
||||
test.skip(!itemCreated, 'Create failed, skipping update');
|
||||
await page.goto('/items');
|
||||
|
||||
const row = page.getByRole('row').filter({ hasText: 'Test Item' });
|
||||
await row.getByRole('button', { name: 'Edit' }).click();
|
||||
|
||||
await page.getByLabel('Name').fill('Updated Item');
|
||||
await page.getByRole('button', { name: 'Save' }).click();
|
||||
await expect(page.getByText('Item updated')).toBeVisible();
|
||||
});
|
||||
|
||||
test('delete item', async ({ page }) => {
|
||||
test.skip(!itemCreated, 'Create failed, skipping delete');
|
||||
await page.goto('/items');
|
||||
|
||||
const row = page.getByRole('row').filter({ hasText: 'Updated Item' });
|
||||
await row.getByRole('button', { name: 'Delete' }).click();
|
||||
|
||||
page.on('dialog', d => d.accept());
|
||||
await expect(row).not.toBeVisible();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
## Example 3: Parameterized Tests
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
const formValidationCases = [
|
||||
{ field: 'Email', value: '', error: 'Email is required' },
|
||||
{ field: 'Email', value: 'not-email', error: 'Invalid email format' },
|
||||
{ field: 'Password', value: '12', error: 'Password must be at least 8 characters' },
|
||||
{ field: 'Username', value: 'a', error: 'Username must be at least 3 characters' },
|
||||
];
|
||||
|
||||
for (const tc of formValidationCases) {
|
||||
test(`validation: ${tc.field} with "${tc.value}"`, async ({ page }) => {
|
||||
await page.goto('/register');
|
||||
if (tc.value) {
|
||||
await page.getByLabel(tc.field).fill(tc.value);
|
||||
}
|
||||
await page.getByRole('button', { name: 'Register' }).click();
|
||||
await expect(page.getByText(tc.error)).toBeVisible();
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## Example 4: Navigation POM Returning Other POMs
|
||||
|
||||
```typescript
|
||||
import type { Locator, Page } from '@playwright/test';
|
||||
import { expect as playExpect } from '@playwright/test';
|
||||
import { DashboardPage } from './dashboard-page';
|
||||
import { SettingsPage } from './settings-page';
|
||||
import { ProfilePage } from './profile-page';
|
||||
|
||||
export class AppNavigationBar {
|
||||
readonly page: Page;
|
||||
readonly dashboardLink: Locator;
|
||||
readonly settingsLink: Locator;
|
||||
readonly profileLink: Locator;
|
||||
|
||||
constructor(page: Page) {
|
||||
this.page = page;
|
||||
this.dashboardLink = page.getByRole('link', { name: 'Dashboard' });
|
||||
this.settingsLink = page.getByRole('link', { name: 'Settings' });
|
||||
this.profileLink = page.getByRole('link', { name: 'Profile' });
|
||||
}
|
||||
|
||||
async openDashboard(): Promise<DashboardPage> {
|
||||
await this.dashboardLink.click();
|
||||
const dashboardPage = new DashboardPage(this.page);
|
||||
await playExpect(dashboardPage.heading).toBeVisible();
|
||||
return dashboardPage;
|
||||
}
|
||||
|
||||
async openSettings(): Promise<SettingsPage> {
|
||||
await this.settingsLink.click();
|
||||
const settingsPage = new SettingsPage(this.page);
|
||||
await playExpect(settingsPage.heading).toBeVisible();
|
||||
return settingsPage;
|
||||
}
|
||||
|
||||
async openProfile(): Promise<ProfilePage> {
|
||||
await this.profileLink.click();
|
||||
const profilePage = new ProfilePage(this.page);
|
||||
await playExpect(profilePage.heading).toBeVisible();
|
||||
return profilePage;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Example 5: Custom Fixtures
|
||||
|
||||
```typescript
|
||||
import { test as base, expect } from '@playwright/test';
|
||||
import { DashboardPage } from './model/dashboard-page';
|
||||
import { ApiClient } from './helpers/api-client';
|
||||
|
||||
type TestFixtures = {
|
||||
dashboardPage: DashboardPage;
|
||||
apiClient: ApiClient;
|
||||
};
|
||||
|
||||
export const test = base.extend<TestFixtures>({
|
||||
dashboardPage: async ({ page }, use) => {
|
||||
await page.goto('/dashboard');
|
||||
const dashboardPage = new DashboardPage(page);
|
||||
await expect(dashboardPage.heading).toBeVisible();
|
||||
await use(dashboardPage);
|
||||
},
|
||||
|
||||
apiClient: async ({ request }, use) => {
|
||||
const client = new ApiClient(request);
|
||||
await use(client);
|
||||
},
|
||||
});
|
||||
|
||||
// Usage in tests
|
||||
test('dashboard shows user data', async ({ dashboardPage, apiClient }) => {
|
||||
const userData = await apiClient.getUser('test-id');
|
||||
await expect(dashboardPage.userName).toHaveText(userData.name);
|
||||
});
|
||||
```
|
||||
|
||||
## Example 6: Polling for Async State
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('build completes within timeout', async ({ page }) => {
|
||||
test.setTimeout(300_000);
|
||||
|
||||
await page.getByRole('button', { name: 'Start Build' }).click();
|
||||
|
||||
const statusIndicator = page.getByRole('status');
|
||||
|
||||
await expect
|
||||
.poll(
|
||||
async () => {
|
||||
const text = await statusIndicator.textContent();
|
||||
return text?.toLowerCase();
|
||||
},
|
||||
{
|
||||
timeout: 240_000,
|
||||
intervals: [5_000, 10_000, 15_000],
|
||||
message: 'Build did not complete in time',
|
||||
},
|
||||
)
|
||||
.toMatch(/success|error/);
|
||||
|
||||
await expect(statusIndicator).toHaveText('Success');
|
||||
});
|
||||
```
|
||||
|
||||
## Example 7: Network Mocking in Tests
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('displays users from API', async ({ page }) => {
|
||||
await page.route('**/api/users', route =>
|
||||
route.fulfill({
|
||||
status: 200,
|
||||
contentType: 'application/json',
|
||||
body: JSON.stringify([
|
||||
{ id: 1, name: 'Alice', role: 'Admin' },
|
||||
{ id: 2, name: 'Bob', role: 'User' },
|
||||
]),
|
||||
}),
|
||||
);
|
||||
|
||||
await page.goto('/users');
|
||||
const rows = page.getByRole('row');
|
||||
await expect(rows).toHaveCount(3); // header + 2 data rows
|
||||
await expect(rows.nth(1)).toContainText('Alice');
|
||||
await expect(rows.nth(2)).toContainText('Bob');
|
||||
});
|
||||
|
||||
test('handles API error gracefully', async ({ page }) => {
|
||||
await page.route('**/api/users', route => route.fulfill({ status: 500, body: 'Internal Server Error' }));
|
||||
|
||||
await page.goto('/users');
|
||||
await expect(page.getByText('Failed to load users')).toBeVisible();
|
||||
await expect(page.getByRole('button', { name: 'Retry' })).toBeEnabled();
|
||||
});
|
||||
```
|
||||
|
||||
## Example 8: Webview / Electron POM
|
||||
|
||||
For extensions running in Electron with webview content:
|
||||
|
||||
```typescript
|
||||
import type { Locator, Page } from '@playwright/test';
|
||||
import { expect as playExpect } from '@playwright/test';
|
||||
|
||||
export class ExtensionPage {
|
||||
readonly page: Page;
|
||||
readonly webview: Page;
|
||||
readonly heading: Locator;
|
||||
readonly actionButton: Locator;
|
||||
|
||||
constructor(page: Page, webview: Page) {
|
||||
this.page = page;
|
||||
this.webview = webview;
|
||||
// Locators target the webview content, not the outer shell
|
||||
this.heading = webview.getByRole('heading', { name: 'My Extension' });
|
||||
this.actionButton = webview.getByRole('button', { name: 'Run Action' });
|
||||
}
|
||||
|
||||
async performAction(): Promise<void> {
|
||||
await playExpect(this.actionButton).toBeEnabled();
|
||||
await this.actionButton.click();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Example 9: Cleanup with afterAll
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
import * as fs from 'node:fs';
|
||||
|
||||
test.afterAll(async ({ page }) => {
|
||||
test.setTimeout(120_000);
|
||||
try {
|
||||
await page.goto('/admin/cleanup');
|
||||
await page.getByRole('button', { name: 'Delete Test Data' }).click();
|
||||
await expect(page.getByText('Cleaned up')).toBeVisible({ timeout: 60_000 });
|
||||
} catch (error) {
|
||||
console.log(`Cleanup error (non-fatal): ${error}`);
|
||||
} finally {
|
||||
if (fs.existsSync('tests/output/temp')) {
|
||||
fs.rmSync('tests/output/temp', { recursive: true });
|
||||
}
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
## Example 10: Checkbox and Select Interactions
|
||||
|
||||
```typescript
|
||||
import { test, expect } from '@playwright/test';
|
||||
|
||||
test('configure build options', async ({ page }) => {
|
||||
await page.goto('/build');
|
||||
|
||||
// Select from dropdown
|
||||
await page.getByLabel('image-select').selectOption({ label: 'my-image:latest' });
|
||||
|
||||
// Checkbox interactions
|
||||
const checkbox = page.getByLabel('enable-optimization');
|
||||
if (!(await checkbox.isChecked())) {
|
||||
await checkbox.check();
|
||||
}
|
||||
await expect(checkbox).toBeChecked();
|
||||
|
||||
// Radio button via role
|
||||
await page.getByRole('button', { name: 'amd64' }).click();
|
||||
|
||||
// Fill path input
|
||||
const pathInput = page.getByLabel('output-path');
|
||||
await pathInput.clear();
|
||||
await pathInput.fill('/tmp/output');
|
||||
await expect(pathInput).toHaveValue('/tmp/output');
|
||||
|
||||
await page.getByRole('button', { name: 'Build' }).click();
|
||||
});
|
||||
```
|
||||
|
|
@ -1,347 +0,0 @@
|
|||
# Playwright Reference
|
||||
|
||||
## Locator API Deep Dive
|
||||
|
||||
### Filtering Locators
|
||||
|
||||
Chain filters to narrow down results:
|
||||
|
||||
```typescript
|
||||
page.getByRole('listitem').filter({ hasText: 'Product A' });
|
||||
page.getByRole('listitem').filter({ has: page.getByRole('button', { name: 'Buy' }) });
|
||||
page.locator('article').filter({ hasNot: page.getByText('Draft') });
|
||||
```
|
||||
|
||||
### Nth Selection
|
||||
|
||||
```typescript
|
||||
page.getByRole('listitem').first();
|
||||
page.getByRole('listitem').last();
|
||||
page.getByRole('listitem').nth(2); // 0-indexed
|
||||
```
|
||||
|
||||
### Chaining Locators
|
||||
|
||||
```typescript
|
||||
const row = page.getByRole('row').filter({ hasText: 'John' });
|
||||
const editBtn = row.getByRole('button', { name: 'Edit' });
|
||||
```
|
||||
|
||||
### Frame Locators
|
||||
|
||||
For iframes and embedded content:
|
||||
|
||||
```typescript
|
||||
const frame = page.frameLocator('#my-iframe');
|
||||
await frame.getByRole('button', { name: 'Submit' }).click();
|
||||
```
|
||||
|
||||
## Advanced Assertions
|
||||
|
||||
### toPass — Retry a Block
|
||||
|
||||
Retry an entire block of code until it passes:
|
||||
|
||||
```typescript
|
||||
await expect(async () => {
|
||||
const response = await page.request.get('/api/status');
|
||||
expect(response.status()).toBe(200);
|
||||
const body = await response.json();
|
||||
expect(body.ready).toBe(true);
|
||||
}).toPass({ timeout: 30_000, intervals: [1_000, 2_000, 5_000] });
|
||||
```
|
||||
|
||||
### Custom Matchers
|
||||
|
||||
Extend `expect` with project-specific matchers:
|
||||
|
||||
```typescript
|
||||
expect.extend({
|
||||
async toBeInViewport(locator: Locator) {
|
||||
const box = await locator.boundingBox();
|
||||
const viewport = locator.page().viewportSize();
|
||||
const pass =
|
||||
box !== null &&
|
||||
viewport !== null &&
|
||||
box.x >= 0 &&
|
||||
box.y >= 0 &&
|
||||
box.x + box.width <= viewport.width &&
|
||||
box.y + box.height <= viewport.height;
|
||||
return { pass, message: () => `Expected element to be in viewport` };
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Snapshot Testing
|
||||
|
||||
```typescript
|
||||
await expect(page).toHaveScreenshot('homepage.png', { maxDiffPixels: 100 });
|
||||
await expect(locator).toHaveScreenshot('component.png');
|
||||
```
|
||||
|
||||
## Network Interception
|
||||
|
||||
### Mock API Responses
|
||||
|
||||
```typescript
|
||||
await page.route('**/api/users', route =>
|
||||
route.fulfill({
|
||||
status: 200,
|
||||
contentType: 'application/json',
|
||||
body: JSON.stringify([{ id: 1, name: 'Test User' }]),
|
||||
}),
|
||||
);
|
||||
```
|
||||
|
||||
### Wait for Network Requests
|
||||
|
||||
```typescript
|
||||
const responsePromise = page.waitForResponse('**/api/data');
|
||||
await page.getByRole('button', { name: 'Load' }).click();
|
||||
const response = await responsePromise;
|
||||
expect(response.status()).toBe(200);
|
||||
```
|
||||
|
||||
### Intercept and Modify
|
||||
|
||||
```typescript
|
||||
await page.route('**/api/config', async route => {
|
||||
const response = await route.fetch();
|
||||
const json = await response.json();
|
||||
json.featureFlag = true;
|
||||
await route.fulfill({ response, json });
|
||||
});
|
||||
```
|
||||
|
||||
## Authentication & State
|
||||
|
||||
### Storage State (Reuse Login)
|
||||
|
||||
```typescript
|
||||
// Save auth state after login
|
||||
await page.context().storageState({ path: 'auth-state.json' });
|
||||
|
||||
// Reuse in config
|
||||
export default defineConfig({
|
||||
projects: [
|
||||
{ name: 'setup', testMatch: /.*\.setup\.ts/ },
|
||||
{
|
||||
name: 'tests',
|
||||
dependencies: ['setup'],
|
||||
use: { storageState: 'auth-state.json' },
|
||||
},
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
## Fixtures
|
||||
|
||||
### Custom Fixtures
|
||||
|
||||
```typescript
|
||||
import { test as base } from '@playwright/test';
|
||||
import { DashboardPage } from './model/dashboard-page';
|
||||
|
||||
type MyFixtures = {
|
||||
dashboardPage: DashboardPage;
|
||||
};
|
||||
|
||||
export const test = base.extend<MyFixtures>({
|
||||
dashboardPage: async ({ page }, use) => {
|
||||
const dashboard = new DashboardPage(page);
|
||||
await page.goto('/dashboard');
|
||||
await use(dashboard);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Worker-Scoped Fixtures
|
||||
|
||||
Shared across all tests in a worker (useful for expensive setup):
|
||||
|
||||
```typescript
|
||||
type WorkerFixtures = {
|
||||
dbConnection: DatabaseClient;
|
||||
};
|
||||
|
||||
export const test = base.extend<{}, WorkerFixtures>({
|
||||
dbConnection: [
|
||||
async ({}, use) => {
|
||||
const db = await DatabaseClient.connect();
|
||||
await use(db);
|
||||
await db.disconnect();
|
||||
},
|
||||
{ scope: 'worker' },
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
## Parameterized Tests
|
||||
|
||||
### Loop-Based
|
||||
|
||||
```typescript
|
||||
const imageTypes = ['QCOW2', 'AMI', 'RAW', 'VMDK', 'ISO', 'VHD'];
|
||||
|
||||
for (const type of imageTypes) {
|
||||
test(`build ${type} image`, async ({ page }) => {
|
||||
// test body using type
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
### Data-Driven with Objects
|
||||
|
||||
```typescript
|
||||
const testCases = [
|
||||
{ name: 'valid email', input: 'user@test.com', expected: 'Success' },
|
||||
{ name: 'invalid email', input: 'not-an-email', expected: 'Invalid email' },
|
||||
];
|
||||
|
||||
for (const tc of testCases) {
|
||||
test(`form submission: ${tc.name}`, async ({ page }) => {
|
||||
await page.getByLabel('Email').fill(tc.input);
|
||||
await page.getByRole('button', { name: 'Submit' }).click();
|
||||
await expect(page.getByText(tc.expected)).toBeVisible();
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
## Multi-Page / Multi-Tab
|
||||
|
||||
```typescript
|
||||
const [newPage] = await Promise.all([
|
||||
page.context().waitForEvent('page'),
|
||||
page.getByRole('link', { name: 'Open in new tab' }).click(),
|
||||
]);
|
||||
await newPage.waitForLoadState();
|
||||
await expect(newPage.getByRole('heading')).toHaveText('New Page');
|
||||
```
|
||||
|
||||
## File Upload / Download
|
||||
|
||||
### Upload
|
||||
|
||||
```typescript
|
||||
await page.getByLabel('Upload file').setInputFiles('path/to/file.pdf');
|
||||
await page.getByLabel('Upload file').setInputFiles(['file1.pdf', 'file2.pdf']);
|
||||
```
|
||||
|
||||
### Download
|
||||
|
||||
```typescript
|
||||
const downloadPromise = page.waitForEvent('download');
|
||||
await page.getByRole('button', { name: 'Download' }).click();
|
||||
const download = await downloadPromise;
|
||||
await download.saveAs('path/to/save.pdf');
|
||||
```
|
||||
|
||||
## Dialog Handling
|
||||
|
||||
```typescript
|
||||
page.on('dialog', async dialog => {
|
||||
expect(dialog.message()).toContain('Are you sure?');
|
||||
await dialog.accept();
|
||||
});
|
||||
await page.getByRole('button', { name: 'Delete' }).click();
|
||||
```
|
||||
|
||||
## Parallel Execution Strategies
|
||||
|
||||
### Worker Isolation
|
||||
|
||||
Each worker gets its own browser context. Tests in the same file share a worker by default.
|
||||
|
||||
### Shard Across CI
|
||||
|
||||
```bash
|
||||
npx playwright test --shard=1/4
|
||||
npx playwright test --shard=2/4
|
||||
npx playwright test --shard=3/4
|
||||
npx playwright test --shard=4/4
|
||||
```
|
||||
|
||||
### Fully Parallel
|
||||
|
||||
```typescript
|
||||
// In config
|
||||
export default defineConfig({
|
||||
fullyParallel: true,
|
||||
});
|
||||
|
||||
// Or per-file
|
||||
test.describe.configure({ mode: 'parallel' });
|
||||
```
|
||||
|
||||
## Trace Configuration
|
||||
|
||||
```typescript
|
||||
// Always capture traces
|
||||
use: {
|
||||
trace: 'on';
|
||||
}
|
||||
|
||||
// Only on first retry (recommended for CI)
|
||||
use: {
|
||||
trace: 'on-first-retry';
|
||||
}
|
||||
|
||||
// Only on failure
|
||||
use: {
|
||||
trace: 'retain-on-failure';
|
||||
}
|
||||
```
|
||||
|
||||
## Global Setup / Teardown
|
||||
|
||||
```typescript
|
||||
// playwright.config.ts
|
||||
export default defineConfig({
|
||||
globalSetup: require.resolve('./global-setup'),
|
||||
globalTeardown: require.resolve('./global-teardown'),
|
||||
});
|
||||
|
||||
// global-setup.ts
|
||||
export default async function globalSetup() {
|
||||
// Start services, seed database, etc.
|
||||
}
|
||||
```
|
||||
|
||||
## Accessibility Testing
|
||||
|
||||
```typescript
|
||||
import AxeBuilder from '@axe-core/playwright';
|
||||
|
||||
test('page passes a11y checks', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
const results = await new AxeBuilder({ page }).analyze();
|
||||
expect(results.violations).toEqual([]);
|
||||
});
|
||||
```
|
||||
|
||||
## API Testing (Without Browser)
|
||||
|
||||
```typescript
|
||||
test('API returns user list', async ({ request }) => {
|
||||
const response = await request.get('/api/users');
|
||||
expect(response.ok()).toBeTruthy();
|
||||
const users = await response.json();
|
||||
expect(users.length).toBeGreaterThan(0);
|
||||
});
|
||||
```
|
||||
|
||||
## Useful CLI Commands
|
||||
|
||||
| Command | Purpose |
|
||||
| ---------------------------------------- | ------------------------ |
|
||||
| `npx playwright test` | Run all tests |
|
||||
| `npx playwright test file.spec.ts` | Run specific file |
|
||||
| `npx playwright test -g "test name"` | Run by test name |
|
||||
| `npx playwright test --headed` | Run with browser visible |
|
||||
| `npx playwright test --debug` | Step-through debugger |
|
||||
| `npx playwright test --ui` | Interactive UI mode |
|
||||
| `npx playwright show-trace trace.zip` | View trace file |
|
||||
| `npx playwright codegen url` | Record actions as code |
|
||||
| `npx playwright test --reporter=html` | Generate HTML report |
|
||||
| `npx playwright test --project=chromium` | Run specific project |
|
||||
| `npx playwright install` | Install browsers |
|
||||
397
.agents/skills/playwright-testing/references/examples.md
Normal file
397
.agents/skills/playwright-testing/references/examples.md
Normal file
|
|
@ -0,0 +1,397 @@
|
|||
# Examples
|
||||
|
||||
Concrete examples drawn from actual Podman Desktop spec files and POMs.
|
||||
|
||||
## Example 1: Minimal Smoke Test Spec
|
||||
|
||||
From `welcome-page-smoke.spec.ts` — the simplest spec pattern:
|
||||
|
||||
```typescript
|
||||
import { RunnerOptions } from '/@/runner/runner-options';
|
||||
import { expect as playExpect, test } from '/@/utility/fixtures';
|
||||
|
||||
test.use({ runnerOptions: new RunnerOptions({ customFolder: 'welcome-podman-desktop' }) });
|
||||
|
||||
test.beforeAll(async ({ runner }) => {
|
||||
runner.setVideoAndTraceName('welcome-page-e2e');
|
||||
});
|
||||
|
||||
test.afterAll(async ({ runner }) => {
|
||||
await runner.close();
|
||||
});
|
||||
|
||||
test.describe.serial(
|
||||
'Basic e2e verification of podman desktop start',
|
||||
{
|
||||
tag: ['@smoke', '@windows_sanity', '@macos_sanity'],
|
||||
},
|
||||
() => {
|
||||
test('Check the Welcome page is displayed', async ({ welcomePage }) => {
|
||||
await playExpect(welcomePage.welcomeMessage).toBeVisible();
|
||||
});
|
||||
|
||||
test('Telemetry checkbox is present and set to true', async ({ welcomePage }) => {
|
||||
await playExpect(welcomePage.telemetryConsent).toBeVisible();
|
||||
await playExpect(welcomePage.telemetryConsent).toBeChecked();
|
||||
});
|
||||
|
||||
test('Redirection from Welcome page to Dashboard works', async ({ welcomePage }) => {
|
||||
const dashboardPage = await welcomePage.closeWelcomePage();
|
||||
await playExpect(dashboardPage.heading).toBeVisible();
|
||||
});
|
||||
},
|
||||
);
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- Imports from `/@/utility/fixtures`
|
||||
- `test.use()` for isolated profile
|
||||
- `runner.setVideoAndTraceName()` in `beforeAll`
|
||||
- `runner.close()` in `afterAll`
|
||||
- `test.describe.serial` with tag array
|
||||
- Destructures `welcomePage` fixture directly
|
||||
|
||||
## Example 2: Full Smoke Test with Podman Machine Setup
|
||||
|
||||
From `image-smoke.spec.ts` — standard pattern for tests needing a running engine:
|
||||
|
||||
```typescript
|
||||
import { expect as playExpect, test } from '/@/utility/fixtures';
|
||||
import { waitForPodmanMachineStartup } from '/@/utility/wait';
|
||||
|
||||
const helloContainer = 'ghcr.io/podmandesktop-ci/hello';
|
||||
|
||||
test.beforeAll(async ({ runner, welcomePage, page }) => {
|
||||
runner.setVideoAndTraceName('pull-image-e2e');
|
||||
await welcomePage.handleWelcomePage(true);
|
||||
await waitForPodmanMachineStartup(page);
|
||||
});
|
||||
|
||||
test.afterAll(async ({ runner }) => {
|
||||
await runner.close();
|
||||
});
|
||||
|
||||
test.describe.serial('Image workflow verification', { tag: '@smoke' }, () => {
|
||||
test.describe.configure({ retries: 1 });
|
||||
|
||||
test('Pull image', async ({ navigationBar }) => {
|
||||
const imagesPage = await navigationBar.openImages();
|
||||
await playExpect(imagesPage.heading).toBeVisible();
|
||||
|
||||
const pullImagePage = await imagesPage.openPullImage();
|
||||
const updatedImages = await pullImagePage.pullImage(helloContainer);
|
||||
await playExpect(updatedImages.heading).toBeVisible({ timeout: 10_000 });
|
||||
|
||||
await playExpect
|
||||
.poll(async () => updatedImages.waitForImageExists(helloContainer, 30_000), { timeout: 0 })
|
||||
.toBeTruthy();
|
||||
});
|
||||
|
||||
test('Delete image', async ({ navigationBar }) => {
|
||||
let imagesPage = await navigationBar.openImages();
|
||||
const imageDetailPage = await imagesPage.openImageDetails(helloContainer);
|
||||
imagesPage = await imageDetailPage.deleteImage();
|
||||
|
||||
await playExpect
|
||||
.poll(async () => await imagesPage.waitForImageDelete(helloContainer, 60_000), { timeout: 0 })
|
||||
.toBeTruthy();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- `welcomePage.handleWelcomePage(true)` dismisses the welcome screen
|
||||
- `waitForPodmanMachineStartup(page)` ensures engine is ready
|
||||
- `test.describe.configure({ retries: 1 })` for flaky-tolerant CI
|
||||
- POM methods chain: `navigationBar.openImages()` → `imagesPage.openPullImage()` → `pullImagePage.pullImage()`
|
||||
- `playExpect.poll()` with `{ timeout: 0 }` wraps methods that have their own internal timeout
|
||||
|
||||
## Example 3: Concrete Page Object (MainPage Subclass)
|
||||
|
||||
Abridged from `images-page.ts`:
|
||||
|
||||
```typescript
|
||||
import type { Locator, Page } from '@playwright/test';
|
||||
import test, { expect as playExpect } from '@playwright/test';
|
||||
|
||||
import { handleConfirmationDialog } from '/@/utility/operations';
|
||||
import { waitUntil, waitWhile } from '/@/utility/wait';
|
||||
|
||||
import { BuildImagePage } from './build-image-page';
|
||||
import { ImageDetailsPage } from './image-details-page';
|
||||
import { MainPage } from './main-page';
|
||||
import { PullImagePage } from './pull-image-page';
|
||||
|
||||
export class ImagesPage extends MainPage {
|
||||
readonly pullImageButton: Locator;
|
||||
readonly pruneImagesButton: Locator;
|
||||
readonly buildImageButton: Locator;
|
||||
|
||||
constructor(page: Page) {
|
||||
super(page, 'images');
|
||||
this.pullImageButton = this.additionalActions.getByRole('button', { name: 'Pull', exact: true });
|
||||
this.pruneImagesButton = this.additionalActions.getByRole('button', { name: 'Prune', exact: true });
|
||||
this.buildImageButton = this.additionalActions.getByRole('button', { name: 'Build', exact: true });
|
||||
}
|
||||
|
||||
async openPullImage(): Promise<PullImagePage> {
|
||||
return test.step('Open pull image page', async () => {
|
||||
await waitWhile(() => this.noContainerEngine(), {
|
||||
timeout: 50_000,
|
||||
message: 'No Container Engine is available, cannot pull an image',
|
||||
});
|
||||
await this.pullImageButton.click();
|
||||
return new PullImagePage(this.page);
|
||||
});
|
||||
}
|
||||
|
||||
async pullImage(image: string): Promise<ImagesPage> {
|
||||
return test.step(`Pull image: ${image}`, async () => {
|
||||
const pullImagePage = await this.openPullImage();
|
||||
await playExpect(pullImagePage.heading).toBeVisible();
|
||||
return await pullImagePage.pullImage(image);
|
||||
});
|
||||
}
|
||||
|
||||
async openImageDetails(name: string): Promise<ImageDetailsPage> {
|
||||
return test.step(`Open image details page for image: ${name}`, async () => {
|
||||
const imageRow = await this.getImageRowByName(name);
|
||||
if (imageRow === undefined) {
|
||||
throw Error(`Image: '${name}' does not exist`);
|
||||
}
|
||||
const imageRowName = imageRow.getByRole('cell').nth(3);
|
||||
await imageRowName.click();
|
||||
return new ImageDetailsPage(this.page, name);
|
||||
});
|
||||
}
|
||||
|
||||
async waitForImageExists(name: string, timeout = 5_000): Promise<boolean> {
|
||||
return test.step(`Wait for image: ${name} to exist`, async () => {
|
||||
await waitUntil(async () => await this.imageExists(name), { timeout });
|
||||
return true;
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- Extends `MainPage` with `super(page, 'images')`
|
||||
- Locators scoped to inherited `this.additionalActions`
|
||||
- Every method wrapped in `test.step()`
|
||||
- Uses `waitWhile`/`waitUntil` from project utilities
|
||||
- Navigation methods return new POM instances
|
||||
- `handleConfirmationDialog` for modal dialogs
|
||||
|
||||
## Example 4: NavigationBar Usage
|
||||
|
||||
The `NavigationBar` POM provides navigation to all main pages:
|
||||
|
||||
```typescript
|
||||
// In a test — navigationBar comes from fixtures
|
||||
test('Navigate to containers', async ({ navigationBar }) => {
|
||||
const containersPage = await navigationBar.openContainers();
|
||||
await playExpect(containersPage.heading).toBeVisible();
|
||||
});
|
||||
|
||||
// Navigate through multiple pages in sequence
|
||||
test('Check all main pages', async ({ navigationBar }) => {
|
||||
const images = await navigationBar.openImages();
|
||||
await playExpect(images.heading).toBeVisible();
|
||||
|
||||
const containers = await navigationBar.openContainers();
|
||||
await playExpect(containers.heading).toBeVisible();
|
||||
|
||||
const volumes = await navigationBar.openVolumes();
|
||||
await playExpect(volumes.heading).toBeVisible();
|
||||
|
||||
const pods = await navigationBar.openPods();
|
||||
await playExpect(pods.heading).toBeVisible();
|
||||
});
|
||||
```
|
||||
|
||||
Available navigation methods:
|
||||
|
||||
- `openDashboard()` → `DashboardPage`
|
||||
- `openImages()` → `ImagesPage`
|
||||
- `openContainers()` → `ContainersPage`
|
||||
- `openPods()` → `PodsPage`
|
||||
- `openVolumes()` → `VolumesPage`
|
||||
- `openSettings()` → `SettingsBar`
|
||||
- `openKubernetes()` → `KubernetesBar`
|
||||
- `openExtensions()` → `ExtensionsPage`
|
||||
- `openNetworks()` → `NetworksPage`
|
||||
|
||||
## Example 5: Test with Timeouts and Build Resources
|
||||
|
||||
From `image-smoke.spec.ts` — building an image with file paths:
|
||||
|
||||
```typescript
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
import { ArchitectureType } from '/@/model/core/platforms';
|
||||
import { expect as playExpect, test } from '/@/utility/fixtures';
|
||||
|
||||
const __filename = fileURLToPath(import.meta.url);
|
||||
const __dirname = path.dirname(__filename);
|
||||
|
||||
test('Build image', async ({ navigationBar }) => {
|
||||
let imagesPage = await navigationBar.openImages();
|
||||
await playExpect(imagesPage.heading).toBeVisible();
|
||||
|
||||
const buildImagePage = await imagesPage.openBuildImage();
|
||||
await playExpect(buildImagePage.heading).toBeVisible();
|
||||
|
||||
const dockerfilePath = path.resolve(__dirname, '..', '..', 'resources', 'test-containerfile');
|
||||
const contextDirectory = path.resolve(__dirname, '..', '..', 'resources');
|
||||
|
||||
imagesPage = await buildImagePage.buildImage('build-image-test', dockerfilePath, contextDirectory);
|
||||
playExpect(await imagesPage.waitForImageExists('docker.io/library/build-image-test')).toBeTruthy();
|
||||
|
||||
const imageDetailsPage = await imagesPage.openImageDetails('docker.io/library/build-image-test');
|
||||
await playExpect(imageDetailsPage.heading).toBeVisible();
|
||||
imagesPage = await imageDetailsPage.deleteImage();
|
||||
playExpect(await imagesPage.waitForImageDelete('docker.io/library/build-image-test')).toBeTruthy();
|
||||
});
|
||||
|
||||
test('Build with target stage', async ({ navigationBar }) => {
|
||||
test.setTimeout(180_000);
|
||||
// ... similar but with stage parameter
|
||||
});
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- `__dirname` derived from `import.meta.url` (ESM)
|
||||
- Resources in `tests/playwright/resources/`
|
||||
- Path resolution relative to the spec file
|
||||
- `test.setTimeout()` for long-running operations
|
||||
|
||||
## Example 6: Conditional Skipping and Platform Checks
|
||||
|
||||
```typescript
|
||||
import { isLinux, isMac, isCI } from '/@/utility/platform';
|
||||
|
||||
// Skip at describe level
|
||||
test.describe.serial('Compose tests', { tag: '@smoke' }, () => {
|
||||
test.skip(isCI && isLinux, 'Compose not available on Linux CI');
|
||||
|
||||
test('deploy compose file', async ({ navigationBar }) => {
|
||||
// ...
|
||||
});
|
||||
});
|
||||
|
||||
// Skip individual tests
|
||||
test('macOS-only feature', async ({ page }) => {
|
||||
test.skip(!isMac, 'Only runs on macOS');
|
||||
// ...
|
||||
});
|
||||
|
||||
// Skip based on env var
|
||||
test('registry push', async ({ page }) => {
|
||||
test.skip(!process.env.REGISTRY_URL, 'No registry configured');
|
||||
// ...
|
||||
});
|
||||
```
|
||||
|
||||
## Example 7: Cleanup Pattern with try/finally
|
||||
|
||||
From typical `afterAll` blocks:
|
||||
|
||||
```typescript
|
||||
test.afterAll(async ({ runner, page }) => {
|
||||
test.setTimeout(90_000);
|
||||
try {
|
||||
const imagesPage = await new NavigationBar(page).openImages();
|
||||
await imagesPage.deleteAllUnusedImages();
|
||||
} catch (error) {
|
||||
console.log(`Cleanup error (non-fatal): ${error}`);
|
||||
} finally {
|
||||
await runner.close();
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
`runner.close()` must always be called, even if cleanup fails.
|
||||
|
||||
## Example 8: Polling with playExpect.poll
|
||||
|
||||
The codebase wraps methods that have internal timeouts with `{ timeout: 0 }`:
|
||||
|
||||
```typescript
|
||||
// The inner method (waitForImageExists) has its own timeout.
|
||||
// The outer poll with timeout: 0 prevents double-timeout issues.
|
||||
await playExpect
|
||||
.poll(async () => updatedImages.waitForImageExists(helloContainer, 30_000), { timeout: 0 })
|
||||
.toBeTruthy();
|
||||
|
||||
// Polling for a count to change
|
||||
await playExpect
|
||||
.poll(async () => await imagesPage.countImagesByName('<none>'), { timeout: 60_000 })
|
||||
.toBeGreaterThan(baselineCount);
|
||||
```
|
||||
|
||||
## Example 9: DetailsPage Subclass
|
||||
|
||||
Pattern for a resource details page:
|
||||
|
||||
```typescript
|
||||
import type { Locator, Page } from '@playwright/test';
|
||||
import test, { expect as playExpect } from '@playwright/test';
|
||||
|
||||
import { DetailsPage } from './details-page';
|
||||
import { ImagesPage } from './images-page';
|
||||
|
||||
export class ImageDetailsPage extends DetailsPage {
|
||||
readonly summaryTab: Locator;
|
||||
readonly historyTab: Locator;
|
||||
readonly inspectTab: Locator;
|
||||
|
||||
constructor(page: Page, imageName: string) {
|
||||
super(page, imageName);
|
||||
this.summaryTab = this.tabs.getByRole('link', { name: 'Summary' });
|
||||
this.historyTab = this.tabs.getByRole('link', { name: 'History' });
|
||||
this.inspectTab = this.tabs.getByRole('link', { name: 'Inspect' });
|
||||
}
|
||||
|
||||
async deleteImage(): Promise<ImagesPage> {
|
||||
return test.step('Delete image from details', async () => {
|
||||
const deleteButton = this.controlActions.getByRole('button', { name: 'Delete Image' });
|
||||
await playExpect(deleteButton).toBeEnabled();
|
||||
await deleteButton.click();
|
||||
await handleConfirmationDialog(this.page);
|
||||
return new ImagesPage(this.page);
|
||||
});
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Key points:
|
||||
|
||||
- Extends `DetailsPage` with resource name
|
||||
- Tab locators scoped to inherited `this.tabs`
|
||||
- Action buttons scoped to inherited `this.controlActions`
|
||||
- Returns parent list page POM after destructive actions
|
||||
|
||||
## Example 10: Using test.use for Custom Runner Options
|
||||
|
||||
```typescript
|
||||
import { RunnerOptions } from '/@/runner/runner-options';
|
||||
import { test } from '/@/utility/fixtures';
|
||||
|
||||
// Disable specific extensions for this test file
|
||||
test.use({
|
||||
runnerOptions: new RunnerOptions({
|
||||
customFolder: 'extension-test',
|
||||
extensionsDisabled: ['compose', 'kind'],
|
||||
autoUpdate: false,
|
||||
autoCheckUpdates: false,
|
||||
}),
|
||||
});
|
||||
|
||||
// Tests in this file get an isolated Podman Desktop profile
|
||||
// with compose and kind extensions disabled
|
||||
```
|
||||
266
.agents/skills/playwright-testing/references/reference.md
Normal file
266
.agents/skills/playwright-testing/references/reference.md
Normal file
|
|
@ -0,0 +1,266 @@
|
|||
# Framework Reference
|
||||
|
||||
Detailed reference for the Podman Desktop Playwright test framework internals.
|
||||
|
||||
## Runner Internals
|
||||
|
||||
### Singleton Pattern
|
||||
|
||||
`Runner` enforces a single Electron instance across all tests in a worker:
|
||||
|
||||
```typescript
|
||||
const runner = await Runner.getInstance({ runnerOptions });
|
||||
```
|
||||
|
||||
Internally this calls `electron.launch()` with options derived from `RunnerOptions`
|
||||
and environment variables, then stores `firstWindow()` as the `page`.
|
||||
|
||||
### Launch Options
|
||||
|
||||
The runner resolves its launch configuration from:
|
||||
|
||||
1. `PODMAN_DESKTOP_BINARY` env var — packaged app path
|
||||
2. `PODMAN_DESKTOP_ARGS` env var — path to repo root (dev mode, uses `node_modules/.bin/electron`)
|
||||
3. `RunnerOptions` — profile directory, settings, extensions, DevTools
|
||||
|
||||
These are mutually exclusive: `PODMAN_DESKTOP_BINARY` takes precedence.
|
||||
|
||||
### Tracing and Video
|
||||
|
||||
- `runner.setVideoAndTraceName('feature-e2e')` — call in `beforeAll` to name artifacts
|
||||
- On `runner.close()`: tracing stops, video saves, process terminates
|
||||
- By default, traces and videos are deleted on pass unless `KEEP_TRACES_ON_PASS` / `KEEP_VIDEOS_ON_PASS` env vars are set, or `RunnerOptions` has `saveTracesOnPass: true` / `saveVideosOnPass: true`
|
||||
|
||||
### Browser Window Access
|
||||
|
||||
For Electron-specific checks:
|
||||
|
||||
```typescript
|
||||
const browserWindow = await runner.getBrowserWindow();
|
||||
const state = await runner.getBrowserWindowState();
|
||||
// state: { isVisible, isDevToolsOpened, isCrashed, isFocused }
|
||||
```
|
||||
|
||||
## RunnerOptions Full API
|
||||
|
||||
```typescript
|
||||
new RunnerOptions({
|
||||
profile: '', // Profile suffix
|
||||
customFolder: 'podman-desktop', // Home directory subfolder
|
||||
customOutputFolder: 'tests/playwright/output/',
|
||||
openDevTools: 'none', // 'none' | 'right' | 'bottom' | 'undocked'
|
||||
autoUpdate: true,
|
||||
autoCheckUpdates: true,
|
||||
extensionsDisabled: [], // Array of extension IDs to disable
|
||||
aiLabModelUploadDisabled: false,
|
||||
binaryPath: undefined, // Override binary path
|
||||
saveTracesOnPass: false,
|
||||
saveVideosOnPass: false,
|
||||
customSettings: {}, // Injected into settings.json
|
||||
});
|
||||
```
|
||||
|
||||
The `createSettingsJson()` method serializes these into the settings file that
|
||||
Runner writes to the profile directory before launch.
|
||||
|
||||
## Fixtures Internals
|
||||
|
||||
### How Fixtures Wire Together
|
||||
|
||||
```
|
||||
runnerOptions (option, overridable via test.use)
|
||||
└─→ runner (Runner.getInstance)
|
||||
└─→ page (runner.getPage())
|
||||
├─→ navigationBar (new NavigationBar(page))
|
||||
├─→ welcomePage (new WelcomePage(page))
|
||||
└─→ statusBar (new StatusBar(page))
|
||||
```
|
||||
|
||||
### Overriding RunnerOptions
|
||||
|
||||
Use `test.use()` at the file level for custom profiles:
|
||||
|
||||
```typescript
|
||||
import { RunnerOptions } from '/@/runner/runner-options';
|
||||
import { test } from '/@/utility/fixtures';
|
||||
|
||||
test.use({
|
||||
runnerOptions: new RunnerOptions({
|
||||
customFolder: 'my-isolated-test',
|
||||
extensionsDisabled: ['podman'],
|
||||
}),
|
||||
});
|
||||
```
|
||||
|
||||
## Wait Utilities API
|
||||
|
||||
### waitUntil
|
||||
|
||||
Polls until a condition becomes `true`:
|
||||
|
||||
```typescript
|
||||
await waitUntil(() => someAsyncCheck(), {
|
||||
timeout: 5_000, // max wait (ms), default 5000
|
||||
diff: 500, // polling interval (ms), default 500
|
||||
sendError: true, // throw on timeout, default true
|
||||
message: '', // custom error message
|
||||
});
|
||||
```
|
||||
|
||||
### waitWhile
|
||||
|
||||
Polls until a condition becomes `false`:
|
||||
|
||||
```typescript
|
||||
await waitWhile(() => dialogIsStillOpen(), { timeout: 10_000, message: 'Dialog did not close' });
|
||||
```
|
||||
|
||||
### waitForPodmanMachineStartup
|
||||
|
||||
Ensures Podman machine is running before tests proceed:
|
||||
|
||||
```typescript
|
||||
await waitForPodmanMachineStartup(page, timeout);
|
||||
```
|
||||
|
||||
This:
|
||||
|
||||
1. Creates a Podman machine via CLI if needed
|
||||
2. Opens the Dashboard page
|
||||
3. Waits for the status label to show "RUNNING"
|
||||
4. If stuck in "STARTING", resets and retries once
|
||||
|
||||
## Operations Utility
|
||||
|
||||
`/@/utility/operations` provides workflow helpers used across specs:
|
||||
|
||||
- `handleConfirmationDialog(page, title?, confirm?, buttonName?, inputText?, timeout?, waitForClose?)` — handles modal confirmation dialogs
|
||||
- `untagImagesFromPodman(imageName)` — CLI-based image untagging
|
||||
- `deleteContainer(page, name)`, `deleteImage(page, name)` — cleanup helpers
|
||||
- `createPodmanMachineFromCLI()`, `resetPodmanMachinesFromCLI()` — machine management
|
||||
|
||||
## Platform Utilities
|
||||
|
||||
```typescript
|
||||
import { isLinux, isMac, isWindows, isCI } from '/@/utility/platform';
|
||||
```
|
||||
|
||||
These are boolean constants, not functions. Use directly in `test.skip()`:
|
||||
|
||||
```typescript
|
||||
test.skip(isLinux, 'Compose not supported on Linux CI');
|
||||
test.skip(isCI && isMac, 'Flaky on macOS CI');
|
||||
```
|
||||
|
||||
## Playwright Configuration
|
||||
|
||||
The config lives at the **repo root** (`playwright.config.ts`), not under `tests/playwright/`:
|
||||
|
||||
```typescript
|
||||
export default defineConfig({
|
||||
outputDir: 'tests/playwright/output/',
|
||||
workers: 1,
|
||||
timeout: 90_000,
|
||||
reporter: [
|
||||
['list'],
|
||||
['junit', { outputFile: 'tests/playwright/output/junit-results.xml' }],
|
||||
['json', { outputFile: 'tests/playwright/output/json-results.json' }],
|
||||
['html', { open: 'never', outputFolder: 'tests/playwright/output/html-results/' }],
|
||||
],
|
||||
projects: [
|
||||
{
|
||||
name: 'chromium',
|
||||
use: { ...devices['Desktop Chrome'] },
|
||||
},
|
||||
],
|
||||
});
|
||||
```
|
||||
|
||||
The `chromium` project is declared for tooling compatibility (`auth-utils.ts`).
|
||||
The actual app under test is Electron, launched by `Runner` — not by Playwright's
|
||||
project configuration.
|
||||
|
||||
Tracing and video are managed by `Runner`, not the config's `use` block.
|
||||
|
||||
## MainPage Shared Methods
|
||||
|
||||
`MainPage` provides methods inherited by all list pages:
|
||||
|
||||
- `pageIsEmpty()` — checks for "No Container Engine" or "No {title}" headings
|
||||
- `noContainerEngine()` — checks engine availability
|
||||
- `getAllTableRows()` — returns all `row` locators from the content table
|
||||
- `getRowByName(name, exact?)` — finds a row by its ARIA label
|
||||
- `waitForRowToExists(name, timeout?)` — polls until a row appears
|
||||
- `waitForRowToBeDelete(name, timeout?)` — polls until a row disappears
|
||||
- `countRowsFromTable()` — counts data rows (excludes header)
|
||||
- `getRowsFromTableByStatus(status)` — filters rows by status cell content
|
||||
- `checkAllRows()` / `uncheckAllRows()` — toggle selection checkbox
|
||||
|
||||
## DetailsPage Shared Methods
|
||||
|
||||
`DetailsPage` provides:
|
||||
|
||||
- `activateTab(tabName)` — clicks a tab in the detail view
|
||||
- `closeButton` / `backLink` — breadcrumb navigation locators
|
||||
- `heading` — scoped to the resource name
|
||||
- `controlActions` — action button group in the header
|
||||
|
||||
## Locator Patterns in This Codebase
|
||||
|
||||
### Region-Scoped Locators
|
||||
|
||||
The Podman Desktop UI uses ARIA regions extensively. POMs scope locators to
|
||||
their containing region:
|
||||
|
||||
```typescript
|
||||
this.mainPage = page.getByRole('region', { name: 'images' });
|
||||
this.header = this.mainPage.getByRole('region', { name: 'header' });
|
||||
this.additionalActions = this.header.getByRole('group', { name: 'additionalActions' });
|
||||
this.pullImageButton = this.additionalActions.getByRole('button', { name: 'Pull', exact: true });
|
||||
```
|
||||
|
||||
### Row-Based Locators
|
||||
|
||||
Table rows are found by intersecting `getByRole('row')` with `getByLabel(name)`:
|
||||
|
||||
```typescript
|
||||
const locator = this.page
|
||||
.getByRole('row')
|
||||
.and(this.page.getByLabel(name, { exact: true }))
|
||||
.first();
|
||||
```
|
||||
|
||||
### Force Clicks
|
||||
|
||||
Navigation links use `click({ force: true })` to bypass actionability checks
|
||||
when the Electron app's focus state can be unreliable:
|
||||
|
||||
```typescript
|
||||
await this.imagesLink.click({ force: true });
|
||||
```
|
||||
|
||||
## Test Tags
|
||||
|
||||
Tags control which tests run in CI:
|
||||
|
||||
| Tag | Meaning |
|
||||
| ----------------- | --------------------------------------------- |
|
||||
| `@smoke` | Core smoke tests — always run |
|
||||
| `@k8s_e2e` | Kubernetes tests — excluded from default runs |
|
||||
| `@windows_sanity` | Windows-specific sanity checks |
|
||||
| `@macos_sanity` | macOS-specific sanity checks |
|
||||
|
||||
Filter with `--grep` / `--grep-invert`:
|
||||
|
||||
```bash
|
||||
npx playwright test --grep @smoke
|
||||
npx playwright test --grep-invert @k8s_e2e
|
||||
```
|
||||
|
||||
## CI/CD Patterns
|
||||
|
||||
- Tests run from the monorepo root with `xvfb-maybe` on Linux
|
||||
- Single worker (`workers: 1`) — Electron app is a singleton
|
||||
- `PODMAN_DESKTOP_BINARY` env var points to the packaged app in CI
|
||||
- Output directory archived for traces, screenshots, and JUnit results
|
||||
- Retries configured per-describe: `test.describe.configure({ retries: 2 })`
|
||||
Loading…
Reference in a new issue