OpenMetadata/skills
IceS2 e9c87c6adb
chore(ingestion): drop pylint, expand ruff (#27774)
* chore(ingestion): drop pylint, expand ruff to Stage 2c

Replace pylint with a coherent ruff-only stack (Stage 2c of the modernize
roadmap). Pylint is dropped from dev deps and CI workflows; ruff selected
ruleset expanded to ~22 families covering style, bug catchers, hygiene,
and the pylint port (PLE/PLC/PLW/PLR with the noisy "too-many-X"
complexity caps + magic-value disabled).

What's selected (with rationale in pyproject.toml):
  E, W, F, I, N         — style + correctness baseline + naming
  UP                    — pyupgrade (py>=3.10 modernizations)
  B, C4, C90, RET, SIM, TRY  — bug catchers
  PIE, ICN, T20, TC, TID, PTH, PERF  — hygiene
  PLE, PLC, PLW, PLR    — pylint port (PLR complexity caps ignored)
  RUF                   — ruff-native (incl. RUF100 unused-noqa)

What's removed:
  - .pylintrc (root) — duplicate of the ingestion pylint config
  - [tool.pylint.*] block in ingestion/pyproject.toml (~140 lines)
  - ingestion/plugins/{print_checker,import_checker}.py + tests + README
    (replaced by built-in T20 + TID251 banned-api respectively)
  - pylint dep from ingestion/setup.py and openmetadata-airflow-apis/pyproject.toml
  - `make lint` Makefile target + the pylint invocation in py_format_check
  - dead pylint TODO comment + ignored test entry in noxfile.py

Cwd-stable config: ruff is invoked both from the repo root (pre-commit,
CI) and from ingestion/ (`make py_format_check`). The `src`,
`extend-exclude`, and per-file-ignores entries are listed twice — once
relative to ingestion/ and once with the `ingestion/` prefix — so
first-party isort detection and exclusions match in both invocations.

Grandfathering: ran `ruff check --add-noqa` once + format-stable
iteration. ~12,130 noqa directives across ~1,400 files. Cleanup is
deferred to follow-up PRs that drop noqas one rule at a time.

Documentation sweep: replaced `make lint` references in CLAUDE.md,
AGENTS.md, DEVELOPER.md, copilot-instructions, and 6 SKILL files with
the apply+verify shape `make py_format && make py_format_check`.
`make py_format` is NOT a strict superset of pylint — it only applies
auto-fixable violations; `make py_format_check` catches the rest.

Basedpyright baseline regenerated: ruff format reflowed multi-line
signatures in ~70 files, shifting type-error column positions. The
basedpyright baseline matches by (file path, error code, range), so
column shifts caused 19 entries to mis-align. Net diff is small
(154 lines in/out of the 13MB baseline.json) — purely positional.

Verified locally:
  - make py_format_check         → All checks passed
  - nox --no-venv -s static-checks → 0 errors, 0 warnings, 0 notes

* chore(ingestion): finish ruff swap — nox lint session + skill docs

Three remaining stale-tooling references after Stage 2c:

  - `ingestion/noxfile.py` `lint` session was still calling `black --check`,
    `isort --check-only`, `pycln --diff`. Those tools aren't installed
    anywhere (we dropped them from dev deps). Replace with the ruff
    equivalents that mirror `make py_format_check`.
  - `skills/standards/code_style.md`: stack listed as `black + isort +
    pycln`; line length claimed 88 (black default). Both wrong: stack is
    ruff, line length is 120.
  - `skills/connector-building/SKILL.md`: `make py_format` comment said
    `# black + isort + pycln`. Same swap.

* chore(ingestion): keep main's baseline + globally ignore TRY400

Per gitar-bot's review on PR #27774:

1. Main's PR #27728 promoted ~60 `logger.warning()` → `logger.error()`
   inside `except` blocks. Those changes landed on main with their own
   baseline updates. Our PR doesn't promote anything — the merge from
   origin/main brought those `error` calls along with their baseline
   entries.

   The bot interpreted the `# noqa: TRY400` we added next to those lines
   as us silencing the rule case-by-case. Cleaner: globally ignore
   TRY400 in pyproject.toml, with a comment explaining why the codebase's
   `logger.error(...)` + separate `logger.debug(traceback.format_exc())`
   pattern is intentional. Strip ~430 per-line `# noqa: TRY400` markers
   from source.

2. Document that `S101` in `per-file-ignores` is a forward-looking
   entry — flake8-bandit (`S`) is not yet selected, so the rule is
   no-op today; the entry stays so when `S` lands later, tests don't
   immediately error.

Reverts the platform pin and Linux Docker–generated baseline. Keep
main's baseline intact and let CI surface the exact column-shifted
entries; the team will decide whether to fix in-place (revert format
on affected files) or add per-line `# pyright: ignore` markers.

* chore(ingestion): regen baseline for new connector type debt

Main's baseline was stale relative to recently-added connectors
(McpConnection, CustomDriveConnection) that lack common attributes
like `hostPort`, `database`, `catalog` etc. — all sites that access
those attributes via the union-typed `serviceConnection.root.config`
fire `reportAttributeAccessIssue` errors that aren't baselined.

71 errors + 58 warnings absorbed. Local macOS regen; pushing to see
CI's drift count. Per the basedpyright-baseline-and-ci PR experience,
macOS↔Linux column drift on this size of regen has historically been
1-7 residuals.
2026-04-28 07:21:59 +02:00
..
.claude-plugin Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00
.github/workflows Add test-locally skill for local Docker deployment (#26323) 2026-03-08 23:35:11 -07:00
agents chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
code-review chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
commands Register connector skills in .claude/skills (#26320) 2026-03-08 22:01:27 -07:00
connector-audit chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
connector-building chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
connector-review Fivetran improvements (#27270) 2026-04-23 18:05:52 -07:00
connector-standards Register connector skills in .claude/skills (#26320) 2026-03-08 22:01:27 -07:00
hooks Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00
openmetadata-workflow chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
planning chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
standards chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
systematic-debugging Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00
tdd chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
test-enforcement Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00
test-locally Add test-locally skill for local Docker deployment (#26323) 2026-03-08 23:35:11 -07:00
verification chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
.markdownlint.yaml Add skills to build connectors (#26309) 2026-03-08 21:45:10 -07:00
README.md Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00

OpenMetadata Skills Plugin (v2.0.0)

Complete AI development workflow for OpenMetadata. Provides structured planning, test-driven development, 90% test coverage enforcement, code review, systematic debugging, and connector development skills.

Inspired by obra/superpowers and everything-claude-code, tailored for OpenMetadata's multi-language, schema-first architecture.

Installation

Claude Code (Plugin)

# From the OpenMetadata repo root
claude plugin install skills/

This installs all skills, agents, hooks, and commands. The plugin auto-loads the OpenMetadata workflow on session start.

Claude Code (In-Repo, Zero Setup)

If you're working in the OpenMetadata repo, .claude/settings.json provides project-level hooks automatically — no plugin install needed. The .claude/skills/ directory also provides skills directly.

Other Tools

Tool Method
Cursor Settings -> Rules -> Add Rule -> select skills/ directory
Codex Add skills/ to workspace context
GitHub Copilot Reference skills/ in workspace instructions
Windsurf Add skills/ to rules configuration
Any Skills follow the Agent Skills open standard

Workflow Overview

The plugin enforces a structured development workflow:

1. /planning          — Design before code (brainstorm, propose, get approval)
2. /tdd               — Write failing test, implement, refactor
3. /test-enforcement   — Verify 90% coverage, integration tests, Playwright E2E
4. /verification       — Show evidence of passing tests before claiming done
5. /code-review        — Two-stage review (spec compliance + code quality)

The openmetadata-workflow meta-skill is loaded at session start and directs Claude to use the appropriate skills for each task type.

Skills

Development Workflow Skills

Skill Command Purpose
OpenMetadata Workflow (auto-loaded) Meta-skill: routes tasks to the right workflow
Planning /planning Brainstorm approaches, get approval, create step-by-step plan
TDD /tdd RED-GREEN-REFACTOR for Java, Python, and TypeScript
Test Enforcement /test-enforcement Enforce 90% line coverage, integration tests, Playwright E2E
Systematic Debugging /systematic-debugging 4-phase root cause analysis
Code Review /code-review Two-stage review: spec compliance then code quality
Verification /verification Evidence-based completion — show test output, not claims

Connector Skills

Skill Command Purpose
Connector Building /scaffold-connector Scaffold a new connector with JSON Schema, Python boilerplate, and AI context
Connector Review /connector-review Review connector code against golden standards with multi-agent analysis
Connector Standards /connector-standards Load connector development standards into agent context
Test Locally /test-locally Build and deploy a full local Docker stack to test your connector

Agents

Language-Specific Reviewers

Agent Purpose
java-reviewer Review Java code — Dropwizard, Flyway, JUnit 5, spotless, 90% coverage
python-reviewer Review Python code — Pydantic 2.x, pytest, connector architecture, 90% coverage
frontend-reviewer Review TypeScript/React — core components, tw: prefix, i18n, no MUI, Jest/Playwright

Connector Agents

Agent Purpose
connector-researcher Research source system APIs, SDKs, auth, and data models
connector-validator Validate connector implementation against standards
comment-resolution-checker Verify PR review comments were substantively addressed

Hooks

The plugin includes hooks (hooks/hooks.json) that fire automatically:

Hook Event What it does
OpenMetadata Workflow SessionStart Loads the meta-skill to route tasks to the right workflow
Block --no-verify PreToolUse Prevents skipping pre-commit hooks
Java format reminder PostToolUse Reminds to run mvn spotless:apply after .java edits
Schema regeneration PostToolUse Reminds to run make generate after JSON schema edits

Additional hooks in .claude/settings.json (in-repo, no plugin needed):

  • Block MUI imports
  • Remind yarn parse-schema after connection schema edits
  • Warn about any type in TypeScript

Test Coverage Targets

The /test-enforcement skill enforces these targets:

Layer Target Tool
Java service (openmetadata-service) 90% line coverage JaCoCo
API endpoints 100% of changed endpoints have integration tests openmetadata-integration-tests
React components 90% line coverage Jest
UI features Playwright E2E for all user-facing changes Playwright
Python ingestion 90% line coverage pytest --cov

Standards

12 core standards + 11 source-type standards in standards/:

Core Standards

Standard Content
main.md Architecture overview, schema-first approach, service types
patterns.md Error handling, logging, pagination, auth, filters
testing.md Unit tests, integration tests, pytest patterns
code_style.md Python and JSON Schema conventions
schema.md Connection schema structure, $ref patterns
connection.md BaseConnection vs function patterns
service_spec.md DefaultDatabaseSpec vs BaseSpec
registration.md Service enum, UI utils, i18n steps
performance.md Pagination, batching, rate limiting
memory.md Memory management, streaming, OOM prevention
lineage.md Lineage extraction methods, dialect mapping, query logs
sql.md SQLAlchemy patterns, URL building, auth, multi-DB

Source-Type Standards

Standard Covers
database.md General database patterns
sql_databases.md MySQL, PostgreSQL, Oracle, MSSQL
data_warehouses.md BigQuery, Snowflake, Redshift, Databricks
nosql_databases.md MongoDB, DynamoDB, Couchbase, Cassandra
dashboard.md Dashboard connectors
pipeline.md Pipeline connectors
messaging.md Messaging connectors
mlmodel.md ML model connectors
storage.md Storage connectors
search.md Search connectors
api.md API connectors

References

Architecture guides and decision trees in connector-building/references/:

Reference Content
architecture-decision-tree.md Service type, connection type, and base class selection
connection-type-guide.md SQLAlchemy vs REST API vs SDK client comparison
capability-mapping.md Capabilities by service type, schema flags, generated files

Review Templates

Template Purpose
full-review-report.md New connector or major refactor review
incremental-review-report.md PR with changes to existing connector
specialized-review-report.md Focused review on one area
pr-review-comment.md Condensed format for GitHub PR comments

Scripts

Script Purpose
gather-connector-context.sh Shell script to collect connector file inventory
analyze_connector.py Python script for structured connector analysis

Architecture

OpenMetadata uses schema-first architecture. One JSON Schema definition cascades through 6 layers:

JSON Schema (single source of truth)
    ├── Python Pydantic models     (make generate)
    ├── Java models                (mvn install)
    ├── TypeScript types           (yarn parse-schema)
    ├── UI config forms            (RJSF auto-renders)
    ├── API request validation     (server uses Java models)
    └── Test fixtures              (tests import Pydantic models)

Quick Start

# 1. Install the plugin
claude plugin install skills/

# 2. Start a new feature
# Just describe what you want — the workflow skill will guide you to /planning

# 3. Implement with TDD
# /tdd "Add new endpoint for data quality scores"

# 4. Verify coverage before PR
# /test-enforcement

# 5. Review your changes
# /code-review

CI

The .github/workflows/lint-standards.yml workflow lints all standards markdown, validates JSON files, and checks symlink integrity on PRs that modify skills/.