* chore(ingestion): drop pylint, expand ruff to Stage 2c
Replace pylint with a coherent ruff-only stack (Stage 2c of the modernize
roadmap). Pylint is dropped from dev deps and CI workflows; ruff selected
ruleset expanded to ~22 families covering style, bug catchers, hygiene,
and the pylint port (PLE/PLC/PLW/PLR with the noisy "too-many-X"
complexity caps + magic-value disabled).
What's selected (with rationale in pyproject.toml):
E, W, F, I, N — style + correctness baseline + naming
UP — pyupgrade (py>=3.10 modernizations)
B, C4, C90, RET, SIM, TRY — bug catchers
PIE, ICN, T20, TC, TID, PTH, PERF — hygiene
PLE, PLC, PLW, PLR — pylint port (PLR complexity caps ignored)
RUF — ruff-native (incl. RUF100 unused-noqa)
What's removed:
- .pylintrc (root) — duplicate of the ingestion pylint config
- [tool.pylint.*] block in ingestion/pyproject.toml (~140 lines)
- ingestion/plugins/{print_checker,import_checker}.py + tests + README
(replaced by built-in T20 + TID251 banned-api respectively)
- pylint dep from ingestion/setup.py and openmetadata-airflow-apis/pyproject.toml
- `make lint` Makefile target + the pylint invocation in py_format_check
- dead pylint TODO comment + ignored test entry in noxfile.py
Cwd-stable config: ruff is invoked both from the repo root (pre-commit,
CI) and from ingestion/ (`make py_format_check`). The `src`,
`extend-exclude`, and per-file-ignores entries are listed twice — once
relative to ingestion/ and once with the `ingestion/` prefix — so
first-party isort detection and exclusions match in both invocations.
Grandfathering: ran `ruff check --add-noqa` once + format-stable
iteration. ~12,130 noqa directives across ~1,400 files. Cleanup is
deferred to follow-up PRs that drop noqas one rule at a time.
Documentation sweep: replaced `make lint` references in CLAUDE.md,
AGENTS.md, DEVELOPER.md, copilot-instructions, and 6 SKILL files with
the apply+verify shape `make py_format && make py_format_check`.
`make py_format` is NOT a strict superset of pylint — it only applies
auto-fixable violations; `make py_format_check` catches the rest.
Basedpyright baseline regenerated: ruff format reflowed multi-line
signatures in ~70 files, shifting type-error column positions. The
basedpyright baseline matches by (file path, error code, range), so
column shifts caused 19 entries to mis-align. Net diff is small
(154 lines in/out of the 13MB baseline.json) — purely positional.
Verified locally:
- make py_format_check → All checks passed
- nox --no-venv -s static-checks → 0 errors, 0 warnings, 0 notes
* chore(ingestion): finish ruff swap — nox lint session + skill docs
Three remaining stale-tooling references after Stage 2c:
- `ingestion/noxfile.py` `lint` session was still calling `black --check`,
`isort --check-only`, `pycln --diff`. Those tools aren't installed
anywhere (we dropped them from dev deps). Replace with the ruff
equivalents that mirror `make py_format_check`.
- `skills/standards/code_style.md`: stack listed as `black + isort +
pycln`; line length claimed 88 (black default). Both wrong: stack is
ruff, line length is 120.
- `skills/connector-building/SKILL.md`: `make py_format` comment said
`# black + isort + pycln`. Same swap.
* chore(ingestion): keep main's baseline + globally ignore TRY400
Per gitar-bot's review on PR #27774:
1. Main's PR #27728 promoted ~60 `logger.warning()` → `logger.error()`
inside `except` blocks. Those changes landed on main with their own
baseline updates. Our PR doesn't promote anything — the merge from
origin/main brought those `error` calls along with their baseline
entries.
The bot interpreted the `# noqa: TRY400` we added next to those lines
as us silencing the rule case-by-case. Cleaner: globally ignore
TRY400 in pyproject.toml, with a comment explaining why the codebase's
`logger.error(...)` + separate `logger.debug(traceback.format_exc())`
pattern is intentional. Strip ~430 per-line `# noqa: TRY400` markers
from source.
2. Document that `S101` in `per-file-ignores` is a forward-looking
entry — flake8-bandit (`S`) is not yet selected, so the rule is
no-op today; the entry stays so when `S` lands later, tests don't
immediately error.
Reverts the platform pin and Linux Docker–generated baseline. Keep
main's baseline intact and let CI surface the exact column-shifted
entries; the team will decide whether to fix in-place (revert format
on affected files) or add per-line `# pyright: ignore` markers.
* chore(ingestion): regen baseline for new connector type debt
Main's baseline was stale relative to recently-added connectors
(McpConnection, CustomDriveConnection) that lack common attributes
like `hostPort`, `database`, `catalog` etc. — all sites that access
those attributes via the union-typed `serviceConnection.root.config`
fire `reportAttributeAccessIssue` errors that aren't baselined.
71 errors + 58 warnings absorbed. Local macOS regen; pushing to see
CI's drift count. Per the basedpyright-baseline-and-ci PR experience,
macOS↔Linux column drift on this size of regen has historically been
1-7 residuals.
|
||
|---|---|---|
| .. | ||
| .claude-plugin | ||
| .github/workflows | ||
| agents | ||
| code-review | ||
| commands | ||
| connector-audit | ||
| connector-building | ||
| connector-review | ||
| connector-standards | ||
| hooks | ||
| openmetadata-workflow | ||
| planning | ||
| standards | ||
| systematic-debugging | ||
| tdd | ||
| test-enforcement | ||
| test-locally | ||
| verification | ||
| .markdownlint.yaml | ||
| README.md | ||
OpenMetadata Skills Plugin (v2.0.0)
Complete AI development workflow for OpenMetadata. Provides structured planning, test-driven development, 90% test coverage enforcement, code review, systematic debugging, and connector development skills.
Inspired by obra/superpowers and everything-claude-code, tailored for OpenMetadata's multi-language, schema-first architecture.
Installation
Claude Code (Plugin)
# From the OpenMetadata repo root
claude plugin install skills/
This installs all skills, agents, hooks, and commands. The plugin auto-loads the OpenMetadata workflow on session start.
Claude Code (In-Repo, Zero Setup)
If you're working in the OpenMetadata repo, .claude/settings.json provides project-level hooks automatically — no plugin install needed. The .claude/skills/ directory also provides skills directly.
Other Tools
| Tool | Method |
|---|---|
| Cursor | Settings -> Rules -> Add Rule -> select skills/ directory |
| Codex | Add skills/ to workspace context |
| GitHub Copilot | Reference skills/ in workspace instructions |
| Windsurf | Add skills/ to rules configuration |
| Any | Skills follow the Agent Skills open standard |
Workflow Overview
The plugin enforces a structured development workflow:
1. /planning — Design before code (brainstorm, propose, get approval)
2. /tdd — Write failing test, implement, refactor
3. /test-enforcement — Verify 90% coverage, integration tests, Playwright E2E
4. /verification — Show evidence of passing tests before claiming done
5. /code-review — Two-stage review (spec compliance + code quality)
The openmetadata-workflow meta-skill is loaded at session start and directs Claude to use the appropriate skills for each task type.
Skills
Development Workflow Skills
| Skill | Command | Purpose |
|---|---|---|
| OpenMetadata Workflow | (auto-loaded) | Meta-skill: routes tasks to the right workflow |
| Planning | /planning |
Brainstorm approaches, get approval, create step-by-step plan |
| TDD | /tdd |
RED-GREEN-REFACTOR for Java, Python, and TypeScript |
| Test Enforcement | /test-enforcement |
Enforce 90% line coverage, integration tests, Playwright E2E |
| Systematic Debugging | /systematic-debugging |
4-phase root cause analysis |
| Code Review | /code-review |
Two-stage review: spec compliance then code quality |
| Verification | /verification |
Evidence-based completion — show test output, not claims |
Connector Skills
| Skill | Command | Purpose |
|---|---|---|
| Connector Building | /scaffold-connector |
Scaffold a new connector with JSON Schema, Python boilerplate, and AI context |
| Connector Review | /connector-review |
Review connector code against golden standards with multi-agent analysis |
| Connector Standards | /connector-standards |
Load connector development standards into agent context |
| Test Locally | /test-locally |
Build and deploy a full local Docker stack to test your connector |
Agents
Language-Specific Reviewers
| Agent | Purpose |
|---|---|
| java-reviewer | Review Java code — Dropwizard, Flyway, JUnit 5, spotless, 90% coverage |
| python-reviewer | Review Python code — Pydantic 2.x, pytest, connector architecture, 90% coverage |
| frontend-reviewer | Review TypeScript/React — core components, tw: prefix, i18n, no MUI, Jest/Playwright |
Connector Agents
| Agent | Purpose |
|---|---|
| connector-researcher | Research source system APIs, SDKs, auth, and data models |
| connector-validator | Validate connector implementation against standards |
| comment-resolution-checker | Verify PR review comments were substantively addressed |
Hooks
The plugin includes hooks (hooks/hooks.json) that fire automatically:
| Hook | Event | What it does |
|---|---|---|
| OpenMetadata Workflow | SessionStart | Loads the meta-skill to route tasks to the right workflow |
Block --no-verify |
PreToolUse | Prevents skipping pre-commit hooks |
| Java format reminder | PostToolUse | Reminds to run mvn spotless:apply after .java edits |
| Schema regeneration | PostToolUse | Reminds to run make generate after JSON schema edits |
Additional hooks in .claude/settings.json (in-repo, no plugin needed):
- Block MUI imports
- Remind
yarn parse-schemaafter connection schema edits - Warn about
anytype in TypeScript
Test Coverage Targets
The /test-enforcement skill enforces these targets:
| Layer | Target | Tool |
|---|---|---|
Java service (openmetadata-service) |
90% line coverage | JaCoCo |
| API endpoints | 100% of changed endpoints have integration tests | openmetadata-integration-tests |
| React components | 90% line coverage | Jest |
| UI features | Playwright E2E for all user-facing changes | Playwright |
| Python ingestion | 90% line coverage | pytest --cov |
Standards
12 core standards + 11 source-type standards in standards/:
Core Standards
| Standard | Content |
|---|---|
| main.md | Architecture overview, schema-first approach, service types |
| patterns.md | Error handling, logging, pagination, auth, filters |
| testing.md | Unit tests, integration tests, pytest patterns |
| code_style.md | Python and JSON Schema conventions |
| schema.md | Connection schema structure, $ref patterns |
| connection.md | BaseConnection vs function patterns |
| service_spec.md | DefaultDatabaseSpec vs BaseSpec |
| registration.md | Service enum, UI utils, i18n steps |
| performance.md | Pagination, batching, rate limiting |
| memory.md | Memory management, streaming, OOM prevention |
| lineage.md | Lineage extraction methods, dialect mapping, query logs |
| sql.md | SQLAlchemy patterns, URL building, auth, multi-DB |
Source-Type Standards
| Standard | Covers |
|---|---|
| database.md | General database patterns |
| sql_databases.md | MySQL, PostgreSQL, Oracle, MSSQL |
| data_warehouses.md | BigQuery, Snowflake, Redshift, Databricks |
| nosql_databases.md | MongoDB, DynamoDB, Couchbase, Cassandra |
| dashboard.md | Dashboard connectors |
| pipeline.md | Pipeline connectors |
| messaging.md | Messaging connectors |
| mlmodel.md | ML model connectors |
| storage.md | Storage connectors |
| search.md | Search connectors |
| api.md | API connectors |
References
Architecture guides and decision trees in connector-building/references/:
| Reference | Content |
|---|---|
| architecture-decision-tree.md | Service type, connection type, and base class selection |
| connection-type-guide.md | SQLAlchemy vs REST API vs SDK client comparison |
| capability-mapping.md | Capabilities by service type, schema flags, generated files |
Review Templates
| Template | Purpose |
|---|---|
| full-review-report.md | New connector or major refactor review |
| incremental-review-report.md | PR with changes to existing connector |
| specialized-review-report.md | Focused review on one area |
| pr-review-comment.md | Condensed format for GitHub PR comments |
Scripts
| Script | Purpose |
|---|---|
| gather-connector-context.sh | Shell script to collect connector file inventory |
| analyze_connector.py | Python script for structured connector analysis |
Architecture
OpenMetadata uses schema-first architecture. One JSON Schema definition cascades through 6 layers:
JSON Schema (single source of truth)
├── Python Pydantic models (make generate)
├── Java models (mvn install)
├── TypeScript types (yarn parse-schema)
├── UI config forms (RJSF auto-renders)
├── API request validation (server uses Java models)
└── Test fixtures (tests import Pydantic models)
Quick Start
# 1. Install the plugin
claude plugin install skills/
# 2. Start a new feature
# Just describe what you want — the workflow skill will guide you to /planning
# 3. Implement with TDD
# /tdd "Add new endpoint for data quality scores"
# 4. Verify coverage before PR
# /test-enforcement
# 5. Review your changes
# /code-review
CI
The .github/workflows/lint-standards.yml workflow lints all standards markdown, validates JSON files, and checks symlink integrity on PRs that modify skills/.