* feat(ingestion): add QuestDB database connector QuestDB speaks the PostgreSQL wire protocol but implements a minimal pg_catalog, so the default PG dialect queries fail on the CHAR->DOUBLE cast in pg_class.relkind. This connector routes SQLAlchemy inspection through information_schema and short-circuits constraint/index lookups (QuestDB has no PK/FK/unique/indexes), letting CommonDbSourceService handle the rest of the topology unchanged. - Fixed /qdb target in the psycopg2 URL regardless of databaseName (which remains the OpenMetadata display name) - get_database_names defaults to 'qdb' instead of 'default' - 12 unit tests + live-verified against QuestDB 9.3.5 on localhost:8812 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(questdb): address review feedback — rename to QuestDB, wire UI Code review fixes for PR #27604: Blockers resolved: - Rename Questdb -> QuestDB across schema, enum, Python classes, and all generated TS files. Matches peer connectors (PinotDB, DynamoDB) and the product's actual brand. Changing post-merge would be a breaking migration. - Remove sslConfig from schema. QuestDB's sslConfig was declared but never wired — ssl_manager.check_ssl_and_init is @singledispatch and has no QuestDBConnection registration, so enabling SSL in the UI was a silent no-op. Can be added in a follow-up with an explicit psycopg2 wiring. Warnings resolved: - authType now in schema's required array — was failing with opaque 401. - Delete dead queries.py (QUESTDB_TEST_GET_TABLES was defined but never imported). - Add bytea -> LargeBinary to the type map (verified via live information_schema probe against QuestDB 9.3.5 — all other native types normalize to standard PG names that were already mapped). - Complete type annotations on utils._get_table_names, _get_columns, _information_schema_type. - Dialect patch test now uses a real PGDialect_psycopg2 instance instead of a MagicMock dialect, so it catches signature drift against the real SQLAlchemy Inspector contract. Added a separate test that verifies get_table_names emits a query against information_schema.tables (not pg_catalog). - Add ingestion_logger() to utils.py with a debug log on dialect patching. - _empty_view_definition now returns None instead of "" to match how other dialects signal the absence of a DDL. Also fixes UI visibility (QuestDB was missing from the service picker): - Regenerate 15 TS enum files via json2ts.sh -> quicktype so the new DatabaseServiceType.QuestDB value flows through the UI. - Register service-icon-questdb.png in ServiceIconUtils.ts. - Add locales/en-US/Database/QuestDB.md connector form docs. - Add quicktype as a devDependency — json2ts.sh needs it and it wasn't installed. Docs: update skills/connector-building and skills/standards/registration to reflect reality — i18n locale files are not needed, icon + locale MD registration steps are, and Services.constant.ts is deprecated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * skill * fix(questdb): restore databaseSchema field for test connection test_connection_db_schema_sources reads service_connection.databaseSchema directly with no hasattr guard. Removing it from the schema in the prior review fix broke GetTables and GetViews steps: 'QuestDBConnection' object has no attribute 'databaseSchema' Restored as an optional string with a clearer description (defaults to public when unset). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix owners * add yaml * Update generated TypeScript types * Sync package.json and yarn.lock with main * Fix: ingestion files , Added Lineage for questdb tests and UI changes, Refactored code * FIX: python_checkstyle * Fix: test and unused param * Fix: yield_table enforcing tabletype to partition, Refactored lineage * Fix: Failing test and remove print statement * FIX: python_checkstyle and added error handling * FIX: Resolved comments * FIX: failing tests and schema cleaning * Minor change * Fix: Failing unit tests * Fix: Unit test unrelated changes ignored * FIX: tests * Fix: Failing test due to extra parameter in yaml --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local> Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| .claude-plugin | ||
| .github/workflows | ||
| agents | ||
| code-review | ||
| commands | ||
| connector-audit | ||
| connector-building | ||
| connector-review | ||
| connector-standards | ||
| hooks | ||
| openmetadata-workflow | ||
| planning | ||
| pr-checklist | ||
| standards | ||
| systematic-debugging | ||
| tdd | ||
| test-enforcement | ||
| test-locally | ||
| verification | ||
| .markdownlint.yaml | ||
| README.md | ||
OpenMetadata Skills Plugin (v2.0.0)
Complete AI development workflow for OpenMetadata. Provides structured planning, test-driven development, 90% test coverage enforcement, code review, systematic debugging, and connector development skills.
Inspired by obra/superpowers and everything-claude-code, tailored for OpenMetadata's multi-language, schema-first architecture.
Installation
Claude Code (Plugin)
# From the OpenMetadata repo root
claude plugin install skills/
This installs all skills, agents, hooks, and commands. The plugin auto-loads the OpenMetadata workflow on session start.
Claude Code (In-Repo, Zero Setup)
If you're working in the OpenMetadata repo, .claude/settings.json provides project-level hooks automatically — no plugin install needed. The .claude/skills/ directory also provides skills directly.
Other Tools
| Tool | Method |
|---|---|
| Cursor | Settings -> Rules -> Add Rule -> select skills/ directory |
| Codex | Add skills/ to workspace context |
| GitHub Copilot | Reference skills/ in workspace instructions |
| Windsurf | Add skills/ to rules configuration |
| Any | Skills follow the Agent Skills open standard |
Workflow Overview
The plugin enforces a structured development workflow:
1. /planning — Design before code (brainstorm, propose, get approval)
2. /tdd — Write failing test, implement, refactor
3. /test-enforcement — Verify 90% coverage, integration tests, Playwright E2E
4. /verification — Show evidence of passing tests before claiming done
5. /code-review — Two-stage review (spec compliance + code quality)
The openmetadata-workflow meta-skill is loaded at session start and directs Claude to use the appropriate skills for each task type.
Skills
Development Workflow Skills
| Skill | Command | Purpose |
|---|---|---|
| OpenMetadata Workflow | (auto-loaded) | Meta-skill: routes tasks to the right workflow |
| Planning | /planning |
Brainstorm approaches, get approval, create step-by-step plan |
| TDD | /tdd |
RED-GREEN-REFACTOR for Java, Python, and TypeScript |
| Test Enforcement | /test-enforcement |
Enforce 90% line coverage, integration tests, Playwright E2E |
| Systematic Debugging | /systematic-debugging |
4-phase root cause analysis |
| Code Review | /code-review |
Two-stage review: spec compliance then code quality |
| Verification | /verification |
Evidence-based completion — show test output, not claims |
| PR Checklist | /pr-checklist |
Walk the PR template (issue link, design, tests + coverage, UI recording, manual tests) and draft the PR body |
Connector Skills
| Skill | Command | Purpose |
|---|---|---|
| Connector Building | /scaffold-connector |
Scaffold a new connector with JSON Schema, Python boilerplate, and AI context |
| Connector Review | /connector-review |
Review connector code against golden standards with multi-agent analysis |
| Connector Standards | /connector-standards |
Load connector development standards into agent context |
| Test Locally | /test-locally |
Build and deploy a full local Docker stack to test your connector |
Agents
Language-Specific Reviewers
| Agent | Purpose |
|---|---|
| java-reviewer | Review Java code — Dropwizard, Flyway, JUnit 5, spotless, 90% coverage |
| python-reviewer | Review Python code — Pydantic 2.x, pytest, connector architecture, 90% coverage |
| frontend-reviewer | Review TypeScript/React — core components, tw: prefix, i18n, no MUI, Jest/Playwright |
Connector Agents
| Agent | Purpose |
|---|---|
| connector-researcher | Research source system APIs, SDKs, auth, and data models |
| connector-validator | Validate connector implementation against standards |
| comment-resolution-checker | Verify PR review comments were substantively addressed |
Hooks
The plugin includes hooks (hooks/hooks.json) that fire automatically:
| Hook | Event | What it does |
|---|---|---|
| OpenMetadata Workflow | SessionStart | Loads the meta-skill to route tasks to the right workflow |
Block --no-verify |
PreToolUse | Prevents skipping pre-commit hooks |
| Java format reminder | PostToolUse | Reminds to run mvn spotless:apply after .java edits |
| Schema regeneration | PostToolUse | Reminds to run make generate after JSON schema edits |
Additional hooks in .claude/settings.json (in-repo, no plugin needed):
- Block MUI imports
- Remind
yarn parse-schemaafter connection schema edits - Warn about
anytype in TypeScript
Test Coverage Targets
The /test-enforcement skill enforces these targets:
| Layer | Target | Tool |
|---|---|---|
Java service (openmetadata-service) |
90% line coverage | JaCoCo |
| API endpoints | 100% of changed endpoints have integration tests | openmetadata-integration-tests |
| React components | 90% line coverage | Jest |
| UI features | Playwright E2E for all user-facing changes | Playwright |
| Python ingestion | 90% line coverage | pytest --cov |
Standards
12 core standards + 11 source-type standards in standards/:
Core Standards
| Standard | Content |
|---|---|
| main.md | Architecture overview, schema-first approach, service types |
| patterns.md | Error handling, logging, pagination, auth, filters |
| testing.md | Unit tests, integration tests, pytest patterns |
| code_style.md | Python and JSON Schema conventions |
| schema.md | Connection schema structure, $ref patterns |
| connection.md | BaseConnection vs function patterns |
| service_spec.md | DefaultDatabaseSpec vs BaseSpec |
| registration.md | Service enum, UI utils, i18n steps |
| performance.md | Pagination, batching, rate limiting |
| memory.md | Memory management, streaming, OOM prevention |
| lineage.md | Lineage extraction methods, dialect mapping, query logs |
| sql.md | SQLAlchemy patterns, URL building, auth, multi-DB |
Source-Type Standards
| Standard | Covers |
|---|---|
| database.md | General database patterns |
| sql_databases.md | MySQL, PostgreSQL, Oracle, MSSQL |
| data_warehouses.md | BigQuery, Snowflake, Redshift, Databricks |
| nosql_databases.md | MongoDB, DynamoDB, Couchbase, Cassandra |
| dashboard.md | Dashboard connectors |
| pipeline.md | Pipeline connectors |
| messaging.md | Messaging connectors |
| mlmodel.md | ML model connectors |
| storage.md | Storage connectors |
| search.md | Search connectors |
| api.md | API connectors |
References
Architecture guides and decision trees in connector-building/references/:
| Reference | Content |
|---|---|
| architecture-decision-tree.md | Service type, connection type, and base class selection |
| connection-type-guide.md | SQLAlchemy vs REST API vs SDK client comparison |
| capability-mapping.md | Capabilities by service type, schema flags, generated files |
Review Templates
| Template | Purpose |
|---|---|
| full-review-report.md | New connector or major refactor review |
| incremental-review-report.md | PR with changes to existing connector |
| specialized-review-report.md | Focused review on one area |
| pr-review-comment.md | Condensed format for GitHub PR comments |
Scripts
| Script | Purpose |
|---|---|
| gather-connector-context.sh | Shell script to collect connector file inventory |
| analyze_connector.py | Python script for structured connector analysis |
Architecture
OpenMetadata uses schema-first architecture. One JSON Schema definition cascades through 6 layers:
JSON Schema (single source of truth)
├── Python Pydantic models (make generate)
├── Java models (mvn install)
├── TypeScript types (yarn parse-schema)
├── UI config forms (RJSF auto-renders)
├── API request validation (server uses Java models)
└── Test fixtures (tests import Pydantic models)
Quick Start
# 1. Install the plugin
claude plugin install skills/
# 2. Start a new feature
# Just describe what you want — the workflow skill will guide you to /planning
# 3. Implement with TDD
# /tdd "Add new endpoint for data quality scores"
# 4. Verify coverage before PR
# /test-enforcement
# 5. Review your changes
# /code-review
CI
The .github/workflows/lint-standards.yml workflow lints all standards markdown, validates JSON files, and checks symlink integrity on PRs that modify skills/.