OpenMetadata/skills
Pere Miquel Brull f4cb7d0f14
feat(ingestion): add QuestDB database connector (#27604)
* feat(ingestion): add QuestDB database connector

QuestDB speaks the PostgreSQL wire protocol but implements a minimal
pg_catalog, so the default PG dialect queries fail on the CHAR->DOUBLE
cast in pg_class.relkind. This connector routes SQLAlchemy inspection
through information_schema and short-circuits constraint/index lookups
(QuestDB has no PK/FK/unique/indexes), letting CommonDbSourceService
handle the rest of the topology unchanged.

- Fixed /qdb target in the psycopg2 URL regardless of databaseName
  (which remains the OpenMetadata display name)
- get_database_names defaults to 'qdb' instead of 'default'
- 12 unit tests + live-verified against QuestDB 9.3.5 on localhost:8812

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(questdb): address review feedback — rename to QuestDB, wire UI

Code review fixes for PR #27604:

Blockers resolved:
- Rename Questdb -> QuestDB across schema, enum, Python classes, and all
  generated TS files. Matches peer connectors (PinotDB, DynamoDB) and the
  product's actual brand. Changing post-merge would be a breaking migration.
- Remove sslConfig from schema. QuestDB's sslConfig was declared but never
  wired — ssl_manager.check_ssl_and_init is @singledispatch and has no
  QuestDBConnection registration, so enabling SSL in the UI was a silent
  no-op. Can be added in a follow-up with an explicit psycopg2 wiring.

Warnings resolved:
- authType now in schema's required array — was failing with opaque 401.
- Delete dead queries.py (QUESTDB_TEST_GET_TABLES was defined but never
  imported).
- Add bytea -> LargeBinary to the type map (verified via live information_schema
  probe against QuestDB 9.3.5 — all other native types normalize to standard
  PG names that were already mapped).
- Complete type annotations on utils._get_table_names, _get_columns,
  _information_schema_type.
- Dialect patch test now uses a real PGDialect_psycopg2 instance instead of
  a MagicMock dialect, so it catches signature drift against the real
  SQLAlchemy Inspector contract. Added a separate test that verifies
  get_table_names emits a query against information_schema.tables (not
  pg_catalog).
- Add ingestion_logger() to utils.py with a debug log on dialect patching.
- _empty_view_definition now returns None instead of "" to match how other
  dialects signal the absence of a DDL.

Also fixes UI visibility (QuestDB was missing from the service picker):
- Regenerate 15 TS enum files via json2ts.sh -> quicktype so the new
  DatabaseServiceType.QuestDB value flows through the UI.
- Register service-icon-questdb.png in ServiceIconUtils.ts.
- Add locales/en-US/Database/QuestDB.md connector form docs.
- Add quicktype as a devDependency — json2ts.sh needs it and it wasn't
  installed.

Docs: update skills/connector-building and skills/standards/registration
to reflect reality — i18n locale files are not needed, icon + locale MD
registration steps are, and Services.constant.ts is deprecated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* skill

* fix(questdb): restore databaseSchema field for test connection

test_connection_db_schema_sources reads service_connection.databaseSchema
directly with no hasattr guard. Removing it from the schema in the prior
review fix broke GetTables and GetViews steps:

  'QuestDBConnection' object has no attribute 'databaseSchema'

Restored as an optional string with a clearer description (defaults to
public when unset).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix owners

* add yaml

* Update generated TypeScript types

* Sync package.json and yarn.lock with main

* Fix: ingestion files , Added Lineage for questdb tests and UI changes, Refactored code

* FIX: python_checkstyle

* Fix: test and unused param

* Fix: yield_table enforcing tabletype to partition, Refactored lineage

* Fix: Failing test and remove print statement

* FIX: python_checkstyle and added error handling

* FIX: Resolved comments

* FIX: failing tests and schema cleaning

* Minor change

* Fix: Failing unit tests

* Fix: Unit test unrelated changes ignored

* FIX: tests

* Fix: Failing test due to extra parameter in yaml

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Akash Verma <akashverma@Akashs-MacBook-Pro-2.local>
Co-authored-by: Akash Verma <138790903+akashverma0786@users.noreply.github.com>
2026-05-11 13:02:32 +05:30
..
.claude-plugin Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00
.github/workflows Add test-locally skill for local Docker deployment (#26323) 2026-03-08 23:35:11 -07:00
agents chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
code-review chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
commands docs(github): require issue link, design, tests, UI recording in PR template (#27891) 2026-05-07 08:05:56 +02:00
connector-audit chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
connector-building feat(ingestion): add QuestDB database connector (#27604) 2026-05-11 13:02:32 +05:30
connector-review Fivetran improvements (#27270) 2026-04-23 18:05:52 -07:00
connector-standards Register connector skills in .claude/skills (#26320) 2026-03-08 22:01:27 -07:00
hooks Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00
openmetadata-workflow docs(github): require issue link, design, tests, UI recording in PR template (#27891) 2026-05-07 08:05:56 +02:00
planning chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
pr-checklist docs(github): require issue link, design, tests, UI recording in PR template (#27891) 2026-05-07 08:05:56 +02:00
standards feat(ingestion): add QuestDB database connector (#27604) 2026-05-11 13:02:32 +05:30
systematic-debugging Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00
tdd chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
test-enforcement Add developer skills for OpenMetadata (#26836) 2026-03-31 16:15:27 -07:00
test-locally Add test-locally skill for local Docker deployment (#26323) 2026-03-08 23:35:11 -07:00
verification chore(ingestion): drop pylint, expand ruff (#27774) 2026-04-28 07:21:59 +02:00
.markdownlint.yaml Add skills to build connectors (#26309) 2026-03-08 21:45:10 -07:00
README.md docs(github): require issue link, design, tests, UI recording in PR template (#27891) 2026-05-07 08:05:56 +02:00

OpenMetadata Skills Plugin (v2.0.0)

Complete AI development workflow for OpenMetadata. Provides structured planning, test-driven development, 90% test coverage enforcement, code review, systematic debugging, and connector development skills.

Inspired by obra/superpowers and everything-claude-code, tailored for OpenMetadata's multi-language, schema-first architecture.

Installation

Claude Code (Plugin)

# From the OpenMetadata repo root
claude plugin install skills/

This installs all skills, agents, hooks, and commands. The plugin auto-loads the OpenMetadata workflow on session start.

Claude Code (In-Repo, Zero Setup)

If you're working in the OpenMetadata repo, .claude/settings.json provides project-level hooks automatically — no plugin install needed. The .claude/skills/ directory also provides skills directly.

Other Tools

Tool Method
Cursor Settings -> Rules -> Add Rule -> select skills/ directory
Codex Add skills/ to workspace context
GitHub Copilot Reference skills/ in workspace instructions
Windsurf Add skills/ to rules configuration
Any Skills follow the Agent Skills open standard

Workflow Overview

The plugin enforces a structured development workflow:

1. /planning          — Design before code (brainstorm, propose, get approval)
2. /tdd               — Write failing test, implement, refactor
3. /test-enforcement   — Verify 90% coverage, integration tests, Playwright E2E
4. /verification       — Show evidence of passing tests before claiming done
5. /code-review        — Two-stage review (spec compliance + code quality)

The openmetadata-workflow meta-skill is loaded at session start and directs Claude to use the appropriate skills for each task type.

Skills

Development Workflow Skills

Skill Command Purpose
OpenMetadata Workflow (auto-loaded) Meta-skill: routes tasks to the right workflow
Planning /planning Brainstorm approaches, get approval, create step-by-step plan
TDD /tdd RED-GREEN-REFACTOR for Java, Python, and TypeScript
Test Enforcement /test-enforcement Enforce 90% line coverage, integration tests, Playwright E2E
Systematic Debugging /systematic-debugging 4-phase root cause analysis
Code Review /code-review Two-stage review: spec compliance then code quality
Verification /verification Evidence-based completion — show test output, not claims
PR Checklist /pr-checklist Walk the PR template (issue link, design, tests + coverage, UI recording, manual tests) and draft the PR body

Connector Skills

Skill Command Purpose
Connector Building /scaffold-connector Scaffold a new connector with JSON Schema, Python boilerplate, and AI context
Connector Review /connector-review Review connector code against golden standards with multi-agent analysis
Connector Standards /connector-standards Load connector development standards into agent context
Test Locally /test-locally Build and deploy a full local Docker stack to test your connector

Agents

Language-Specific Reviewers

Agent Purpose
java-reviewer Review Java code — Dropwizard, Flyway, JUnit 5, spotless, 90% coverage
python-reviewer Review Python code — Pydantic 2.x, pytest, connector architecture, 90% coverage
frontend-reviewer Review TypeScript/React — core components, tw: prefix, i18n, no MUI, Jest/Playwright

Connector Agents

Agent Purpose
connector-researcher Research source system APIs, SDKs, auth, and data models
connector-validator Validate connector implementation against standards
comment-resolution-checker Verify PR review comments were substantively addressed

Hooks

The plugin includes hooks (hooks/hooks.json) that fire automatically:

Hook Event What it does
OpenMetadata Workflow SessionStart Loads the meta-skill to route tasks to the right workflow
Block --no-verify PreToolUse Prevents skipping pre-commit hooks
Java format reminder PostToolUse Reminds to run mvn spotless:apply after .java edits
Schema regeneration PostToolUse Reminds to run make generate after JSON schema edits

Additional hooks in .claude/settings.json (in-repo, no plugin needed):

  • Block MUI imports
  • Remind yarn parse-schema after connection schema edits
  • Warn about any type in TypeScript

Test Coverage Targets

The /test-enforcement skill enforces these targets:

Layer Target Tool
Java service (openmetadata-service) 90% line coverage JaCoCo
API endpoints 100% of changed endpoints have integration tests openmetadata-integration-tests
React components 90% line coverage Jest
UI features Playwright E2E for all user-facing changes Playwright
Python ingestion 90% line coverage pytest --cov

Standards

12 core standards + 11 source-type standards in standards/:

Core Standards

Standard Content
main.md Architecture overview, schema-first approach, service types
patterns.md Error handling, logging, pagination, auth, filters
testing.md Unit tests, integration tests, pytest patterns
code_style.md Python and JSON Schema conventions
schema.md Connection schema structure, $ref patterns
connection.md BaseConnection vs function patterns
service_spec.md DefaultDatabaseSpec vs BaseSpec
registration.md Service enum, UI utils, i18n steps
performance.md Pagination, batching, rate limiting
memory.md Memory management, streaming, OOM prevention
lineage.md Lineage extraction methods, dialect mapping, query logs
sql.md SQLAlchemy patterns, URL building, auth, multi-DB

Source-Type Standards

Standard Covers
database.md General database patterns
sql_databases.md MySQL, PostgreSQL, Oracle, MSSQL
data_warehouses.md BigQuery, Snowflake, Redshift, Databricks
nosql_databases.md MongoDB, DynamoDB, Couchbase, Cassandra
dashboard.md Dashboard connectors
pipeline.md Pipeline connectors
messaging.md Messaging connectors
mlmodel.md ML model connectors
storage.md Storage connectors
search.md Search connectors
api.md API connectors

References

Architecture guides and decision trees in connector-building/references/:

Reference Content
architecture-decision-tree.md Service type, connection type, and base class selection
connection-type-guide.md SQLAlchemy vs REST API vs SDK client comparison
capability-mapping.md Capabilities by service type, schema flags, generated files

Review Templates

Template Purpose
full-review-report.md New connector or major refactor review
incremental-review-report.md PR with changes to existing connector
specialized-review-report.md Focused review on one area
pr-review-comment.md Condensed format for GitHub PR comments

Scripts

Script Purpose
gather-connector-context.sh Shell script to collect connector file inventory
analyze_connector.py Python script for structured connector analysis

Architecture

OpenMetadata uses schema-first architecture. One JSON Schema definition cascades through 6 layers:

JSON Schema (single source of truth)
    ├── Python Pydantic models     (make generate)
    ├── Java models                (mvn install)
    ├── TypeScript types           (yarn parse-schema)
    ├── UI config forms            (RJSF auto-renders)
    ├── API request validation     (server uses Java models)
    └── Test fixtures              (tests import Pydantic models)

Quick Start

# 1. Install the plugin
claude plugin install skills/

# 2. Start a new feature
# Just describe what you want — the workflow skill will guide you to /planning

# 3. Implement with TDD
# /tdd "Add new endpoint for data quality scores"

# 4. Verify coverage before PR
# /test-enforcement

# 5. Review your changes
# /code-review

CI

The .github/workflows/lint-standards.yml workflow lints all standards markdown, validates JSON files, and checks symlink integrity on PRs that modify skills/.