* chore(ingestion): replace black/isort/pycln with ruff
- Swap formatter + import-sorter + unused-import tooling for ruff
(line-length 120, target py3.10) in ingestion + openmetadata-airflow-apis
- Drop dead [tool.mypy] config; basedpyright is the active type checker
- Bump requires-python to >=3.10 to match noxfile and CLAUDE.md (3.9 is
documented as broken on Mac in noxfile.py)
- Bump pre-commit-hooks v2.3 -> v5.0; the new check-json catches four
pre-existing JSON issues now excluded with an inline TODO
- Update Makefile py_format / py_format_check targets to call ruff
* chore(ingestion): grandfather ruff lint violations and apply ruff format
- 253 noqa markers added via 'ruff check --add-noqa' across 128 files,
freezing existing violations so this PR is a tooling-only swap. Per-rule
cleanup tracked in the TODO comment in ingestion/pyproject.toml.
- Bulk reformat from black 22.3 -> ruff format @ line-length 120.
Cosmetic only: imports balanced (-32/+32), structural keywords balanced
(-2221/+2221), no logic changes.
- Star-import rules (F403/F405) globally ignored; refactoring wildcard
imports across connectors is a separate effort.
* chore(ingestion): fix pylint findings surfaced after ruff format
- filters.py: drop redundant parens around re.match(...) in `if`
(C0325 superfluous-parens) — exposed when ruff format unwrapped them
- nosql_adaptor.py: move `# pylint: disable=unused-argument` from the
`column:` line to the `def` line so it covers `table` too (W0613) —
scope was line-based, lost when ruff split params onto multiple lines
- action1xx.py: replace `arguments-differ` with `signature-differs` in
the disable directive (was always wrong code) and drop the now-useless
`unused-argument` suppression (I0021)
* fix(ingestion): make ruff extend-exclude robust to multi-root invocations
CI's `make py_format_check` runs from the repo root and passes both
`ingestion/` and `./openmetadata-airflow-apis/` to ruff in a single
invocation. With multiple root paths, ruff's parallel file discovery
races on extend-exclude matching against the project root, so files
under `ingestion/src/metadata/generated/` were intermittently scanned
and produced ~830 I001 violations.
20-run repro: 10/20 fail without the fix, 20/20 pass with the fix.
Each excluded directory now appears twice in extend-exclude:
- the project-root-relative pattern (cwd = ingestion/)
- the prefixed pattern (cwd = repo root, multi-root invocation)
* chore(ingestion): address gitar-bot findings + cross-version pylint disable
- openmetadata-airflow-apis/pyproject.toml: switch coverage to module-name
source + [tool.coverage.paths] glob remap (matches the ingestion pattern).
Drops the hardcoded `env/lib/python3.9/site-packages/...` source path,
which broke after the requires-python bump to 3.10. (Finding 1)
- ingestion/setup.py: remove dead python_version<'3.9' / >='3.9' guards on
mysql-connector-python and testcontainers; promote locust to a regular
test dep (was conditionally added under sys.version_info >= (3, 9)). Also
remove the now-unused `import sys`. (Finding 3)
- ingestion/src/metadata/great_expectations/action1xx.py: cover both
arguments-differ (great_expectations 0.18.x parent) and signature-differs
(great_expectations 1.x parent) in the pylint disable comment, since
CI installs 0.18.x and local often has 1.x. unused-argument covers the
unused action_context. The opposite rule fires as I0021 useless-suppression
on each environment, which is informational and does not affect pylint's
exit code.
* tests: refactor
refactor tests and consolidate common functionality in integrations.conftest
this enables writing tests more concisely.
demonstrated with postgres and mssql.
will migrate more
* format
* removed helpers
* changed scope of fictures
* changed scope of fixtures
* added profiler test for mssql
* fixed import in data_quality test
* json safe serialization
* format
* set MARS_Connection
* use SerializableTableData instead of TableData
* deleted file test_postgres.py
* fixed tests
* added more test cases
* format
* changed name test_models.py
* removed the logic for serializing table data
* wip
* changed mapping in common type map
* changed mapping in common type map
* reverted TableData imports
* reverted TableData imports
* reverted TableData imports
