Commit graph

3 commits

Author SHA1 Message Date
Andre Manoel
982ce79ca9
feat: add processor plugin support (#299)
* feat: add processor plugin support

Add PluginType.PROCESSOR to the plugin system, enabling third-party
processor plugins via entry points. Includes a demo plugin package
with RegexFilterProcessor (process_before_batch) and
SemanticDedupProcessor (process_after_generation).

- Add PluginType.PROCESSOR with processor_type discriminator
- Create processor_types.py for ProcessorConfigT with plugin injection
- Register plugin processors in engine ProcessorRegistry
- Use RLock in PluginRegistry to prevent deadlocks during discovery
- Add demo package: data-designer-demo-processors
- Update processor and plugin documentation

* test: add processor plugin registration test

Verify that processor plugins from PluginRegistry are picked up
by create_default_processor_registry and registered correctly.

* test: simplify processor plugin registration test

* move ProcessorConfig to base and convert demo to e2e test

- Move ProcessorConfig from processors.py to config.base to guard
  against circular deps (alongside SingleColumnConfig)
- Delete demo/ directory with regex_filter and semantic_dedup plugins
- Add regex_filter as an e2e processor plugin test in tests_e2e/

* move plan to plans/299/
2026-02-25 16:40:01 -03:00
Johnny Greco
11143c788f
docs: restructure plugin docs with multi-file layout and seed reader type (#302)
* docs: restructure plugin docs with multi-file layout and seed reader type

- Update plugin overview to document both column generator and seed
  reader plugin types
- Restructure example plugin to use separate config.py, impl.py, and
  plugin.py files instead of a single-file approach
- Add sections for plugin validation and multiple plugins per package
- Document required config class methods (get_column_emoji,
  required_columns, side_effect_columns)

* docs: clarify benefits of multi-file plugin structure

Expand explanation to mention circular dependency prevention
as a key reason for separating config, impl, and plugin modules.

* docs: fix import ordering in plugin example

* import spacing

* better example column name

* add a bit to the comment

* Updated plugin docs

* update plugin overview call-to-action wording

---------

Co-authored-by: Kirit93 <kthadaka@nvidia.com>
2026-02-09 16:03:56 -05:00
Johnny Greco
48fdc8c838
docs: add initial plugin documentation (#107)
* add docstrings

* add analysis modules

* include toc for plugins section

* add plugin docs

* remove scope creep

* Update docs/plugins/example.md

Co-authored-by: Nabin Mulepati <nmulepati@nvidia.com>

* address feedback

---------

Co-authored-by: Nabin Mulepati <nmulepati@nvidia.com>
2025-12-11 16:05:11 -05:00