DataDesigner/tests/engine/analysis
Nabin Mulepati 8370e4a00b
feat: support native embedding generation (#106)
* Add generation type to ModelConfig

* pass tests

* added generate_text_embeddings

* tests

* remove sensitive=True old artifact no longer needed

* Slight refactor

* slight refactor

* Added embedding generator

* chunk_separator -> chunk_pattern

* update tests

* rename for consistency

* Restructure InferenceParameters -> CompletionInferenceParameters, BaseInferenceParameters, EmbeddingInferenceParameters

* Remove purpose from consolidated kwargs

* WithModelConfiguration.inference_parameters should should be typed with BaseInferenceParameters

* Type as WithModelGeneration

* Add image generation modality

* update return type for generate_kwargs

* make generation_type a field of ModelConfig as opposed to a prop resolved based on the type of InferenceParameters

* remove regex based chunking from embedding generator

* Remove image generation for now

* more tests and updates

* column_type_is_llm_generated -> column_type_is_model_generated

* change set to list: fix flaky tests

* CompletionInferenceParameters -> ChatCompletionInferenceParameters for consistency with generation_type

* Update docs

* fix deprecation warning originating from cli model settings

* update display of inference parameters in cli list

* save prog on inference parameter

* updates for the ocnfig builder

* update cli readme

* update cli for inference parmeters

* update inference parameter names

* flip order of vars

* WithCompletion -> WithChatCompletion

* specify InferenceParamsT

* Update columns.md with EmbeddingColumnConfig info

* make generation_type a descriminator field in inference params. add configuration support for max_parallel_requests and timeout

* DRY out some stuff in field.py

* Update nomenclature. prompt tokens -> input tokens, completion tokens -> output tokens in column statistics for consistency

* Add nvidia-embedding and openai-embedding to default model configs

* Fix typo in docs

* Make generate collab notebooks

* fine-tune -> adjust
2025-12-15 11:03:33 -07:00
..
column_profilers fix: analysis report when there is a column with mixed data types (#131) 2025-12-15 10:36:34 -05:00
test_data/artifacts/dataset chore: split person samplers and use parameters in sql exec (#48) 2025-11-19 10:30:15 -05:00
utils feat: support native embedding generation (#106) 2025-12-15 11:03:33 -07:00
conftest.py don't lowercase score names when using it to dynamically create pydantic objects (#122) 2025-12-11 13:52:49 -07:00
test_column_statistics_calculator.py feat: support native embedding generation (#106) 2025-12-15 11:03:33 -07:00
test_dataset_profiler.py feat: 🔌 Initial plugin system implementation (#23) 2025-11-11 15:36:52 -05:00
test_errors.py initial port 2025-10-27 14:29:12 -04:00