Commit graph

9 commits

Author SHA1 Message Date
Johnny Greco
f8c201e085
chore: update header script to check for diffs (#195)
* update script

* update headers

* refactor a bit and add test script

* update headers

* update for edge case

* update headers

* add step to get file creation date

* use git history to get copyright year

* generation type is printed with inference parameters

* fix unit test
2026-01-09 17:10:58 -05:00
Johnny Greco
69cd989285
refactor: update required resources treatment and use subclasses over mixins (#184)
* removing required resources

* fix tests

* add get required resources method to base column generator

* move classification functions to engine; remove required resources

* drop single from subclass names

* update model config logging

* fix unit test

* typo

* update type hint

* move tests
2026-01-09 14:42:09 -05:00
Andre Manoel
ca1a7b26e8
fix: Decimal in structured generation leads to errors (#171)
* fix: Decimals in structured generation lead to errors

* lint

* committed something by mistake

* removing comment

* convert to Decimal with proper precision instead
2026-01-07 18:23:09 -03:00
Nabin Mulepati
8370e4a00b
feat: support native embedding generation (#106)
* Add generation type to ModelConfig

* pass tests

* added generate_text_embeddings

* tests

* remove sensitive=True old artifact no longer needed

* Slight refactor

* slight refactor

* Added embedding generator

* chunk_separator -> chunk_pattern

* update tests

* rename for consistency

* Restructure InferenceParameters -> CompletionInferenceParameters, BaseInferenceParameters, EmbeddingInferenceParameters

* Remove purpose from consolidated kwargs

* WithModelConfiguration.inference_parameters should should be typed with BaseInferenceParameters

* Type as WithModelGeneration

* Add image generation modality

* update return type for generate_kwargs

* make generation_type a field of ModelConfig as opposed to a prop resolved based on the type of InferenceParameters

* remove regex based chunking from embedding generator

* Remove image generation for now

* more tests and updates

* column_type_is_llm_generated -> column_type_is_model_generated

* change set to list: fix flaky tests

* CompletionInferenceParameters -> ChatCompletionInferenceParameters for consistency with generation_type

* Update docs

* fix deprecation warning originating from cli model settings

* update display of inference parameters in cli list

* save prog on inference parameter

* updates for the ocnfig builder

* update cli readme

* update cli for inference parmeters

* update inference parameter names

* flip order of vars

* WithCompletion -> WithChatCompletion

* specify InferenceParamsT

* Update columns.md with EmbeddingColumnConfig info

* make generation_type a descriminator field in inference params. add configuration support for max_parallel_requests and timeout

* DRY out some stuff in field.py

* Update nomenclature. prompt tokens -> input tokens, completion tokens -> output tokens in column statistics for consistency

* Add nvidia-embedding and openai-embedding to default model configs

* Fix typo in docs

* Make generate collab notebooks

* fine-tune -> adjust
2025-12-15 11:03:33 -07:00
Andre Manoel
f55211cf3c
feat: processor to easily export part of dataset to JSONL (#26) 2025-12-10 20:59:30 -03:00
Mike Knepper
32515ba724
style: Sort imports traditionally instead of within sections (#103) 2025-12-08 09:01:58 -06:00
Johnny Greco
7c88230d74
chore: porting nmp (#11)
* porting nmp

* remove unused seed dataset datastore tests

* remove load_dataset
2025-11-04 18:09:55 -05:00
Johnny Greco
6d9836e2ee add and run pre-commit 2025-10-27 18:10:36 -04:00
Johnny Greco
7ed5e78741 initial port 2025-10-27 14:29:12 -04:00