DataDesigner/packages/data-designer-engine/tests/engine/models/conftest.py
Nabin Mulepati ebd33227c3
refactor: Decouple ModelFacade from LiteLLM via ModelClient adapter (#373)
* plans for model facade overhaul

* update plan

* add review

* address feedback + add more details after several self reviews

* update plan doc

* address nits

* Add cannonical objects

* self-review feedback + address

* add LiteLLMRouter protocol to strongly type bridge router param

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* simplify some things

* add a protol for http response like object

* move HttpResponse

* update PR-1 architecture notes for lifecycle and router protocol

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Address PR #359 feedback: exception wrapping, shared parsing, test improvements

- Wrap all LiteLLM router calls in try/except to normalize raw exceptions
  into canonical ProviderError at the bridge boundary (blocking review item)
- Extract reusable response-parsing helpers into clients/parsing.py for
  shared use across future native adapters
- Add async image parsing path using httpx.AsyncClient to avoid blocking
  the event loop in agenerate_image
- Add retry_after field to ProviderError for future retry engine support
- Fix _to_int_or_none to parse numeric strings from providers
- Create test conftest.py with shared mock_router/bridge_client fixtures
- Parametrize duplicate image generation and error mapping tests
- Add tests for exception wrapping across all bridge methods

* Use contextlib to dry out some code

* Address Greptile feedback: HTTP-date retry-after parsing, docstring clarity

- Parse RFC 7231 HTTP-date strings in Retry-After header (used by
  Azure and Anthropic during rate-limiting) in addition to numeric
  delay-seconds
- Clarify collect_non_none_optional_fields docstring explaining why
  f.default is None is the correct check for optional field forwarding
- Add tests for HTTP-date and garbage Retry-After values

* Address Greptile feedback: FastAPI detail parsing, comment fixes

- Fix misleading comment about prompt field defaults in _IMAGE_EXCLUDE
- Handle list-format detail arrays in _extract_structured_message for
  FastAPI/Pydantic validation errors
- Document scope boundary for vision content in collect_raw_image_candidates

* add PR-2 architecture notes for model facade overhaul

* save progress on pr2

* small refactor

* address feedback

* Address greptile comment in pr1

* refactor ProviderError from dataclass to regular Exception

- Replace @dataclass + __post_init__ with explicit __init__ that calls
  super().__init__ properly, avoiding brittle field-ordering dependency
- Store cause via __cause__ only, removing the redundant .cause attr
- Update match pattern in handle_llm_exceptions for non-dataclass type
- Rename shadowed local `fields` to `optional_fields` in TransportKwargs

* Address greptile feedback

* PR feedback

* track usage tracking in finally block for images

* pr feedback

* wrap facade close in try/catch

* clean up stray params

* fix stray inclusion of metadata

* small regression fix

* address more feedback

---------

Co-authored-by: Johnny Greco <jogreco@nvidia.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 14:30:40 -06:00

102 lines
3.2 KiB
Python

# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
from __future__ import annotations
from pathlib import Path
from unittest.mock import MagicMock
import pytest
from data_designer.config.models import (
ChatCompletionInferenceParams,
EmbeddingInferenceParams,
ModelConfig,
)
from data_designer.engine.model_provider import ModelProvider, ModelProviderRegistry
from data_designer.engine.models.clients.base import ModelClient
from data_designer.engine.models.factory import create_model_registry
from data_designer.engine.models.registry import ModelRegistry
from data_designer.engine.secret_resolver import SecretsFileResolver
from data_designer.engine.testing import StubMCPFacade, StubMCPRegistry
@pytest.fixture
def stub_secrets_resolver() -> SecretsFileResolver:
module_path = Path(__file__).parent
return SecretsFileResolver(module_path / "stub_secrets.json")
@pytest.fixture
def stub_model_provider_registry() -> ModelProviderRegistry:
return ModelProviderRegistry(
providers=[
ModelProvider(
name="stub-model-provider",
endpoint="https://api.example.com/v1",
provider_type="openai",
api_key="STUB_API_KEY",
)
]
)
@pytest.fixture
def stub_model_configs() -> list[ModelConfig]:
return [
ModelConfig(
alias="stub-text",
model="stub-model-text",
provider="stub-model-provider",
inference_parameters=ChatCompletionInferenceParams(
temperature=0.80, top_p=0.95, max_tokens=100, max_parallel_requests=10, timeout=100
),
),
ModelConfig(
alias="stub-reasoning",
model="stub-model-reasoning",
provider="stub-model-provider",
inference_parameters=ChatCompletionInferenceParams(
temperature=0.80, top_p=0.95, max_tokens=100, max_parallel_requests=10, timeout=100
),
),
ModelConfig(
alias="stub-embedding",
model="stub-model-embedding",
provider="stub-model-provider",
inference_parameters=EmbeddingInferenceParams(
dimensions=100,
),
),
]
@pytest.fixture
def stub_model_registry(
stub_model_configs: list[ModelConfig],
stub_secrets_resolver: SecretsFileResolver,
stub_model_provider_registry: ModelProviderRegistry,
) -> ModelRegistry:
return create_model_registry(
model_configs=stub_model_configs,
secret_resolver=stub_secrets_resolver,
model_provider_registry=stub_model_provider_registry,
)
@pytest.fixture
def stub_model_client() -> MagicMock:
"""Mock ModelClient for testing ModelFacade without a real LiteLLM router."""
return MagicMock(spec=ModelClient)
@pytest.fixture
def stub_mcp_facade_for_model() -> StubMCPFacade:
"""Default stub MCP facade with max_tool_call_turns=3."""
return StubMCPFacade()
@pytest.fixture
def stub_mcp_registry_for_model(stub_mcp_facade_for_model: StubMCPFacade) -> StubMCPRegistry:
"""Default stub MCP registry."""
return StubMCPRegistry(stub_mcp_facade_for_model)