Elgato_dark/unsloth
1
0
Fork 0
mirror of https://github.com/unslothai/unsloth synced 2026-04-21 13:37:39 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Let recipes use the model loaded in Chat (#4840)

Browse source 
* feat: inject local model provider into recipe jobs via JWT

* feat: auto-generate JWT for local model providers in recipes

* feat: add is_local flag to model provider config types and utils

* fix(studio): skip endpoint validation for local providers

* feat(studio): add local/external model source toggle to provider dialog

* feat(studio): thread localProviderNames through model config dialog chain

* feat(studio): show 'Local model (Chat)' label for local model_provider configs

* fix: hardcode loopback for local endpoint, clear stale creds on toggle

* fix: document TOCTOU/JWT rotation, add deferred import comments, fix is_local serialization

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): clear stale local model state on provider toggle and validation

* fix(studio): override empty local endpoint in validation and skip model gate for unused providers

* fix(studio): resolve loopback port from app.state, clear stale local provider fields, sync model id on toggle

Address review feedback on the local-model-provider flow:

- Backend (jobs.py): _resolve_local_v1_endpoint now reads the actual bound
  port from app.state.server_port (set in run.py after binding) instead of
  parsing it out of request.base_url, which is wrong behind any reverse
  proxy or non-default port. The two duplicated urlparse blocks are gone.
- Backend (jobs.py): defensively pop api_key_env, extra_headers, extra_body
  from local providers so a previously external provider that flipped to
  local cannot leak invalid JSON or rogue auth headers into the local /v1
  call. Also dedupe the post-loop assignment and tighten the local-name
  intersection so empty names cannot match.
- Backend (jobs.py): hoist datetime and urllib.parse imports to the top
  import block for consistency with the rest of the file.
- Backend (run.py): expose the bound port on app.state.server_port after
  the uvicorn server is constructed.
- Frontend (model-provider-dialog.tsx): clear extra_headers and extra_body
  when toggling to local mode. Hidden inputs would otherwise keep stale
  JSON blocking validate/run.
- Frontend (model-config-dialog.tsx): factor the local-aware provider
  selection logic into applyProviderChange and call it from both
  onValueChange and onBlur, so manually typing a provider name and tabbing
  away keeps the model field consistent.
- Frontend (recipe-studio.ts store): handle both directions of the
  is_local toggle in the cascade. external -> local now backfills
  model: "local" on already-linked model_configs so they pass validation
  immediately, mirroring the existing local -> external clear path.
- Frontend (validate.ts + build-payload.ts): thread localProviderNames
  into validateModelConfigProviders and skip the "model is required"
  check for local-linked configs. Local providers do not need a real
  model id since the inference endpoint uses the loaded Chat model.

* fix(studio): narrow store cascade types, sync model placeholder on graph relink and node removal, harden ephemeral port path

Loop 2 review fixes:

- recipe-studio.ts: type-narrow next.is_local by also checking
  next.kind === "model_provider". TS otherwise raised TS2339 because
  next was typed as the union NodeConfig after the spread. The behavior
  is unchanged but the code now compiles cleanly.
- model-config-dialog.tsx: convert the lastProviderRef / providerInputRef
  ref-during-render pattern (pre-existing react-hooks/refs lint error)
  to a useEffect that syncs providerInputRef from config.provider. The
  combobox blur path still uses applyProviderChange and remains stable.
- recipe-graph-connection.ts: when a graph drag links a model_provider
  to a model_config, mirror the dialog applyProviderChange behavior:
  fill model: "local" if the new provider is local and the model field
  is blank, clear model when relinking from a local placeholder to an
  external provider, otherwise leave the model alone.
- reference-sync.ts: when a referenced provider node is removed, clear
  the synthetic model: "local" placeholder along with the provider
  field, so a future relink to an external provider does not pass
  validation with a stale value that fails at runtime.
- run.py: only publish app.state.server_port when the bound port is a
  real positive integer; for ephemeral binds (port==0) leave it unset
  and let request handlers fall back to request.base_url.
- jobs.py: _resolve_local_v1_endpoint also falls back when
  app.state.server_port is non-positive, and uses `is None` instead of
  the truthy fallback so a literal 0 is handled correctly.

* fix(studio): strict is_local check, narrow loaded-model gate to LLM-reachable configs, add scope-server port fallback

Loop 3 review fixes:

- jobs.py, validate.py: require `is_local is True` instead of truthy
  check. Malformed payloads such as is_local: "false" or is_local: 1
  would otherwise be treated as local and silently rewritten to the
  loopback endpoint.
- jobs.py: _resolve_local_v1_endpoint now tries request.scope["server"]
  (the actual uvicorn-assigned (host, port) tuple) as a second
  resolution step before falling back to parsing request.base_url.
  This covers direct-uvicorn startup paths and ephemeral binds that
  never publish app.state.server_port.
- jobs.py: new _used_llm_model_aliases helper collects the set of
  model_aliases that an LLM column actually references, and the
  "Chat model loaded" gate is now only triggered when a local
  provider is reachable from that set. Orphan model_config nodes on
  the canvas no longer block unrelated recipe runs.

* fix(studio): force skip_health_check on local-linked configs, skip JSON parsing for local providers, local-aware inline editor

Loop 4 review fixes:

- jobs.py: after rewriting local providers, also force
  skip_health_check: true on any model_config linked to a local
  provider. The /v1/models endpoint only advertises the real loaded
  model id, so data_designer's default model-availability health check
  would otherwise fail against the placeholder "local" id before the
  first chat completion call. The inference route already ignores the
  model id in chat completions, so skipping the check is safe.
- builders-model.ts: buildModelProvider now short-circuits for local
  providers and emits only { name, endpoint: "", provider_type, is_local }
  without running parseJsonObject on the hidden extra_headers/extra_body
  inputs. Imported or hydrated recipes with stale invalid JSON in those
  fields no longer block client-side validate/run.
- inline-model.tsx: the model_config branch now accepts an optional
  localProviderNames prop and mirrors the dialog applyProviderChange
  behavior. Changing provider to/from a local one auto-fills or clears
  the "local" placeholder consistently with the other edit paths.
- recipe-graph-node.tsx: derive localProviderNames from the store via
  useMemo (stable identity) and pass it through renderNodeBody to
  <InlineModel>. Hooks order is preserved by declaring them above the
  early return for markdown_note nodes.
- run.py: minor comment tweak - loop 3 already added the scope-server
  fallback path, note that in the comment.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <info@unsloth.ai>
This commit is contained in:
Wasim Yousef Said 2026-04-08 12:48:22 +02:00 committed by GitHub
parent c3d2d58046
commit 8e977445d4
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
20 changed files with 518 additions and 95 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

164
studio/backend/routes/data_recipe/jobs.py
View file

@ -5,7 +5,9 @@
from __future__ import annotations
from datetime import timedelta
from typing import Any
from urllib.parse import urlparse
from fastapi import APIRouter, HTTPException, Query, Request
from fastapi.responses import JSONResponse, StreamingResponse
@ -26,6 +28,161 @@ from models.data_recipe import (
router = APIRouter()
def _resolve_local_v1_endpoint(request: Request) -> str:
"""Return the loopback /v1 URL for the actual backend listen port.
Resolution order:
1. ``app.state.server_port`` - explicitly published by run.py after
the uvicorn server has bound. This is the most reliable source
because it survives reverse proxies, TLS terminators and tunnels.
2. ``request.scope["server"]`` - the real (host, port) tuple uvicorn
sets when the request is dispatched. Used when Studio is started
outside ``run_server`` (e.g. ``uvicorn studio.backend.main:app``).
3. ``request.base_url`` parsed - last resort for test fixtures that
do not route through a live uvicorn server.
"""
port: Any = getattr(request.app.state, "server_port", None)
if not isinstance(port, int) or port <= 0:
server = request.scope.get("server")
if (
isinstance(server, tuple)
and len(server) >= 2
and isinstance(server[1], int)
and server[1] > 0
):
port = server[1]
else:
parsed = urlparse(str(request.base_url))
port = parsed.port if parsed.port is not None else 8888
return f"http://127.0.0.1:{int(port)}/v1"
def _used_llm_model_aliases(recipe: dict[str, Any]) -> set[str]:
"""Return the set of model_aliases that are actually referenced by an
LLM column. Used to narrow the "Chat model loaded" gate so that orphan
model_config nodes on the canvas do not block unrelated recipe runs.
The ``llm-`` prefix matches the existing convention in
``core/data_recipe/service.py::_recipe_has_llm_columns`` and covers all
LLM column types emitted by the frontend (llm-text, llm-code,
llm-structured, llm-judge).
"""
aliases: set[str] = set()
for column in recipe.get("columns", []):
if not isinstance(column, dict):
continue
column_type = column.get("column_type")
if not isinstance(column_type, str) or not column_type.startswith("llm-"):
continue
alias = column.get("model_alias")
if isinstance(alias, str) and alias:
aliases.add(alias)
return aliases
def _inject_local_providers(recipe: dict[str, Any], request: Request) -> None:
"""
Mutate recipe dict in-place: for any provider with is_local=True,
generate a JWT and fill in the endpoint pointing at this server.
"""
providers = recipe.get("model_providers")
if not providers:
return
# Collect local providers and pop is_local from ALL dicts unconditionally.
# Strict `is True` guard so malformed payloads (is_local: 1,
# is_local: "true") do not accidentally trigger the loopback rewrite.
local_indices: list[int] = []
for i, provider in enumerate(providers):
if not isinstance(provider, dict):
continue
is_local = provider.pop("is_local", None)
if is_local is True:
local_indices.append(i)
if not local_indices:
return
endpoint = _resolve_local_v1_endpoint(request)
# Only gate on model-loaded if a local provider is actually reachable
# from an LLM column through a model_config. Orphan model_config nodes
# that reference a local provider but that no LLM column uses should
# not block runs; the recipe would never call /v1 for them.
local_names = {
providers[i].get("name") for i in local_indices if providers[i].get("name")
}
used_aliases = _used_llm_model_aliases(recipe)
referenced_providers = {
mc.get("provider")
for mc in recipe.get("model_configs", [])
if (
isinstance(mc, dict)
and mc.get("provider")
and mc.get("alias") in used_aliases
)
}
token = ""
if local_names & referenced_providers:
# Verify a model is loaded.
# NOTE: This is a point-in-time check (TOCTOU). The model could be unloaded
# or swapped after this check but before the recipe subprocess calls /v1.
# The inference endpoint returns a clear 400 in that case.
#
# Imports are deferred to avoid circular dependencies with inference modules.
from routes.inference import get_llama_cpp_backend
from core.inference import get_inference_backend
llama = get_llama_cpp_backend()
model_loaded = llama.is_loaded
if not model_loaded:
backend = get_inference_backend()
model_loaded = bool(backend.active_model_name)
if not model_loaded:
raise ValueError(
"No model loaded in Chat. Load a model first, then run the recipe."
)
from auth.authentication import (
create_access_token,
) # deferred: avoids circular import
# Uses the "unsloth" admin subject. If the user changes their password,
# the JWT secret rotates and this token becomes invalid mid-run.
# Acceptable for v1 - recipes typically finish well within one session.
token = create_access_token(
subject = "unsloth",
expires_delta = timedelta(hours = 24),
)
# Defensively strip any stale "external"-only fields the frontend may
# have left on the dict (extra_headers/extra_body/api_key_env). The UI
# hides these inputs in local mode but the payload builder still serializes
# them, so a previously external provider that flipped to local can carry
# invalid JSON or rogue auth headers into the local /v1 call.
for i in local_indices:
providers[i]["endpoint"] = endpoint
providers[i]["api_key"] = token
providers[i]["provider_type"] = "openai"
providers[i].pop("api_key_env", None)
providers[i].pop("extra_headers", None)
providers[i].pop("extra_body", None)
# Force skip_health_check on any model_config that references a local
# provider. The local /v1/models endpoint only lists the real loaded
# model (e.g. "unsloth/llama-3.2-1b") and not the placeholder "local"
# that the recipe sends as the model id, so data_designer's pre-flight
# health check would otherwise fail before the first completion call.
# The backend route ignores the model id field in chat completions, so
# skipping the check is safe.
for mc in recipe.get("model_configs", []):
if not isinstance(mc, dict):
continue
if mc.get("provider") in local_names:
mc["skip_health_check"] = True
def _normalize_run_name(value: Any) -> str | None:
if value is None:
return None
@ -40,7 +197,7 @@ def _normalize_run_name(value: Any) -> str | None:
@router.post("/jobs", response_class = JSONResponse, response_model = JobCreateResponse)
def create_job(payload: RecipePayload):
def create_job(payload: RecipePayload, request: Request):
recipe = payload.recipe
if not recipe.get("columns"):
raise HTTPException(status_code = 400, detail = "Recipe must include columns.")
@ -67,6 +224,11 @@ def create_job(payload: RecipePayload):
status_code = 400, detail = f"invalid run_config: {exc}"
) from exc
try:
_inject_local_providers(recipe, request)
except ValueError as exc:
raise HTTPException(status_code = 400, detail = str(exc)) from exc
mgr = get_job_manager()
try:
job_id = mgr.start(recipe = recipe, run = run)

16
studio/backend/routes/data_recipe/validate.py
View file

@ -68,6 +68,20 @@ def _collect_validation_errors(recipe: dict[str, Any]) -> list[ValidateError]:
return errors
def _patch_local_providers(recipe: dict[str, Any]) -> None:
"""Strip is_local and fill a dummy endpoint so validation doesn't choke.
Uses a strict `is True` check to match _inject_local_providers in
jobs.py - malformed payloads with truthy but non-boolean is_local
values should not be treated as local.
"""
for provider in recipe.get("model_providers", []):
if not isinstance(provider, dict):
continue
if provider.pop("is_local", None) is True:
provider["endpoint"] = "http://127.0.0.1"
@router.post("/validate", response_model = ValidateResponse)
def validate(payload: RecipePayload) -> ValidateResponse:
recipe = payload.recipe
@ -77,6 +91,8 @@ def validate(payload: RecipePayload) -> ValidateResponse:
errors = [ValidateError(message = "Recipe must include columns.")],
)
_patch_local_providers(recipe)
try:
validate_recipe(recipe)
except RuntimeError as exc:

8
studio/backend/run.py
View file

@ -324,6 +324,14 @@ def run_server(
_server = uvicorn.Server(config)
_shutdown_event = Event()
# Expose the actual bound port so request-handling code can build
# loopback URLs that point at the real backend, not whatever port a
# reverse proxy or tunnel exposed in the request URL. Only publish
# an explicit value when we know the concrete port; for ephemeral
# binds (port==0) leave it unset and let request handlers fall back
# to the ASGI request scope or request.base_url.
app.state.server_port = port if port and port > 0 else None
# Run server in a daemon thread
def _run():
asyncio.run(_server.serve())

2
studio/frontend/src/features/recipe-studio/blocks/render-dialog.tsx
View file

@ -28,6 +28,7 @@ export function renderBlockDialog(
categoryOptions: SamplerConfig[],
modelConfigAliases: string[],
modelProviderOptions: string[],
localProviderNames: Set<string>,
toolProfileAliases: string[],
datetimeOptions: string[],
onUpdate: (id: string, patch: Partial<NodeConfig>) => void,
@ -109,6 +110,7 @@ export function renderBlockDialog(
<ModelConfigDialog
config={config}
providerOptions={modelProviderOptions}
localProviderNames={localProviderNames}
onUpdate={update}
/>
) : null;

39
studio/frontend/src/features/recipe-studio/components/inline/inline-model.tsx
View file

@ -10,11 +10,21 @@ type InlineModelPatch = Partial<ModelProviderConfig> | Partial<ModelConfig>;
type InlineModelProps = {
config: ModelProviderConfig | ModelConfig;
localProviderNames?: Set<string>;
onUpdate: (patch: InlineModelPatch) => void;
};
export function InlineModel(props: InlineModelProps): ReactElement {
if (props.config.kind === "model_provider") {
if (props.config.is_local) {
return (
<div className="flex items-center gap-2 px-1 py-0.5">
<span className="text-xs font-medium text-muted-foreground">
Local model (Chat)
</span>
</div>
);
}
return (
<div className="grid gap-3 sm:grid-cols-2">
<InlineField label="Endpoint">
@ -42,21 +52,40 @@ export function InlineModel(props: InlineModelProps): ReactElement {
);
}
// model_config branch - mirror the local-aware provider sync from the
// dialog path so inline edits do not leave stale "local" placeholders
// on external providers and fill the placeholder when switching to local.
const localNames = props.localProviderNames ?? new Set<string>();
const modelConfig = props.config;
const handleProviderChange = (nextProvider: string) => {
const isLocal = localNames.has(nextProvider);
if (isLocal && !modelConfig.model.trim()) {
props.onUpdate({ provider: nextProvider, model: "local" });
return;
}
if (!isLocal && modelConfig.model === "local") {
props.onUpdate({ provider: nextProvider, model: "" });
return;
}
props.onUpdate({ provider: nextProvider });
};
const isLinkedToLocal = localNames.has(modelConfig.provider);
return (
<div className="grid gap-3 sm:grid-cols-2">
<InlineField label="Provider">
<Input
className="nodrag h-8 w-full text-xs"
placeholder="provider alias"
value={props.config.provider}
onChange={(event) => props.onUpdate({ provider: event.target.value })}
value={modelConfig.provider}
onChange={(event) => handleProviderChange(event.target.value)}
/>
</InlineField>
<InlineField label="Model">
<Input
className="nodrag h-8 w-full text-xs"
placeholder="gpt-4o-mini"
value={props.config.model}
placeholder={isLinkedToLocal ? "local" : "gpt-4o-mini"}
value={modelConfig.model}
onChange={(event) => props.onUpdate({ model: event.target.value })}
/>
</InlineField>
@ -65,7 +94,7 @@ export function InlineModel(props: InlineModelProps): ReactElement {
className="nodrag h-8 w-full text-xs"
type="number"
placeholder="0.7"
value={props.config.inference_temperature ?? ""}
value={modelConfig.inference_temperature ?? ""}
onChange={(event) =>
props.onUpdate({
// biome-ignore lint/style/useNamingConvention: api schema

28
studio/frontend/src/features/recipe-studio/components/recipe-graph-node.tsx
View file

@ -30,7 +30,7 @@ import {
Position,
useUpdateNodeInternals,
} from "@xyflow/react";
import { type ReactElement, memo, useEffect } from "react";
import { type ReactElement, memo, useEffect, useMemo } from "react";
import {
MAX_NODE_WIDTH,
MAX_NOTE_NODE_WIDTH,
@ -287,6 +287,7 @@ function renderNodeBody(
config: NodeConfig | undefined,
summary: string,
updateConfig: (id: string, patch: Partial<NodeConfig>) => void,
localProviderNames: Set<string>,
): ReactElement {
if (config?.kind === "markdown_note") {
return <MarkdownPreview markdown={config.markdown} />;
@ -300,7 +301,13 @@ function renderNodeBody(
return <InlineSampler config={config} onUpdate={onUpdate} />;
}
if (config.kind === "model_provider" || config.kind === "model_config") {
return <InlineModel config={config} onUpdate={onUpdate} />;
return (
<InlineModel
config={config}
localProviderNames={localProviderNames}
onUpdate={onUpdate}
/>
);
}
if (config.kind === "llm") {
return <InlineLlm config={config} onUpdate={onUpdate} />;
@ -355,6 +362,16 @@ function RecipeGraphNodeBase({
const config = useRecipeStudioStore((state) => state.configs[id]);
const openConfig = useRecipeStudioStore((state) => state.openConfig);
const updateConfig = useRecipeStudioStore((state) => state.updateConfig);
const allConfigs = useRecipeStudioStore((state) => state.configs);
const localProviderNames = useMemo(() => {
const names = new Set<string>();
for (const cfg of Object.values(allConfigs)) {
if (cfg.kind === "model_provider" && cfg.is_local === true) {
names.add(cfg.name);
}
}
return names;
}, [allConfigs]);
const llmAuxVisible = useRecipeStudioStore(
(state) => state.llmAuxVisibility[id] ?? false,
);
@ -418,7 +435,12 @@ function RecipeGraphNodeBase({
data.kind === "tool_config" ||
data.kind === "validator";
const summary = getConfigSummary(config);
const nodeBody = renderNodeBody(config, summary, updateConfig);
const nodeBody = renderNodeBody(
config,
summary,
updateConfig,
localProviderNames,
);
const canShowLlmAux =
config?.kind === "llm" &&
(Boolean(config.prompt.trim()) ||

3
studio/frontend/src/features/recipe-studio/dialogs/config-dialog.tsx
View file

@ -18,6 +18,7 @@ type ConfigDialogProps = {
categoryOptions: SamplerConfig[];
modelConfigAliases: string[];
modelProviderOptions: string[];
localProviderNames: Set<string>;
toolProfileAliases: string[];
datetimeOptions: string[];
onUpdate: (id: string, patch: Partial<NodeConfig>) => void;
@ -32,6 +33,7 @@ export function ConfigDialog({
categoryOptions,
modelConfigAliases,
modelProviderOptions,
localProviderNames,
toolProfileAliases,
datetimeOptions,
onUpdate,
@ -101,6 +103,7 @@ export function ConfigDialog({
categoryOptions,
modelConfigAliases,
modelProviderOptions,
localProviderNames,
toolProfileAliases,
datetimeOptions,
onUpdate,

38
studio/frontend/src/features/recipe-studio/dialogs/models/model-config-dialog.tsx
View file

@ -17,7 +17,7 @@ import {
} from "@/components/ui/combobox";
import { Input } from "@/components/ui/input";
import { Textarea } from "@/components/ui/textarea";
import { type ReactElement, useRef, useState } from "react";
import { type ReactElement, useEffect, useRef, useState } from "react";
import type { ModelConfig } from "../../types";
import { CollapsibleSectionTriggerButton } from "../shared/collapsible-section-trigger";
import { FieldLabel } from "../shared/field-label";
@ -26,14 +26,17 @@ import { NameField } from "../shared/name-field";
type ModelConfigDialogProps = {
config: ModelConfig;
providerOptions: string[];
localProviderNames: Set<string>;
onUpdate: (patch: Partial<ModelConfig>) => void;
};
export function ModelConfigDialog({
config,
providerOptions,
localProviderNames,
onUpdate,
}: ModelConfigDialogProps): ReactElement {
const isLinkedToLocal = localProviderNames.has(config.provider);
const [optionalOpen, setOptionalOpen] = useState(false);
const modelId = `${config.id}-model`;
const providerId = `${config.id}-provider`;
@ -44,11 +47,13 @@ export function ModelConfigDialog({
const extraBodyId = `${config.id}-inference-extra-body`;
const providerAnchorRef = useRef<HTMLDivElement>(null);
const providerInputRef = useRef(config.provider);
const lastProviderRef = useRef(config.provider);
if (lastProviderRef.current !== config.provider) {
lastProviderRef.current = config.provider;
// Sync providerInputRef with the current provider value. Updating a ref in
// an effect (vs reading/writing it during render) satisfies the
// react-hooks/refs rule and keeps the combobox blur path stable across
// re-renders.
useEffect(() => {
providerInputRef.current = config.provider;
}
}, [config.provider]);
const updateField = <K extends keyof ModelConfig>(
key: K,
value: ModelConfig[K],
@ -56,6 +61,21 @@ export function ModelConfigDialog({
onUpdate({ [key]: value } as Partial<ModelConfig>);
};
// Apply provider selection while keeping the local-provider model autofill
// consistent across both dropdown selection and free-typed + blur input.
const applyProviderChange = (selectedProvider: string) => {
const isLocal = localProviderNames.has(selectedProvider);
if (isLocal && !config.model.trim()) {
onUpdate({ provider: selectedProvider, model: "local" });
return;
}
if (!isLocal && config.model === "local") {
onUpdate({ provider: selectedProvider, model: "" });
return;
}
updateField("provider", selectedProvider);
};
return (
<div className="space-y-4">
<NameField
@ -84,7 +104,7 @@ export function ModelConfigDialog({
filteredItems={providerOptions}
filter={null}
value={config.provider || null}
onValueChange={(value) => updateField("provider", value ?? "")}
onValueChange={(value) => applyProviderChange(value ?? "")}
onInputValueChange={(value) => {
providerInputRef.current = value;
}}
@ -98,7 +118,7 @@ export function ModelConfigDialog({
onBlur={() => {
const next = providerInputRef.current;
if (next !== config.provider) {
updateField("provider", next);
applyProviderChange(next);
}
}}
/>
@ -124,12 +144,12 @@ export function ModelConfigDialog({
<FieldLabel
label="Model ID"
htmlFor={modelId}
hint="The exact model name sent to the connection."
hint={isLinkedToLocal ? "Uses the model loaded in Chat. Any value works here." : "The exact model name sent to the connection."}
/>
<Input
id={modelId}
className="nodrag"
placeholder="gpt-4o-mini"
placeholder={isLinkedToLocal ? "local" : "gpt-4o-mini"}
value={config.model}
onChange={(event) => updateField("model", event.target.value)}
/>

218
studio/frontend/src/features/recipe-studio/dialogs/models/model-provider-dialog.tsx
View file

@ -24,6 +24,7 @@ export function ModelProviderDialog({
onUpdate,
}: ModelProviderDialogProps): ReactElement {
const [optionalOpen, setOptionalOpen] = useState(false);
const isLocal = config.is_local ?? false;
const endpointId = `${config.id}-endpoint`;
const apiKeyEnvId = `${config.id}-api-key-env`;
const apiKeyId = `${config.id}-api-key`;
@ -43,94 +44,163 @@ export function ModelProviderDialog({
value={config.name}
onChange={(value) => onUpdate({ name: value })}
/>
<div className="rounded-2xl border border-border/60 bg-muted/10 px-4 py-3">
<p className="text-sm font-semibold text-foreground">
Start with the endpoint you want this model to use
</p>
<p className="mt-1 text-xs text-muted-foreground">
Most connections only need an endpoint. Add an API key if that
service requires one.
</p>
</div>
{/* Model source toggle */}
<div className="grid gap-1.5">
<FieldLabel
label="Endpoint"
htmlFor={endpointId}
hint="Base URL for the model service or gateway."
/>
<Input
id={endpointId}
className="nodrag"
placeholder="https://..."
value={config.endpoint}
onChange={(event) => updateField("endpoint", event.target.value)}
/>
<p className="text-sm font-semibold text-foreground">Model source</p>
<div className="grid grid-cols-2 gap-2">
<button
type="button"
className={`rounded-xl border px-4 py-3 text-left transition-colors ${
isLocal
? "border-primary/40 bg-primary/5"
: "border-border/60 bg-muted/10 hover:border-border"
}`}
onClick={() =>
onUpdate({
is_local: true,
endpoint: "",
api_key: "",
api_key_env: "",
extra_headers: "",
extra_body: "",
})
}
>
<p className="text-sm font-semibold text-foreground">
Local model
</p>
<p className="mt-0.5 text-xs text-muted-foreground">
Use the model loaded in the Chat tab
</p>
</button>
<button
type="button"
className={`rounded-xl border px-4 py-3 text-left transition-colors ${
!isLocal
? "border-primary/40 bg-primary/5"
: "border-border/60 bg-muted/10 hover:border-border"
}`}
onClick={() => onUpdate({ is_local: false })}
>
<p className="text-sm font-semibold text-foreground">
External endpoint
</p>
<p className="mt-0.5 text-xs text-muted-foreground">
Connect to an API like OpenAI, Together, or a custom server
</p>
</button>
</div>
</div>
<div className="grid gap-1.5">
<FieldLabel
label="API key (optional)"
htmlFor={apiKeyId}
hint="Paste a key here, or use an environment variable below."
/>
<Input
id={apiKeyId}
className="nodrag"
value={config.api_key ?? ""}
onChange={(event) => updateField("api_key", event.target.value)}
/>
</div>
<Collapsible open={optionalOpen} onOpenChange={setOptionalOpen}>
<CollapsibleTrigger asChild={true}>
<CollapsibleSectionTriggerButton
label="Advanced request overrides"
open={optionalOpen}
/>
</CollapsibleTrigger>
<CollapsibleContent className="mt-3 space-y-4">
{isLocal ? (
<div className="rounded-2xl border border-border/60 bg-muted/10 px-4 py-3">
<p className="text-sm font-semibold text-foreground">
Ready to go
</p>
<p className="mt-1 text-xs text-muted-foreground">
Recipes will use whatever model is loaded in the Chat tab when you
hit run. No endpoint or API key needed.
</p>
</div>
) : (
<>
<div className="rounded-2xl border border-border/60 bg-muted/10 px-4 py-3">
<p className="text-sm font-semibold text-foreground">
Start with the endpoint you want this model to use
</p>
<p className="mt-1 text-xs text-muted-foreground">
Most connections only need an endpoint. Add an API key if that
service requires one.
</p>
</div>
<div className="grid gap-1.5">
<FieldLabel
label="API key environment variable"
htmlFor={apiKeyEnvId}
hint="Name of the environment variable that stores the key."
label="Endpoint"
htmlFor={endpointId}
hint="Base URL for the model service or gateway."
/>
<Input
id={apiKeyEnvId}
id={endpointId}
className="nodrag"
placeholder="OPENAI_API_KEY"
value={config.api_key_env ?? ""}
onChange={(event) => updateField("api_key_env", event.target.value)}
placeholder="https://..."
value={config.endpoint}
onChange={(event) => updateField("endpoint", event.target.value)}
/>
</div>
<div className="grid gap-1.5">
<FieldLabel
label="Extra headers (JSON)"
htmlFor={extraHeadersId}
hint="Optional headers to send with every request."
label="API key (optional)"
htmlFor={apiKeyId}
hint="Paste a key here, or use an environment variable below."
/>
<Textarea
id={extraHeadersId}
className="corner-squircle nodrag"
placeholder='{"X-Header": "value"}'
value={config.extra_headers ?? ""}
onChange={(event) => updateField("extra_headers", event.target.value)}
<Input
id={apiKeyId}
className="nodrag"
value={config.api_key ?? ""}
onChange={(event) => updateField("api_key", event.target.value)}
/>
</div>
<div className="grid gap-1.5">
<FieldLabel
label="Extra body (JSON)"
htmlFor={extraBodyId}
hint="Optional request fields to send every time."
/>
<Textarea
id={extraBodyId}
className="corner-squircle nodrag"
placeholder='{"key": "value"}'
value={config.extra_body ?? ""}
onChange={(event) => updateField("extra_body", event.target.value)}
/>
</div>
</CollapsibleContent>
</Collapsible>
<Collapsible open={optionalOpen} onOpenChange={setOptionalOpen}>
<CollapsibleTrigger asChild={true}>
<CollapsibleSectionTriggerButton
label="Advanced request overrides"
open={optionalOpen}
/>
</CollapsibleTrigger>
<CollapsibleContent className="mt-3 space-y-4">
<div className="grid gap-1.5">
<FieldLabel
label="API key environment variable"
htmlFor={apiKeyEnvId}
hint="Name of the environment variable that stores the key."
/>
<Input
id={apiKeyEnvId}
className="nodrag"
placeholder="OPENAI_API_KEY"
value={config.api_key_env ?? ""}
onChange={(event) =>
updateField("api_key_env", event.target.value)
}
/>
</div>
<div className="grid gap-1.5">
<FieldLabel
label="Extra headers (JSON)"
htmlFor={extraHeadersId}
hint="Optional headers to send with every request."
/>
<Textarea
id={extraHeadersId}
className="corner-squircle nodrag"
placeholder='{"X-Header": "value"}'
value={config.extra_headers ?? ""}
onChange={(event) =>
updateField("extra_headers", event.target.value)
}
/>
</div>
<div className="grid gap-1.5">
<FieldLabel
label="Extra body (JSON)"
htmlFor={extraBodyId}
hint="Optional request fields to send every time."
/>
<Textarea
id={extraBodyId}
className="corner-squircle nodrag"
placeholder='{"key": "value"}'
value={config.extra_body ?? ""}
onChange={(event) =>
updateField("extra_body", event.target.value)
}
/>
</div>
</CollapsibleContent>
</Collapsible>
</>
)}
</div>
);
}

1
studio/frontend/src/features/recipe-studio/recipe-studio-page.tsx
View file

@ -776,6 +776,7 @@ export function RecipeStudioPage({
categoryOptions={dialogOptions.categoryOptions}
modelConfigAliases={dialogOptions.modelConfigAliases}
modelProviderOptions={dialogOptions.modelProviderOptions}
localProviderNames={dialogOptions.localProviderNames}
toolProfileAliases={dialogOptions.toolProfileAliases}
datetimeOptions={dialogOptions.datetimeOptions}
onUpdate={updateConfig}

10
studio/frontend/src/features/recipe-studio/stores/helpers/reference-sync.ts
View file

@ -137,7 +137,15 @@ export function applyRemovalToConfig(
}
if (config.kind === "model_config" && config.provider === ref) {
const base = next as ModelConfig;
next = { ...base, provider: "" };
// Clear the synthetic "local" placeholder when the provider that was
// a local provider is removed; otherwise the stale placeholder would
// pass validation against a future external provider and then fail
// at runtime against a real API ("model not found").
next = {
...base,
provider: "",
model: base.model === "local" ? "" : base.model,
};
}
if (config.kind === "llm" && config.model_alias === ref) {
const base = next as LlmConfig;

28
studio/frontend/src/features/recipe-studio/stores/recipe-studio.ts
View file

@ -736,6 +736,34 @@ export const useRecipeStudioStore = create<RecipeStudioState>((set, get) => ({
configs = applyRenameToConfigs(configs, oldName, newName);
}
// When a provider toggles between local and external, keep already
// linked model_config nodes in sync. applyRenameToConfigs above has
// already propagated any name change, so providerName here is the
// post-rename value.
if (current.kind === "model_provider" && next.kind === "model_provider") {
const prevIsLocal = current.is_local === true;
const nextIsLocal = next.is_local === true;
if (prevIsLocal !== nextIsLocal) {
const providerName = next.name;
for (const [cfgId, cfg] of Object.entries(configs)) {
if (cfg.kind !== "model_config" || cfg.provider !== providerName) {
continue;
}
if (nextIsLocal && !cfg.model.trim()) {
// external -> local: auto fill the placeholder model id so the
// config does not fail "model is required" validation.
configs = { ...configs, [cfgId]: { ...cfg, model: "local" } };
continue;
}
if (!nextIsLocal && cfg.model === "local") {
// local -> external: clear the placeholder so the user picks a
// real model id for the new external endpoint.
configs = { ...configs, [cfgId]: { ...cfg, model: "" } };
}
}
}
}
return { configs, nodes, edges };
};
set(applyUpdate);

2
studio/frontend/src/features/recipe-studio/types/index.ts
View file

@ -252,6 +252,8 @@ export type ModelProviderConfig = {
extra_headers?: string;
// biome-ignore lint/style/useNamingConvention: api schema
extra_body?: string;
// biome-ignore lint/style/useNamingConvention: api schema
is_local?: boolean;
};
export type ModelConfig = {

2
studio/frontend/src/features/recipe-studio/utils/config-factories.ts
View file

@ -240,6 +240,8 @@ export function makeModelProviderConfig(
extra_headers: "",
// biome-ignore lint/style/useNamingConvention: api schema
extra_body: "",
// biome-ignore lint/style/useNamingConvention: api schema
is_local: false,
};
}

12
studio/frontend/src/features/recipe-studio/utils/graph/recipe-graph-connection.ts
View file

@ -386,7 +386,17 @@ export function applyRecipeConnection(
nextBaseEdges,
);
if (source.kind === "model_provider" && target.kind === "model_config") {
const next = { ...target, provider: source.name };
// Keep the model_config.model field in sync with provider mode when the
// link is changed via graph drag (the model-config dialog path has its
// own applyProviderChange helper that does the same thing).
const isSourceLocal = source.is_local === true;
let nextModel = target.model;
if (isSourceLocal && !nextModel.trim()) {
nextModel = "local";
} else if (!isSourceLocal && nextModel === "local") {
nextModel = "";
}
const next = { ...target, provider: source.name, model: nextModel };
return { edges: nextEdges, configs: { ...configs, [target.id]: next } };
}
if (source.kind === "model_config" && target.kind === "llm") {

2
studio/frontend/src/features/recipe-studio/utils/import/parsers/model-parser.ts
View file

@ -35,6 +35,8 @@ export function parseModelProvider(
extra_body: isRecord(provider.extra_body)
? JSON.stringify(provider.extra_body, null, 2)
: "",
// biome-ignore lint/style/useNamingConvention: api schema
is_local: provider.is_local === true,
};
}

5
studio/frontend/src/features/recipe-studio/utils/payload/build-payload.ts
View file

@ -104,6 +104,7 @@ export function buildRecipePayload(
const columns: Record<string, unknown>[] = [];
const modelAliases = new Set<string>();
const modelProviderNames = new Set<string>();
const localProviderNames = new Set<string>();
const modelProviders: Record<string, unknown>[] = [];
const mcpProviders: Record<string, unknown>[] = [];
const modelConfigs: Record<string, unknown>[] = [];
@ -200,6 +201,9 @@ export function buildRecipePayload(
}
if (config.kind === "model_provider") {
modelProviderNames.add(config.name);
if (config.is_local) {
localProviderNames.add(config.name);
}
modelProviders.push(buildModelProvider(config, errors));
modelProviderConfigs.push(config);
continue;
@ -240,6 +244,7 @@ export function buildRecipePayload(
modelConfigConfigs,
modelAliases,
modelProviderNames,
localProviderNames,
errors,
);
validateUsedProviders(modelProviderConfigs, modelConfigConfigs, errors);

20
studio/frontend/src/features/recipe-studio/utils/payload/builders-model.ts
View file

@ -8,6 +8,26 @@ export function buildModelProvider(
config: ModelProviderConfig,
errors: string[],
): Record<string, unknown> {
// Local providers do not use any of the advanced request overrides -
// the backend overrides endpoint/api_key/provider_type and strips the
// extra fields in _inject_local_providers. Skip parsing the hidden
// JSON inputs here so imported or hydrated recipes with stale extra
// headers/body cannot block the client-side validation step.
if (config.is_local === true) {
return {
name: config.name,
endpoint: "",
// biome-ignore lint/style/useNamingConvention: api schema
provider_type: "openai",
// biome-ignore lint/style/useNamingConvention: api schema
extra_headers: {},
// biome-ignore lint/style/useNamingConvention: api schema
extra_body: {},
// biome-ignore lint/style/useNamingConvention: api schema
is_local: true,
};
}
const extraHeaders = parseJsonObject(
config.extra_headers,
`Provider ${config.name} extra_headers`,

9
studio/frontend/src/features/recipe-studio/utils/payload/validate.ts
View file

@ -84,12 +84,16 @@ export function validateModelConfigProviders(
modelConfigConfigs: ModelConfig[],
modelAliases: Set<string>,
modelProviderNames: Set<string>,
localProviderNames: Set<string>,
errors: string[],
): void {
for (const config of modelConfigConfigs) {
const provider = config.provider.trim();
const alias = config.name;
if (modelAliases.has(alias) && !config.model.trim()) {
const isLocal = localProviderNames.has(provider);
// Local providers do not require a real model id - the loaded Chat
// model is used regardless of what gets sent in the payload.
if (!isLocal && modelAliases.has(alias) && !config.model.trim()) {
errors.push(`Model config ${alias}: model is required.`);
}
if (provider && !modelProviderNames.has(provider)) {
@ -110,6 +114,9 @@ export function validateUsedProviders(
if (!usedProviders.has(provider.name)) {
continue;
}
if (provider.is_local) {
continue;
}
if (!provider.endpoint.trim()) {
errors.push(`Model provider ${provider.name}: endpoint is required.`);
}

6
studio/frontend/src/features/recipe-studio/utils/recipe-studio-view.ts
View file

@ -7,6 +7,7 @@ export type DialogOptions = {
categoryOptions: SamplerConfig[];
modelConfigAliases: string[];
modelProviderOptions: string[];
localProviderNames: Set<string>;
toolProfileAliases: string[];
datetimeOptions: string[];
};
@ -15,6 +16,7 @@ export function buildDialogOptions(configList: NodeConfig[]): DialogOptions {
const categoryOptions: SamplerConfig[] = [];
const modelConfigAliases: string[] = [];
const modelProviderOptions: string[] = [];
const localProviderNames = new Set<string>();
const toolProfileAliases: string[] = [];
const datetimeOptions: string[] = [];
@ -34,6 +36,9 @@ export function buildDialogOptions(configList: NodeConfig[]): DialogOptions {
}
if (config.kind === "model_provider") {
modelProviderOptions.push(config.name);
if (config.is_local) {
localProviderNames.add(config.name);
}
continue;
}
if (config.kind === "tool_config") {
@ -45,6 +50,7 @@ export function buildDialogOptions(configList: NodeConfig[]): DialogOptions {
categoryOptions,
modelConfigAliases,
modelProviderOptions,
localProviderNames,
toolProfileAliases,
datetimeOptions,
};