2 Commits

Author	SHA1	Message	Date
Patrick Buckley	32fd8f29c7	feat(providers): api_surface toggle + mistral medium reasoning fix (#469) ... * feat(providers): api_surface toggle + mistral medium reasoning fix Mistral medium open-weights served by vLLM expects reasoning_effort via the Responses API (`reasoning.effort`), not as a `chat_template_kwargs` entry on Chat Completions. The session was unconditionally injecting `{"reasoning_effort": ...}` into `chat_template_kwargs` for every openai-compatible request, which corrupted the prompt rendering for any backend whose chat template didn't consume that key (Mistral medium, Mistral cloud, Groq, OpenRouter). Changes: - Add `api_surface` ("chat" | "responses") to `ModelConfig.server_compat` and thread it through `create_provider` / `model_registry.get_provider`. `openai-compatible` defaults to Chat Completions; operators can flip individual aliases to Responses for endpoints that support it. - New `vllm-mistral-medium` profile that pre-fills api_surface=responses on Detect for known Mistral medium model ids. - Drop the unconditional `reasoning_effort` injection into `chat_template_kwargs`. Operators running gpt-oss-style local templates that consume `reasoning_effort` from the chat template now opt in via `server_compat.extra_body.chat_template_kwargs`. - New "API Surface" select in the Models admin tab; allowlist-validated server-side at create/update time; pre-filled by Detect via the profile suggestion. - Evict the cached provider singleton in `ModelRegistry.reload()` when api_surface changes (previously only cfg.provider triggered eviction). - Fix `_run_agent` fallback path to inherit the session's primary alias for capability and server_compat resolution; previously the fallback passed `alias=None`, which silently dropped per-model caps on the agent path. Tests: 5117 passed (-m "not live"); ruff + mypy clean. * fix(providers): don't auto-suggest Responses for Mistral medium vLLM's Responses API surface for Mistral medium open-weights doesn't wire up the Mistral tool-call parser as of vLLM 0.x — tool calls leak into the response as ``[TOOL_CALLS]<name>{...}`` text instead of structured tool_calls. Chat Completions on the same engine handles tools cleanly via ``--tool-call-parser mistral``, and reasoning can be turned on via the vLLM CLI ``--reasoning-parser`` flag. Drop the auto-suggest mapping so Detect falls back to the generic ``vllm`` profile. Keep the ``vllm-mistral-medium`` profile definition in place so an operator who specifically wants per-request effort and accepts the tool-calling limitation can still pick "Responses API" manually in the admin UI. * fix(providers): address Copilot review on PR #469 - providers/__init__.py: drop the redundant *_responses_provider / *_chat_provider names; have create_provider use _openai_provider and _openai_compat_provider directly so they're not flagged as unused globals. - console/server.py: tighten _validate_api_surface to a strict equality match against the canonical {"chat", "responses"} set. The previous strip().lower() membership check accepted ' Responses '/'CHAT' but stored the raw string verbatim, which then failed to round-trip through the admin <select>. - console/static/admin.js: gate the entire server_compat block (server type, api_surface, extra_body) on provider == "openai-compatible" at save time so toggling provider away can't leave a stale hidden surface selection in the persisted capabilities JSON. - tests/test_session.py: splat the bad kwarg via **dict so CodeQL no longer flags the call as a wrong-name keyword (the point of the test is the runtime contract, not the static type). - tests/test_admin_model_registry_refresh.py: add endpoint-level tests for the api_surface validation on both create and update — covers the bogus-value rejection, non-canonical-string rejection, and the happy path persisting through to the refreshed registry.	2026-05-03 13:37:50 -07:00
Patrick Buckley	eb59cdefda	feat: pass resolved capabilities through to providers, add server com… (#352) ... * feat: pass resolved capabilities through to providers, add server compat layer The LLMProvider protocol previously forced providers to re-derive capabilities from static lookup tables, ignoring config overrides set via the admin UI or config.toml (e.g. thinking_mode, token_param). This adds an optional capabilities parameter to create_streaming and create_completion so the session can pass its config-merged ModelCapabilities through to providers. On top of this, adds a server compatibility layer for local model servers (vLLM, llama.cpp). Profiles suggest thinking mode and server workarounds (skip_special_tokens for vLLM, reasoning_format for llama.cpp) during model detection, with structured admin UI fields for server type, thinking mode, and extra body params. Verified against real vLLM (Gemma 4 31B) and llama.cpp (Gemma 4 E4B) servers. * fix: defensive copy in _finalize_extra_body, expose thinking_param in UI Shallow-copy extra_params and its chat_template_kwargs in the provider before _apply_thinking_mode mutates them, so callers that reuse the same dict across models are safe. Replace the hidden thinking_param input with a visible text field that appears when thinking mode is enabled. Shows the default "enable_thinking" and hints that Granite/DeepSeek use "thinking". * fix: address Copilot review feedback on admin UI and server compat - Preserve unrepresentable thinking_mode values (e.g. "adaptive") in raw capabilities JSON instead of silently dropping on edit round-trip - Validate capabilities and extra body JSON are plain objects, not arrays or primitives - Deep-merge chat_template_kwargs from extra_body instead of silently dropping, so operators can extend/override template kwargs * fix: hide server compat section for non-local providers The Server Compatibility fields (server type, thinking mode, extra body) only apply to openai-compatible (local model servers). Hide the entire section when the provider is openai, anthropic, or google. * fix: normalize capsObj to plain object on edit load Defend against DB rows where capabilities is a JSON literal null, an array, or a primitive — previous code would crash on the capsObj.server_compat / capsObj.thinking_mode reads. Same defensive check also applied to the server_compat nested value. * refactor: extract _isPlainObject helper for JSON type checks Consolidates the null/array/typeof check that was inlined at three different call sites into a single helper. Keeps the intent obvious at each use site and avoids the awkward multi-condition ternary.	2026-04-14 11:05:51 -07:00