mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
feat: render rich media embeds for MCP tool results (#292)
* feat: render rich media embeds for MCP tool results Detect structured media JSON (stream_url, results, sessions) in MCP tool output and render interactive cards instead of plain text. Web UI: media cards with thumbnail, title, metadata, and click-to-play video/audio. HLS via lazy-loaded hls.js with direct-stream preference. Collapsed raw JSON (API keys redacted) for inspection. Discord: rich embeds with proxied thumbnail images (fetched by the bot since Discord CDN cannot reach private media servers). Search results as numbered lists, session state as "Now Playing" cards. Stream URLs never exposed in embeds — web_url used for safe clickable links. CI: vendor hls.js 1.6.15 with renovate tracking and update script. * fix: address PR #292 review — SSRF guards, streaming fetch, tests - URL validation: reject non-http(s) schemes and userinfo in thumbnail URLs. Private IPs intentionally allowed (media servers are on LAN). - Streaming fetch: use http.stream() with aiter_bytes() and a running byte count to enforce the 2MB cap without buffering the full response. Validate content-type is image/* before downloading. - Resilience: wrap try_build_media_embed in try/except in bot.py so a media embed failure falls through to the code-block path. - LICENSE: download hls.js LICENSE from npm on update instead of only copying from old dir. - Tests: add 19 new tests — try_parse_media (8 cases), _is_safe_image_url (7 cases), embed builders (4 cases including stream_url exclusion and string season/episode safety). * chore: add LICENSE file for vendored hls.js * fix: remove ANSI escape codes from tool preview fields Preview text (tool args, URLs, queries) was wrapped in DIM/RESET ANSI codes at the source in session.py, which leaked into SSE events and rendered as raw escape sequences in Discord and the web UI. Move ANSI styling to the CLI consumer (cli.py) where it belongs. Also escape markdown in Discord tool name titles to prevent __ from being interpreted as underline formatting. * fix: drop [MCP: server] prefix from tool descriptions The prefix made MCP tools look second-class compared to builtins, causing models to hesitate using them. The server name is already encoded in the tool name (mcp__server__tool). * feat: pretty-print JSON tool output, player error state, broader key redaction - JSON tool results are detected and pretty-printed with 2-space indent instead of rendering as a wall of text - API key redaction extended to cover api_key, apiKey, api-key, and token query params across all tool output (not just media embeds) - Video/audio player shows styled error message when stream fails to load instead of leaving a broken player element - Both appendToolOutput and replayHistory use shared renderToolOutput() * fix: designer review — player error retry, contrast, tool-cmd cap - Player error: role="alert" for screen readers, retry button that reuses existing play handler, includes media title in error message - Light theme: darken --red from #dc2626 to #b91c1c (5.7:1 contrast on --code-bg, was 4.3:1 failing WCAG AA at 12px) - Pretty-print collapsed raw JSON in media embeds (was missed earlier) - Cap .tool-cmd at 120px to prevent tools with many args from making approval blocks disproportionately tall in history replay - Dedicated .media-player-error class instead of reusing .tool-output * fix: Discord tool info name matching regression, suppress deprecation warning The escape_markdown call on tool names was stored for matching against ToolResultEvent.name, but event.name is raw/unescaped. The escaped name never matched, so the "Running → Done" transition silently failed and previews disappeared from the status embed. Fix: store raw name for matching, use escaped name only for display. Also suppress discord.py's re.sub count deprecation warning (Python 3.13+ issue, fixed upstream). * fix: update MCP tool description tests to match prefix removal * fix: address PR #292 review round 2 - Retry button: handle missing span children in click handler so retry buttons from player error state don't throw - Footer count: use len(lines) instead of min(len(results), 10) to reflect actual rendered count after char budget truncation - Null display: use "null" instead of "None" in JS tool arg preview - Broader redaction: also redact JSON "api_key": "..." patterns - SSRF hardening: block loopback and link-local IPs plus cloud metadata hostnames in thumbnail fetch (private LAN IPs still allowed)
This commit is contained in:
Patrick Buckley
@@ -41,6 +41,14 @@
|
||||
"matchStrings": ["mermaid-(?<currentValue>[\\d.]+)/"],
|
||||
"depNameTemplate": "mermaid",
|
||||
"datasourceTemplate": "npm"
|
||||
},
|
||||
{
|
||||
"customType": "regex",
|
||||
"description": "Track vendored hls.js version",
|
||||
"managerFilePatterns": ["/pyproject\\.toml$/"],
|
||||
"matchStrings": ["hls-(?<currentValue>[\\d.]+)/"],
|
||||
"depNameTemplate": "hls.js",
|
||||
"datasourceTemplate": "npm"
|
||||
}
|
||||
],
|
||||
"packageRules": [
|
||||
@@ -91,7 +99,7 @@
|
||||
{
|
||||
"description": "Vendored JS — CI workflow downloads files automatically",
|
||||
"groupName": "Vendored JS",
|
||||
"matchPackageNames": ["katex", "highlight.js", "mermaid"],
|
||||
"matchPackageNames": ["katex", "highlight.js", "mermaid", "hls.js"],
|
||||
"schedule": ["before 9am on the first day of the month"],
|
||||
"automerge": false
|
||||
},
|
||||
|
||||
@@ -95,3 +95,22 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
================================================================================
|
||||
|
||||
hls.js 1.6.15
|
||||
https://github.com/video-dev/hls.js
|
||||
|
||||
Copyright 2017 Dailymotion
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
@@ -80,6 +80,7 @@ include = [
|
||||
"turnstone/shared_static/katex-0.16.44/**/*",
|
||||
"turnstone/shared_static/hljs-11.11.1/**/*",
|
||||
"turnstone/shared_static/mermaid-11.14.0/**/*",
|
||||
"turnstone/shared_static/hls-1.6.15/**/*",
|
||||
"turnstone/sdk/py.typed",
|
||||
"turnstone/deploy/*.yaml",
|
||||
]
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
# scripts/update-vendored-js.sh katex 0.16.39
|
||||
# scripts/update-vendored-js.sh hljs 11.12.0
|
||||
# scripts/update-vendored-js.sh mermaid 11.14.0
|
||||
# scripts/update-vendored-js.sh hls 1.6.15
|
||||
#
|
||||
# This script:
|
||||
# 1. Downloads the new version from CDN
|
||||
@@ -18,7 +19,7 @@ STATIC_DIR="turnstone/shared_static"
|
||||
CDN="https://cdn.jsdelivr.net/npm"
|
||||
|
||||
usage() {
|
||||
echo "Usage: $0 <katex|hljs|mermaid> <version>"
|
||||
echo "Usage: $0 <katex|hljs|mermaid|hls> <version>"
|
||||
echo "Example: $0 katex 0.16.39"
|
||||
exit 1
|
||||
}
|
||||
@@ -147,6 +148,32 @@ case "$LIB" in
|
||||
echo "Done. Old directory removed: ${OLD_DIR}"
|
||||
;;
|
||||
|
||||
hls)
|
||||
OLD_VERSION=$(detect_old_version "hls")
|
||||
check_same_version "$OLD_VERSION" "$VERSION" "hls"
|
||||
OLD_DIR="${STATIC_DIR}/hls-${OLD_VERSION}"
|
||||
NEW_DIR="${STATIC_DIR}/hls-${VERSION}"
|
||||
|
||||
echo "Updating hls.js ${OLD_VERSION} -> ${VERSION}"
|
||||
mkdir -p "${NEW_DIR}"
|
||||
|
||||
echo " Downloading hls.min.js..."
|
||||
curl -sSfL "${CDN}/hls.js@${VERSION}/dist/hls.min.js" -o "${NEW_DIR}/hls.min.js"
|
||||
|
||||
echo " Downloading LICENSE..."
|
||||
if ! curl -sSfL "${CDN}/hls.js@${VERSION}/LICENSE" -o "${NEW_DIR}/LICENSE" 2>/dev/null; then
|
||||
if [[ -f "${OLD_DIR}/LICENSE" ]]; then
|
||||
cp "${OLD_DIR}/LICENSE" "${NEW_DIR}/LICENSE"
|
||||
else
|
||||
echo " WARNING: Could not obtain LICENSE for hls.js ${VERSION}"
|
||||
fi
|
||||
fi
|
||||
|
||||
update_refs "hls-${OLD_VERSION}" "hls-${VERSION}"
|
||||
rm -rf "${OLD_DIR}"
|
||||
echo "Done. Old directory removed: ${OLD_DIR}"
|
||||
;;
|
||||
|
||||
*)
|
||||
echo "Unknown library: ${LIB}"
|
||||
usage
|
||||
|
||||
@@ -8,6 +8,13 @@ from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
# discord.utils.escape_markdown passes 'count' as positional to re.sub,
|
||||
# which is deprecated in Python 3.13+. This is a discord.py bug (fixed
|
||||
# in newer releases); suppress here to keep the test output clean.
|
||||
pytestmark = pytest.mark.filterwarnings(
|
||||
"ignore:.*'count' is passed as positional argument:DeprecationWarning"
|
||||
)
|
||||
|
||||
discord = pytest.importorskip("discord")
|
||||
|
||||
|
||||
@@ -886,6 +893,192 @@ class TestFormatToolResult:
|
||||
assert result.count("```") == 2
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Media embed detection and rendering
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestTryParseMedia:
|
||||
"""Tests for try_parse_media in _formatter.py."""
|
||||
|
||||
def test_stream_url_detected(self):
|
||||
import json
|
||||
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
data = json.dumps({"stream_url": "http://jf:8096/Videos/abc/stream", "container": "mp4"})
|
||||
result = try_parse_media(data)
|
||||
assert result is not None
|
||||
assert result["stream_url"] == "http://jf:8096/Videos/abc/stream"
|
||||
|
||||
def test_media_details_detected(self):
|
||||
import json
|
||||
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
data = json.dumps({"id": "abc", "name": "Test Movie", "type": "Movie", "year": 2024})
|
||||
result = try_parse_media(data)
|
||||
assert result is not None
|
||||
assert result["name"] == "Test Movie"
|
||||
|
||||
def test_search_results_detected(self):
|
||||
import json
|
||||
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
data = json.dumps({"results": [{"id": "1", "name": "Hit"}], "total_count": 1})
|
||||
result = try_parse_media(data)
|
||||
assert result is not None
|
||||
assert len(result["results"]) == 1
|
||||
|
||||
def test_sessions_detected(self):
|
||||
import json
|
||||
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
data = json.dumps({"sessions": [{"id": "s1", "user_name": "ptrck"}]})
|
||||
result = try_parse_media(data)
|
||||
assert result is not None
|
||||
|
||||
def test_empty_results_returns_none(self):
|
||||
import json
|
||||
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
assert try_parse_media(json.dumps({"results": []})) is None
|
||||
|
||||
def test_plain_text_returns_none(self):
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
assert try_parse_media("just a string") is None
|
||||
|
||||
def test_non_dict_json_returns_none(self):
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
assert try_parse_media("[1, 2, 3]") is None
|
||||
|
||||
def test_unrelated_dict_returns_none(self):
|
||||
import json
|
||||
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
assert try_parse_media(json.dumps({"foo": "bar"})) is None
|
||||
|
||||
|
||||
class TestIsSafeImageUrl:
|
||||
"""Tests for _is_safe_image_url in _formatter.py."""
|
||||
|
||||
def test_http_url(self):
|
||||
from turnstone.channels._formatter import _is_safe_image_url
|
||||
|
||||
assert _is_safe_image_url("http://jellyfin:8096/Items/abc/Images/Primary") is True
|
||||
|
||||
def test_https_url(self):
|
||||
from turnstone.channels._formatter import _is_safe_image_url
|
||||
|
||||
assert _is_safe_image_url("https://jellyfin.example.com/Items/abc/Images/Primary") is True
|
||||
|
||||
def test_ftp_rejected(self):
|
||||
from turnstone.channels._formatter import _is_safe_image_url
|
||||
|
||||
assert _is_safe_image_url("ftp://evil.com/image.jpg") is False
|
||||
|
||||
def test_file_rejected(self):
|
||||
from turnstone.channels._formatter import _is_safe_image_url
|
||||
|
||||
assert _is_safe_image_url("file:///etc/passwd") is False
|
||||
|
||||
def test_userinfo_rejected(self):
|
||||
from turnstone.channels._formatter import _is_safe_image_url
|
||||
|
||||
assert _is_safe_image_url("http://user:pass@jellyfin:8096/image") is False
|
||||
|
||||
def test_empty_rejected(self):
|
||||
from turnstone.channels._formatter import _is_safe_image_url
|
||||
|
||||
assert _is_safe_image_url("") is False
|
||||
|
||||
def test_private_ip_allowed(self):
|
||||
from turnstone.channels._formatter import _is_safe_image_url
|
||||
|
||||
assert _is_safe_image_url("http://192.168.0.6:8096/Items/abc/Images/Primary") is True
|
||||
|
||||
|
||||
class TestBuildMediaEmbed:
|
||||
"""Tests for try_build_media_embed and embed builders."""
|
||||
|
||||
def test_single_item_embed_uses_web_url_not_stream_url(self):
|
||||
import json
|
||||
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
data = {
|
||||
"name": "Test Movie",
|
||||
"type": "Movie",
|
||||
"year": 2024,
|
||||
"stream_url": "http://jf:8096/Videos/abc/stream?api_key=SECRET",
|
||||
"web_url": "http://jf:8096/web/#/details?id=abc",
|
||||
"overview": "A test movie.",
|
||||
}
|
||||
parsed = try_parse_media(json.dumps(data))
|
||||
assert parsed is not None
|
||||
|
||||
from turnstone.channels._formatter import _build_single_media_embed
|
||||
|
||||
embed = _build_single_media_embed(parsed, "mcp__mediamcp__get_stream_url")
|
||||
# web_url should be the embed URL, never stream_url
|
||||
assert embed.url == "http://jf:8096/web/#/details?id=abc"
|
||||
assert "SECRET" not in str(embed.to_dict())
|
||||
|
||||
def test_search_results_embed_format(self):
|
||||
import json
|
||||
|
||||
from turnstone.channels._formatter import try_parse_media
|
||||
|
||||
data = {
|
||||
"results": [
|
||||
{"name": "Movie A", "year": 2020, "type": "Movie", "runtime_minutes": 120},
|
||||
{"name": "Movie B", "year": 2021, "type": "Movie"},
|
||||
],
|
||||
"total_count": 2,
|
||||
}
|
||||
parsed = try_parse_media(json.dumps(data))
|
||||
|
||||
from turnstone.channels._formatter import _build_search_results_embed
|
||||
|
||||
embed = _build_search_results_embed(parsed)
|
||||
assert "Movie A" in embed.description
|
||||
assert "Movie B" in embed.description
|
||||
assert "2 of 2" in embed.footer.text
|
||||
|
||||
def test_build_media_embed_returns_none_for_plain_text(self):
|
||||
from turnstone.channels._formatter import try_build_media_embed
|
||||
|
||||
http = MagicMock()
|
||||
result = _run(try_build_media_embed("tool", "plain text", http=http))
|
||||
assert result is None
|
||||
|
||||
def test_season_episode_string_values(self):
|
||||
"""Season/episode numbers as strings should not raise."""
|
||||
|
||||
from turnstone.channels._formatter import _build_search_results_embed
|
||||
|
||||
data = {
|
||||
"results": [
|
||||
{
|
||||
"name": "Pilot",
|
||||
"type": "Episode",
|
||||
"series_name": "Show",
|
||||
"season_number": "1",
|
||||
"episode_number": "1",
|
||||
},
|
||||
],
|
||||
"total_count": 1,
|
||||
}
|
||||
embed = _build_search_results_embed(data)
|
||||
assert "S01E01" in embed.description
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Thinking indicator lifecycle
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -1103,6 +1296,7 @@ class TestToolResultEvent:
|
||||
bot._tool_info_msgs = {}
|
||||
bot._pending_approval_msgs = {}
|
||||
bot._notify_reply_channels = {}
|
||||
bot._http_client = MagicMock()
|
||||
bot._should_auto_approve = MagicMock(return_value=False)
|
||||
bot._on_ws_event = TurnstoneBot._on_ws_event.__get__(bot, TurnstoneBot)
|
||||
return bot
|
||||
|
||||
@@ -141,7 +141,7 @@ class TestMcpToOpenai:
|
||||
assert result["type"] == "function"
|
||||
func = result["function"]
|
||||
assert func["name"] == "mcp__github__search_repos"
|
||||
assert "[MCP: github]" in func["description"]
|
||||
assert func["description"] == "Search GitHub repos"
|
||||
assert func["parameters"]["type"] == "object"
|
||||
assert "query" in func["parameters"]["properties"]
|
||||
|
||||
@@ -164,7 +164,7 @@ class TestMcpToOpenai:
|
||||
tool.description = ""
|
||||
tool.inputSchema = {"type": "object", "properties": {}}
|
||||
result = _mcp_to_openai("test", tool)
|
||||
assert result["function"]["description"] == "[MCP: test] "
|
||||
assert result["function"]["description"] == ""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -1,12 +1,17 @@
|
||||
"""Message formatting utilities for channel adapters.
|
||||
|
||||
Handles chunking long messages for platforms with character limits, formatting
|
||||
tool-approval requests, and plan-review prompts.
|
||||
tool-approval requests, plan-review prompts, and rich media embeds for
|
||||
platforms that support them (e.g. Discord).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
import json
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
import httpx
|
||||
|
||||
|
||||
def chunk_message(text: str, max_length: int = 2000) -> list[str]:
|
||||
@@ -164,3 +169,298 @@ def truncate(text: str, max_length: int = 200) -> str:
|
||||
if len(text) <= max_length:
|
||||
return text
|
||||
return text[: max_length - 1] + "\u2026"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Rich media embed helpers (Discord)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def try_parse_media(output: str) -> dict[str, Any] | None:
|
||||
"""Attempt to parse tool output as a media result.
|
||||
|
||||
Returns the parsed dict when the output looks like structured media
|
||||
(single item, search results, or session list), otherwise ``None``.
|
||||
"""
|
||||
try:
|
||||
data = json.loads(output)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None
|
||||
if not isinstance(data, dict):
|
||||
return None
|
||||
# Single item with stream URL or detailed metadata.
|
||||
if "stream_url" in data or ("name" in data and "type" in data and "id" in data):
|
||||
return data
|
||||
# Search results.
|
||||
if "results" in data and isinstance(data["results"], list) and data["results"]:
|
||||
return data
|
||||
# Active sessions.
|
||||
if "sessions" in data and isinstance(data["sessions"], list):
|
||||
return data
|
||||
return None
|
||||
|
||||
|
||||
_BLOCKED_HOSTNAMES = frozenset({"localhost", "metadata.google.internal"})
|
||||
|
||||
|
||||
def _is_safe_image_url(url: str) -> bool:
|
||||
"""Validate that *url* uses http(s), has no embedded credentials, and does
|
||||
not target loopback or cloud metadata endpoints.
|
||||
|
||||
Private/LAN IPs are intentionally allowed (media servers are typically
|
||||
on the local network).
|
||||
"""
|
||||
import ipaddress
|
||||
from urllib.parse import urlparse
|
||||
|
||||
try:
|
||||
parsed = urlparse(url)
|
||||
except Exception: # noqa: BLE001
|
||||
return False
|
||||
if parsed.scheme not in ("http", "https"):
|
||||
return False
|
||||
if parsed.username or parsed.password:
|
||||
return False
|
||||
hostname = parsed.hostname
|
||||
if not hostname:
|
||||
return False
|
||||
if hostname in _BLOCKED_HOSTNAMES:
|
||||
return False
|
||||
try:
|
||||
ip = ipaddress.ip_address(hostname)
|
||||
if ip.is_loopback or ip.is_link_local:
|
||||
return False
|
||||
except ValueError:
|
||||
pass # Not an IP literal — hostname is fine
|
||||
return True
|
||||
|
||||
|
||||
async def _fetch_thumbnail(
|
||||
http: httpx.AsyncClient,
|
||||
url: str,
|
||||
*,
|
||||
timeout: float = 5.0,
|
||||
max_bytes: int = 2 * 1024 * 1024,
|
||||
) -> tuple[bytes, str] | None:
|
||||
"""Fetch a thumbnail image, returning ``(bytes, filename)`` or ``None``.
|
||||
|
||||
Never raises — a failed image fetch must not break tool result
|
||||
rendering. Private/LAN URLs are intentionally allowed (media servers
|
||||
are typically on the local network), but scheme is restricted to
|
||||
http(s) and userinfo is rejected.
|
||||
"""
|
||||
if not _is_safe_image_url(url):
|
||||
return None
|
||||
try:
|
||||
async with http.stream("GET", url, timeout=timeout) as resp:
|
||||
if resp.status_code != 200:
|
||||
return None
|
||||
cl = resp.headers.get("content-length")
|
||||
if cl and cl.isdigit() and int(cl) > max_bytes:
|
||||
return None
|
||||
content_type = resp.headers.get("content-type", "image/jpeg").lower()
|
||||
if not content_type.startswith("image/"):
|
||||
return None
|
||||
ext = "jpg"
|
||||
if "png" in content_type:
|
||||
ext = "png"
|
||||
elif "webp" in content_type:
|
||||
ext = "webp"
|
||||
data = bytearray()
|
||||
async for chunk in resp.aiter_bytes():
|
||||
data.extend(chunk)
|
||||
if len(data) > max_bytes:
|
||||
return None
|
||||
return bytes(data), f"poster.{ext}"
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
|
||||
|
||||
async def try_build_media_embed(
|
||||
tool_name: str,
|
||||
output: str,
|
||||
*,
|
||||
http: httpx.AsyncClient,
|
||||
) -> tuple[Any, Any | None] | None:
|
||||
"""Attempt to build a rich Discord embed from media tool output.
|
||||
|
||||
Returns ``(embed, optional_file)`` if the output is parseable as media,
|
||||
or ``None`` to fall through to the default code-block formatter.
|
||||
|
||||
The ``discord`` library is imported lazily since this module is shared
|
||||
across adapters and ``discord.py`` is an optional dependency.
|
||||
"""
|
||||
data = try_parse_media(output)
|
||||
if data is None:
|
||||
return None
|
||||
|
||||
import io
|
||||
|
||||
import discord
|
||||
|
||||
# Dispatch on result shape.
|
||||
if "results" in data and isinstance(data["results"], list):
|
||||
embed = _build_search_results_embed(data)
|
||||
elif "sessions" in data and isinstance(data["sessions"], list):
|
||||
embed = _build_sessions_embed(data)
|
||||
else:
|
||||
embed = _build_single_media_embed(data, tool_name)
|
||||
|
||||
# Proxy thumbnail image.
|
||||
thumbnail_url = data.get("thumbnail_url") or data.get("image_url")
|
||||
if not thumbnail_url and data.get("results"):
|
||||
first = data["results"][0]
|
||||
thumbnail_url = first.get("thumbnail_url") or first.get("image_url")
|
||||
|
||||
file: discord.File | None = None
|
||||
if thumbnail_url:
|
||||
fetched = await _fetch_thumbnail(http, thumbnail_url)
|
||||
if fetched:
|
||||
image_bytes, filename = fetched
|
||||
file = discord.File(io.BytesIO(image_bytes), filename=filename)
|
||||
embed.set_thumbnail(url=f"attachment://{filename}")
|
||||
|
||||
return embed, file
|
||||
|
||||
|
||||
# -- Private embed builders ------------------------------------------------
|
||||
|
||||
|
||||
def _build_single_media_embed(data: dict[str, Any], tool_name: str) -> Any:
|
||||
"""Build a Discord embed for a single media item."""
|
||||
import discord
|
||||
|
||||
title = data.get("name", "Unknown")
|
||||
if data.get("year"):
|
||||
title += f" ({data['year']})"
|
||||
|
||||
embed = discord.Embed(
|
||||
title=title,
|
||||
url=data.get("web_url"), # safe link — NOT stream_url
|
||||
description=truncate(data.get("overview", ""), 200),
|
||||
color=discord.Color.teal(),
|
||||
)
|
||||
|
||||
# Metadata fields (inline).
|
||||
meta_parts: list[str] = []
|
||||
if data.get("type"):
|
||||
meta_parts.append(data["type"])
|
||||
if data.get("official_rating"):
|
||||
meta_parts.append(data["official_rating"])
|
||||
if data.get("runtime_minutes"):
|
||||
hours = int(data["runtime_minutes"] // 60)
|
||||
mins = int(data["runtime_minutes"] % 60)
|
||||
meta_parts.append(f"{hours}h {mins}m" if hours else f"{mins}m")
|
||||
if meta_parts:
|
||||
embed.add_field(name="Info", value=" \u00b7 ".join(meta_parts), inline=True)
|
||||
|
||||
if data.get("genres"):
|
||||
embed.add_field(name="Genres", value=", ".join(data["genres"][:5]), inline=True)
|
||||
|
||||
if data.get("community_rating"):
|
||||
embed.add_field(
|
||||
name="Rating",
|
||||
value=f"{data['community_rating']:.1f}/10",
|
||||
inline=True,
|
||||
)
|
||||
|
||||
# Extract server name from tool_name (mcp__servername__toolname).
|
||||
parts = tool_name.split("__")
|
||||
if len(parts) >= 3:
|
||||
embed.set_footer(text=parts[1])
|
||||
|
||||
return embed
|
||||
|
||||
|
||||
def _build_search_results_embed(data: dict[str, Any]) -> Any:
|
||||
"""Build a Discord embed for a list of search results."""
|
||||
import discord
|
||||
|
||||
results = data.get("results", [])
|
||||
total = data.get("total_count", len(results))
|
||||
|
||||
lines: list[str] = []
|
||||
char_count = 0
|
||||
for i, r in enumerate(results[:10], 1):
|
||||
line = f"**{i}.** {r.get('name', '?')}"
|
||||
if r.get("year"):
|
||||
line += f" ({r['year']})"
|
||||
meta: list[str] = []
|
||||
if r.get("type"):
|
||||
meta.append(r["type"])
|
||||
if r.get("series_name"):
|
||||
meta.append(r["series_name"])
|
||||
if r.get("season_number") is not None and r.get("episode_number") is not None:
|
||||
meta.append(f"S{int(r['season_number']):02d}E{int(r['episode_number']):02d}")
|
||||
if r.get("runtime_minutes"):
|
||||
mins = r["runtime_minutes"]
|
||||
meta.append(f"{int(mins // 60)}h {int(mins % 60)}m" if mins >= 60 else f"{int(mins)}m")
|
||||
if meta:
|
||||
line += " \u00b7 " + " \u00b7 ".join(meta)
|
||||
if char_count + len(line) + 1 > 4000:
|
||||
break
|
||||
lines.append(line)
|
||||
char_count += len(line) + 1
|
||||
|
||||
embed = discord.Embed(
|
||||
title="Search results",
|
||||
description="\n".join(lines),
|
||||
color=discord.Color.teal(),
|
||||
)
|
||||
embed.set_footer(text=f"showing {len(lines)} of {total}")
|
||||
return embed
|
||||
|
||||
|
||||
def _build_sessions_embed(data: dict[str, Any]) -> Any:
|
||||
"""Build a Discord embed for active playback sessions."""
|
||||
import discord
|
||||
|
||||
sessions = data.get("sessions", [])
|
||||
if not sessions:
|
||||
embed = discord.Embed(
|
||||
title="Now Playing",
|
||||
description="No active sessions.",
|
||||
color=discord.Color.light_grey(),
|
||||
)
|
||||
return embed
|
||||
|
||||
lines: list[str] = []
|
||||
has_active = False
|
||||
for s in sessions:
|
||||
np = s.get("now_playing")
|
||||
device = s.get("device_name", "Unknown device")
|
||||
user = s.get("user_name", "")
|
||||
if np:
|
||||
has_active = True
|
||||
title = np.get("name", "Unknown")
|
||||
if np.get("year"):
|
||||
title += f" ({np['year']})"
|
||||
ps = s.get("play_state", {}) or {}
|
||||
pos = ps.get("position_seconds")
|
||||
runtime_min = np.get("runtime_minutes")
|
||||
time_str = ""
|
||||
if pos is not None and runtime_min:
|
||||
total_sec = int(runtime_min * 60)
|
||||
pos_i = int(pos)
|
||||
time_str = (
|
||||
f" {pos_i // 3600}:{pos_i % 3600 // 60:02d}:{pos_i % 60:02d}"
|
||||
f" / {total_sec // 3600}:{total_sec % 3600 // 60:02d}:{total_sec % 60:02d}"
|
||||
)
|
||||
paused = ps.get("is_paused", False)
|
||||
icon = "\u23f8" if paused else "\u25b6"
|
||||
line = f"**{title}** on {device}\n{icon}{time_str}"
|
||||
if user:
|
||||
line += f" \u00b7 {user}"
|
||||
lines.append(line)
|
||||
else:
|
||||
line = f"*{device}* \u2014 idle"
|
||||
if user:
|
||||
line += f" ({user})"
|
||||
lines.append(line)
|
||||
|
||||
embed = discord.Embed(
|
||||
title="Now Playing",
|
||||
description="\n\n".join(lines),
|
||||
color=discord.Color.green() if has_active else discord.Color.light_grey(),
|
||||
)
|
||||
return embed
|
||||
|
||||
@@ -16,7 +16,7 @@ import contextlib
|
||||
import json
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import httpx
|
||||
|
||||
@@ -558,15 +558,16 @@ class TurnstoneBot:
|
||||
# authorize this?" while the running embed says "this tool is
|
||||
# executing." Both can coexist in the thread.
|
||||
for it in event.items:
|
||||
name = it.get("func_name") or it.get("approval_label") or "tool"
|
||||
raw_name = it.get("func_name") or it.get("approval_label") or "tool"
|
||||
display_name = discord.utils.escape_markdown(raw_name)
|
||||
raw_preview = it.get("preview", "")
|
||||
# Sanitize preview: escape backticks to prevent markdown
|
||||
# breakout and strip @-mentions.
|
||||
# Escape backticks to prevent markdown breakout and
|
||||
# strip @-mentions.
|
||||
raw_preview = raw_preview.replace("`", "\\`")
|
||||
raw_preview = discord.utils.escape_mentions(raw_preview)
|
||||
preview = truncate(raw_preview, max_length=120) or None
|
||||
embed = discord.Embed(
|
||||
title=name,
|
||||
title=display_name,
|
||||
description=preview,
|
||||
color=discord.Color.light_grey(),
|
||||
)
|
||||
@@ -581,8 +582,9 @@ class TurnstoneBot:
|
||||
else:
|
||||
msg = await thread.send(embed=embed)
|
||||
call_id = it.get("call_id", "")
|
||||
# Store raw (unescaped) name for matching against ToolResultEvent.name
|
||||
self._tool_info_msgs.setdefault(ws_id, []).append(
|
||||
(call_id, name, preview or "", msg)
|
||||
(call_id, raw_name, preview or "", msg)
|
||||
)
|
||||
|
||||
# If no items consumed the thinking message (empty event), clean up.
|
||||
@@ -614,7 +616,7 @@ class TurnstoneBot:
|
||||
status = "Error" if event.is_error else "Done"
|
||||
status_color = discord.Color.red() if event.is_error else discord.Color.dark_grey()
|
||||
status_embed = discord.Embed(
|
||||
title=f"{event.name} \u2014 {status}",
|
||||
title=f"{discord.utils.escape_markdown(event.name)} \u2014 {status}",
|
||||
description=matched_preview or None,
|
||||
color=status_color,
|
||||
)
|
||||
@@ -624,14 +626,40 @@ class TurnstoneBot:
|
||||
log.debug("discord.tool_info_status_edit_failed", ws_id=ws_id)
|
||||
|
||||
# Send the result as a separate message.
|
||||
desc = format_tool_result(event.output)
|
||||
color = discord.Color.red() if event.is_error else discord.Color.dark_grey()
|
||||
result_embed = discord.Embed(
|
||||
title=event.name,
|
||||
description=desc,
|
||||
color=color,
|
||||
)
|
||||
await thread.send(embed=result_embed)
|
||||
if not event.is_error:
|
||||
from turnstone.channels._formatter import try_build_media_embed
|
||||
|
||||
media_result = None
|
||||
try:
|
||||
media_result = await try_build_media_embed(
|
||||
event.name,
|
||||
event.output,
|
||||
http=self._http_client,
|
||||
)
|
||||
except Exception:
|
||||
log.debug("discord.media_embed_failed", ws_id=ws_id, tool=event.name)
|
||||
if media_result is not None:
|
||||
embed, file = media_result
|
||||
kwargs: dict[str, Any] = {"embed": embed}
|
||||
if file is not None:
|
||||
kwargs["file"] = file
|
||||
await thread.send(**kwargs)
|
||||
else:
|
||||
desc = format_tool_result(event.output)
|
||||
result_embed = discord.Embed(
|
||||
title=event.name,
|
||||
description=desc,
|
||||
color=discord.Color.dark_grey(),
|
||||
)
|
||||
await thread.send(embed=result_embed)
|
||||
else:
|
||||
desc = format_tool_result(event.output)
|
||||
result_embed = discord.Embed(
|
||||
title=event.name,
|
||||
description=desc,
|
||||
color=discord.Color.red(),
|
||||
)
|
||||
await thread.send(embed=result_embed)
|
||||
|
||||
elif isinstance(event, ApproveRequestEvent):
|
||||
# Evaluate admin tool policies before auto-approve.
|
||||
|
||||
+2
-1
@@ -176,7 +176,8 @@ class TerminalUI(SessionUI):
|
||||
else:
|
||||
sys.stdout.write(f" {yellow(item['header'])}\n")
|
||||
if item.get("preview"):
|
||||
sys.stdout.write(item["preview"] + "\n")
|
||||
styled = dim(item["preview"]) if not item.get("error") else red(item["preview"])
|
||||
sys.stdout.write(styled + "\n")
|
||||
verdict = item.get("_heuristic_verdict")
|
||||
if verdict:
|
||||
risk = verdict.get("risk_level", "medium")
|
||||
|
||||
@@ -67,7 +67,7 @@ def _mcp_to_openai(server_name: str, tool: Any) -> dict[str, Any]:
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": f"mcp__{server_name}__{tool.name}",
|
||||
"description": f"[MCP: {server_name}] {description}",
|
||||
"description": description,
|
||||
"parameters": input_schema,
|
||||
},
|
||||
}
|
||||
|
||||
+11
-11
@@ -2936,7 +2936,7 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": func_name,
|
||||
"header": f"\u2717 {func_name}: {exc}",
|
||||
"preview": f" {RED}{preview}{RESET}",
|
||||
"preview": f" {preview}",
|
||||
"needs_approval": False,
|
||||
"error": (
|
||||
f"JSON parse error for tool '{func_name}': {exc}\n"
|
||||
@@ -3528,7 +3528,7 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": "man",
|
||||
"header": "\u2717 man: invalid page name",
|
||||
"preview": f" {RED}{page}{RESET}",
|
||||
"preview": f" {page}",
|
||||
"needs_approval": False,
|
||||
"error": f"Error: invalid page name {page!r}",
|
||||
}
|
||||
@@ -3574,7 +3574,7 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": "web_fetch",
|
||||
"header": "\u2717 web_fetch: invalid url",
|
||||
"preview": f" {RED}{url}{RESET}",
|
||||
"preview": f" {url}",
|
||||
"needs_approval": False,
|
||||
"error": f"Error: URL must start with http:// or https:// (got {url!r})",
|
||||
}
|
||||
@@ -3585,12 +3585,12 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": "web_fetch",
|
||||
"header": "\u2717 web_fetch: blocked (private network)",
|
||||
"preview": f" {RED}{url}{RESET}",
|
||||
"preview": f" {url}",
|
||||
"needs_approval": False,
|
||||
"error": f"Error: {ssrf_err}",
|
||||
}
|
||||
q_preview = question[:200] + ("..." if len(question) > 200 else "")
|
||||
preview = f" {DIM}{url}\n Q: {q_preview}{RESET}"
|
||||
preview = f" {url}\n Q: {q_preview}"
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "web_fetch",
|
||||
@@ -3636,7 +3636,7 @@ class ChatSession:
|
||||
if topic not in ("general", "news", "finance"):
|
||||
topic = "general"
|
||||
q_preview = query[:200] + ("..." if len(query) > 200 else "")
|
||||
preview = f" {DIM}{q_preview}{RESET}"
|
||||
preview = f" {q_preview}"
|
||||
return {
|
||||
"call_id": call_id,
|
||||
"func_name": "web_search",
|
||||
@@ -3675,7 +3675,7 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": "tool_search",
|
||||
"header": f"\u2699 tool_search: {query[:80]}",
|
||||
"preview": f" {DIM}{query}{RESET}",
|
||||
"preview": f" {query}",
|
||||
"needs_approval": False,
|
||||
"execute": self._exec_tool_search,
|
||||
"query": query,
|
||||
@@ -3709,7 +3709,7 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": "task_agent",
|
||||
"header": "\u2699 task_agent (autonomous agent)",
|
||||
"preview": f" {DIM}{preview_text}{RESET}",
|
||||
"preview": f" {preview_text}",
|
||||
"needs_approval": True,
|
||||
"approval_label": "task_agent",
|
||||
"execute": self._exec_task,
|
||||
@@ -3733,7 +3733,7 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": "plan_agent",
|
||||
"header": "\u2699 plan_agent (planning agent)",
|
||||
"preview": f" {DIM}{preview_text}{RESET}",
|
||||
"preview": f" {preview_text}",
|
||||
"needs_approval": True,
|
||||
"approval_label": "plan_agent",
|
||||
"execute": self._exec_plan,
|
||||
@@ -4287,7 +4287,7 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": func_name,
|
||||
"header": f"\u2699 mcp:{display}",
|
||||
"preview": f"{DIM}{preview}{RESET}",
|
||||
"preview": preview,
|
||||
"needs_approval": True,
|
||||
"approval_label": func_name,
|
||||
"execute": self._exec_mcp_tool,
|
||||
@@ -4365,7 +4365,7 @@ class ChatSession:
|
||||
"call_id": call_id,
|
||||
"func_name": "read_resource",
|
||||
"header": "\u2699 read_resource",
|
||||
"preview": f"{DIM} uri: {uri}{RESET}",
|
||||
"preview": f" uri: {uri}",
|
||||
"needs_approval": True,
|
||||
"approval_label": f"mcp_resource__{self._normalize_resource_uri(uri)}",
|
||||
"execute": self._exec_read_resource,
|
||||
|
||||
@@ -66,7 +66,7 @@
|
||||
--accent-dim: rgba(140, 94, 27, 0.1);
|
||||
--accent-glow: rgba(140, 94, 27, 0.05);
|
||||
--green: #047857;
|
||||
--red: #dc2626;
|
||||
--red: #b91c1c;
|
||||
--yellow: #b45309;
|
||||
--cyan: #0e7490;
|
||||
--magenta: #7c3aed;
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
Copyright (c) 2017 Dailymotion (http://www.dailymotion.com)
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
|
||||
src/remux/mp4-generator.js and src/demux/exp-golomb.ts implementation in this project
|
||||
are derived from the HLS library for video.js (https://github.com/videojs/videojs-contrib-hls)
|
||||
|
||||
That work is also covered by the Apache 2 License, following copyright:
|
||||
Copyright (c) 2013-2015 Brightcove
|
||||
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
||||
THE SOFTWARE.
|
||||
File diff suppressed because one or more lines are too long
+437
-16
@@ -861,13 +861,23 @@ Pane.prototype.replayHistory = function (messages) {
|
||||
cmd.className = "tool-cmd";
|
||||
try {
|
||||
var args = JSON.parse(tc.arguments);
|
||||
var preview = Object.values(args)[0] || "";
|
||||
if (tc.name === "bash") {
|
||||
var preview = Object.values(args)[0] || "";
|
||||
cmd.innerHTML =
|
||||
'<span class="dollar">$ </span>' +
|
||||
escapeHtml(String(preview));
|
||||
} else {
|
||||
cmd.textContent = String(preview).substring(0, 200);
|
||||
var parts = [];
|
||||
var keys = Object.keys(args);
|
||||
for (var k = 0; k < keys.length; k++) {
|
||||
var val = args[keys[k]];
|
||||
var valStr =
|
||||
val === null || val === undefined ? "null" : String(val);
|
||||
if (valStr.length > 80)
|
||||
valStr = valStr.substring(0, 77) + "...";
|
||||
parts.push(keys[k] + ": " + valStr);
|
||||
}
|
||||
cmd.textContent = parts.join("\n");
|
||||
}
|
||||
} catch (e) {
|
||||
cmd.textContent = tc.arguments.substring(0, 100);
|
||||
@@ -906,16 +916,21 @@ Pane.prototype.replayHistory = function (messages) {
|
||||
/^Blocked/.test(stripped);
|
||||
var isToolError = !!msg.is_error;
|
||||
if (stripped && !isDenied) {
|
||||
var out = document.createElement("div");
|
||||
out.className =
|
||||
"tool-output" + (isToolError ? " tool-output-error" : "");
|
||||
out.textContent = stripped;
|
||||
if (stripped.split("\n").length > 10) {
|
||||
makeCollapsible(out);
|
||||
var media = !isToolError ? tryParseMedia(stripped) : null;
|
||||
if (media) {
|
||||
var embed = buildMediaEmbed(media, stripped);
|
||||
var bdg = lastToolBlock.querySelector(".approval-badge");
|
||||
if (bdg) lastToolBlock.insertBefore(embed, bdg);
|
||||
else lastToolBlock.appendChild(embed);
|
||||
} else {
|
||||
var out = renderToolOutput(stripped, isToolError);
|
||||
if (out.textContent.split("\n").length > 10) {
|
||||
makeCollapsible(out);
|
||||
}
|
||||
var bdg = lastToolBlock.querySelector(".approval-badge");
|
||||
if (bdg) lastToolBlock.insertBefore(out, bdg);
|
||||
else lastToolBlock.appendChild(out);
|
||||
}
|
||||
var bdg = lastToolBlock.querySelector(".approval-badge");
|
||||
if (bdg) lastToolBlock.insertBefore(out, bdg);
|
||||
else lastToolBlock.appendChild(out);
|
||||
}
|
||||
if (isToolError && !lastToolBlock.classList.contains("denied")) {
|
||||
lastToolBlock.classList.add("error");
|
||||
@@ -1221,10 +1236,18 @@ Pane.prototype.appendToolOutput = function (callId, name, output, isError) {
|
||||
var stripped = stripAnsi(output || "").trim();
|
||||
if (!stripped) return;
|
||||
|
||||
// Style tool output as error when indicated by isError flag
|
||||
var out = document.createElement("div");
|
||||
out.className = "tool-output" + (isError ? " tool-output-error" : "");
|
||||
out.textContent = stripped;
|
||||
// Detect structured media output and render interactive embed
|
||||
if (!isError) {
|
||||
var media = tryParseMedia(stripped);
|
||||
if (media) {
|
||||
var embed = buildMediaEmbed(media, stripped);
|
||||
target.after(embed);
|
||||
this.scrollToBottom();
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
var out = renderToolOutput(stripped, isError);
|
||||
|
||||
// Mark the parent approval block as errored
|
||||
if (isError) {
|
||||
@@ -1239,7 +1262,7 @@ Pane.prototype.appendToolOutput = function (callId, name, output, isError) {
|
||||
}
|
||||
}
|
||||
|
||||
if (stripped.split("\n").length > 10) {
|
||||
if (out.textContent.split("\n").length > 10) {
|
||||
makeCollapsible(out);
|
||||
}
|
||||
|
||||
@@ -3170,6 +3193,404 @@ function makeCollapsible(el) {
|
||||
});
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// 12a. Media embed renderer (MCP tool output with stream_url / results)
|
||||
// ===========================================================================
|
||||
|
||||
function tryParseMedia(text) {
|
||||
try {
|
||||
var obj = JSON.parse(text);
|
||||
} catch (e) {
|
||||
return null;
|
||||
}
|
||||
if (obj && typeof obj.stream_url === "string") return obj;
|
||||
if (obj && obj.name && obj.type && obj.id) return obj;
|
||||
if (obj && Array.isArray(obj.results) && obj.results.length > 0) return obj;
|
||||
if (obj && Array.isArray(obj.sessions)) return obj;
|
||||
return null;
|
||||
}
|
||||
|
||||
function _formatRuntime(item) {
|
||||
var mins = 0;
|
||||
if (typeof item.runtime_minutes === "number") {
|
||||
mins = Math.round(item.runtime_minutes);
|
||||
} else if (typeof item.runtime_ticks === "number") {
|
||||
mins = Math.round(item.runtime_ticks / 600000000);
|
||||
}
|
||||
if (!mins) return "";
|
||||
var h = Math.floor(mins / 60);
|
||||
var m = mins % 60;
|
||||
return h > 0 ? h + "h " + m + "m" : m + "m";
|
||||
}
|
||||
|
||||
function _redactApiKeys(text) {
|
||||
// Query-string style: api_key=VALUE
|
||||
var redacted = text.replace(
|
||||
/(?:api_key|apiKey|api-key|token)=[^&\s"]+/g,
|
||||
function (m) {
|
||||
return m.split("=")[0] + "=***";
|
||||
},
|
||||
);
|
||||
// JSON style: "api_key": "VALUE"
|
||||
redacted = redacted.replace(
|
||||
/(["'](?:api_key|apiKey|api-key|token)["']\s*:\s*["'])([^"']*)(['"])/gi,
|
||||
"$1***$3",
|
||||
);
|
||||
return redacted;
|
||||
}
|
||||
|
||||
/**
|
||||
* Try to pretty-print JSON text with indentation and API key redaction.
|
||||
* Returns a formatted string if valid JSON, otherwise null.
|
||||
*/
|
||||
function _tryPrettyJson(text) {
|
||||
try {
|
||||
var obj = JSON.parse(text);
|
||||
} catch (e) {
|
||||
return null;
|
||||
}
|
||||
return _redactApiKeys(JSON.stringify(obj, null, 2));
|
||||
}
|
||||
|
||||
/**
|
||||
* Render tool output text into a DOM element.
|
||||
* If the text is valid JSON, pretty-prints it with indentation.
|
||||
* Otherwise renders as plain text. Always redacts API keys.
|
||||
*/
|
||||
function renderToolOutput(stripped, isError) {
|
||||
var out = document.createElement("div");
|
||||
out.className = "tool-output" + (isError ? " tool-output-error" : "");
|
||||
if (!isError) {
|
||||
var pretty = _tryPrettyJson(stripped);
|
||||
if (pretty) {
|
||||
out.textContent = pretty;
|
||||
return out;
|
||||
}
|
||||
}
|
||||
out.textContent = _redactApiKeys(stripped);
|
||||
return out;
|
||||
}
|
||||
|
||||
function buildMediaEmbed(media, rawJson) {
|
||||
var wrapper = document.createElement("div");
|
||||
wrapper.className = "media-embed";
|
||||
|
||||
if (media.stream_url) {
|
||||
var card = buildMediaCard(media);
|
||||
card.querySelector(".media-card-info").appendChild(buildPlayButton(media));
|
||||
wrapper.appendChild(card);
|
||||
} else if (media.results) {
|
||||
wrapper.appendChild(
|
||||
buildMediaResultsList(media.results, media.total_count),
|
||||
);
|
||||
} else if (media.sessions) {
|
||||
wrapper.appendChild(buildMediaResultsList(media.sessions, null));
|
||||
} else if (media.name && media.type && media.id) {
|
||||
wrapper.appendChild(buildMediaCard(media));
|
||||
}
|
||||
|
||||
// Collapsed raw JSON for inspection (with redacted API keys)
|
||||
var raw = document.createElement("div");
|
||||
raw.className = "tool-output";
|
||||
raw.textContent = _tryPrettyJson(rawJson) || _redactApiKeys(rawJson);
|
||||
makeCollapsible(raw);
|
||||
wrapper.appendChild(raw);
|
||||
|
||||
return wrapper;
|
||||
}
|
||||
|
||||
function buildMediaCard(item) {
|
||||
var card = document.createElement("div");
|
||||
card.className = "media-card";
|
||||
|
||||
// Thumbnail
|
||||
var thumbUrl = item.thumbnail_url || item.image_url || "";
|
||||
if (thumbUrl) {
|
||||
var img = document.createElement("img");
|
||||
img.className = "media-card-thumb";
|
||||
img.loading = "lazy";
|
||||
img.alt = item.title || item.name || "Media thumbnail";
|
||||
img.onerror = function () {
|
||||
this.style.display = "none";
|
||||
};
|
||||
img.src = thumbUrl;
|
||||
card.appendChild(img);
|
||||
}
|
||||
|
||||
// Info container
|
||||
var info = document.createElement("div");
|
||||
info.className = "media-card-info";
|
||||
|
||||
// Title (Year)
|
||||
var title = document.createElement("div");
|
||||
title.className = "media-card-title";
|
||||
var titleText = item.title || item.name || "Untitled";
|
||||
if (item.year || item.production_year) {
|
||||
titleText += " (" + (item.year || item.production_year) + ")";
|
||||
}
|
||||
title.textContent = titleText;
|
||||
info.appendChild(title);
|
||||
|
||||
// Metadata line: type, runtime, genres
|
||||
var metaParts = [];
|
||||
if (item.type || item.media_type) {
|
||||
metaParts.push(item.type || item.media_type);
|
||||
}
|
||||
var runtime = _formatRuntime(item);
|
||||
if (runtime) metaParts.push(runtime);
|
||||
if (item.genres && item.genres.length) {
|
||||
metaParts.push(item.genres.join(", "));
|
||||
}
|
||||
if (metaParts.length) {
|
||||
var meta = document.createElement("div");
|
||||
meta.className = "media-card-meta";
|
||||
meta.textContent = metaParts.join(" \u00b7 ");
|
||||
info.appendChild(meta);
|
||||
}
|
||||
|
||||
card.appendChild(info);
|
||||
return card;
|
||||
}
|
||||
|
||||
function buildPlayButton(media) {
|
||||
var btn = document.createElement("button");
|
||||
btn.className = "media-play-btn";
|
||||
btn.type = "button";
|
||||
btn.dataset.streamUrl = media.stream_url || "";
|
||||
btn.dataset.hlsUrl = media.hls_url || "";
|
||||
btn.dataset.audioOnly =
|
||||
media.audio_only === true ||
|
||||
(media.container &&
|
||||
/^(mp3|flac|ogg|aac|wma|wav|m4a|opus)$/i.test(media.container))
|
||||
? "true"
|
||||
: "false";
|
||||
btn.dataset.directStream =
|
||||
media.supports_direct_play || media.supports_direct_stream
|
||||
? "true"
|
||||
: "false";
|
||||
|
||||
btn.setAttribute(
|
||||
"aria-label",
|
||||
"Play " + (media.title || media.name || "media"),
|
||||
);
|
||||
|
||||
var icon = document.createElement("span");
|
||||
icon.textContent = "\u25b6";
|
||||
btn.appendChild(icon);
|
||||
var label = document.createElement("span");
|
||||
label.textContent = "Play";
|
||||
btn.appendChild(label);
|
||||
return btn;
|
||||
}
|
||||
|
||||
function buildMediaResultsList(results, totalCount) {
|
||||
var container = document.createElement("div");
|
||||
container.className = "media-results-list";
|
||||
|
||||
for (var i = 0; i < results.length; i++) {
|
||||
var item = results[i];
|
||||
var row = document.createElement("div");
|
||||
row.className = "media-result-row";
|
||||
|
||||
// Small thumbnail
|
||||
var thumbUrl = item.thumbnail_url || item.image_url || "";
|
||||
if (thumbUrl) {
|
||||
var img = document.createElement("img");
|
||||
img.className = "media-result-thumb";
|
||||
img.loading = "lazy";
|
||||
img.alt = item.name || item.title || "Media thumbnail";
|
||||
img.onerror = function () {
|
||||
this.style.display = "none";
|
||||
};
|
||||
img.src = thumbUrl;
|
||||
row.appendChild(img);
|
||||
}
|
||||
|
||||
// Title (Year)
|
||||
var titleSpan = document.createElement("span");
|
||||
titleSpan.className = "media-result-title";
|
||||
var titleText = item.name || item.title || "Untitled";
|
||||
if (item.year || item.production_year) {
|
||||
titleText += " (" + (item.year || item.production_year) + ")";
|
||||
}
|
||||
titleSpan.textContent = titleText;
|
||||
row.appendChild(titleSpan);
|
||||
|
||||
// Metadata: type, runtime or season info
|
||||
var metaParts = [];
|
||||
if (item.type || item.media_type) {
|
||||
metaParts.push(item.type || item.media_type);
|
||||
}
|
||||
var runtime = _formatRuntime(item);
|
||||
if (runtime) metaParts.push(runtime);
|
||||
if (item.season_name) metaParts.push(item.season_name);
|
||||
if (
|
||||
typeof item.index_number === "number" &&
|
||||
typeof item.parent_index_number === "number"
|
||||
) {
|
||||
metaParts.push(
|
||||
"S" +
|
||||
String(item.parent_index_number).padStart(2, "0") +
|
||||
"E" +
|
||||
String(item.index_number).padStart(2, "0"),
|
||||
);
|
||||
}
|
||||
if (metaParts.length) {
|
||||
var metaSpan = document.createElement("span");
|
||||
metaSpan.className = "media-result-meta";
|
||||
metaSpan.textContent = " \u00b7 " + metaParts.join(" \u00b7 ");
|
||||
row.appendChild(metaSpan);
|
||||
}
|
||||
|
||||
container.appendChild(row);
|
||||
}
|
||||
|
||||
// "showing X of Y results" footer
|
||||
if (typeof totalCount === "number" && totalCount > results.length) {
|
||||
var count = document.createElement("div");
|
||||
count.className = "media-results-count";
|
||||
count.textContent =
|
||||
"showing " + results.length + " of " + totalCount + " results";
|
||||
container.appendChild(count);
|
||||
}
|
||||
|
||||
return container;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// HLS lazy-loader (follows mermaid.js pattern from renderer.js:724-751)
|
||||
// ---------------------------------------------------------------------------
|
||||
var _hlsState = "idle";
|
||||
var _hlsQueue = [];
|
||||
|
||||
function _loadHls(callback) {
|
||||
if (_hlsState === "ready") {
|
||||
callback();
|
||||
return;
|
||||
}
|
||||
_hlsQueue.push(callback);
|
||||
if (_hlsState === "loading") return;
|
||||
_hlsState = "loading";
|
||||
var script = document.createElement("script");
|
||||
script.src = "/shared/hls-1.6.15/hls.min.js";
|
||||
script.onload = function () {
|
||||
_hlsState = "ready";
|
||||
var q = _hlsQueue;
|
||||
_hlsQueue = [];
|
||||
for (var i = 0; i < q.length; i++) q[i]();
|
||||
};
|
||||
script.onerror = function () {
|
||||
_hlsState = "idle";
|
||||
var q = _hlsQueue;
|
||||
_hlsQueue = [];
|
||||
// Fall through — _activatePlayer will use stream_url since Hls is undefined
|
||||
for (var i = 0; i < q.length; i++) q[i]();
|
||||
};
|
||||
document.head.appendChild(script);
|
||||
}
|
||||
|
||||
function _isHlsUrl(url) {
|
||||
return typeof url === "string" && /\.m3u8(\?|$)/i.test(url);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Click-to-play delegated handler (follows img-placeholder pattern)
|
||||
// ---------------------------------------------------------------------------
|
||||
function _activatePlayer(btn) {
|
||||
var url = btn.dataset.streamUrl;
|
||||
var hlsUrl = btn.dataset.hlsUrl;
|
||||
var isAudio = btn.dataset.audioOnly === "true";
|
||||
var directStream = btn.dataset.directStream === "true";
|
||||
|
||||
var player = document.createElement(isAudio ? "audio" : "video");
|
||||
player.controls = true;
|
||||
player.autoplay = true;
|
||||
player.className = "media-player";
|
||||
|
||||
// Prefer direct stream when the source supports it; fall back to HLS
|
||||
// only when transcoding is needed.
|
||||
if (directStream && url) {
|
||||
player.src = url;
|
||||
} else if (
|
||||
hlsUrl &&
|
||||
!isAudio &&
|
||||
typeof Hls !== "undefined" &&
|
||||
Hls.isSupported()
|
||||
) {
|
||||
var hls = new Hls();
|
||||
hls.loadSource(hlsUrl);
|
||||
hls.attachMedia(player);
|
||||
} else if (
|
||||
hlsUrl &&
|
||||
!isAudio &&
|
||||
player.canPlayType("application/vnd.apple.mpegurl")
|
||||
) {
|
||||
player.src = hlsUrl;
|
||||
} else {
|
||||
player.src = url;
|
||||
}
|
||||
|
||||
player.addEventListener("error", function () {
|
||||
var card = player.closest(".media-embed");
|
||||
var titleEl = card ? card.querySelector(".media-card-title") : null;
|
||||
var label = titleEl ? ": " + titleEl.textContent : "";
|
||||
|
||||
var err = document.createElement("div");
|
||||
err.className = "media-player-error";
|
||||
err.setAttribute("role", "alert");
|
||||
err.textContent = "Failed to load stream" + label;
|
||||
|
||||
var retry = document.createElement("button");
|
||||
retry.className = "media-play-btn";
|
||||
retry.type = "button";
|
||||
retry.dataset.streamUrl = url;
|
||||
retry.dataset.hlsUrl = hlsUrl || "";
|
||||
retry.dataset.audioOnly = String(isAudio);
|
||||
retry.dataset.directStream = String(directStream);
|
||||
retry.setAttribute("aria-label", "Retry" + label);
|
||||
retry.appendChild(document.createTextNode("\u25b6 Retry"));
|
||||
|
||||
var container = document.createElement("div");
|
||||
container.appendChild(err);
|
||||
container.appendChild(retry);
|
||||
player.replaceWith(container);
|
||||
});
|
||||
|
||||
btn.replaceWith(player);
|
||||
}
|
||||
|
||||
document.addEventListener("click", function (e) {
|
||||
var btn = e.target.closest(".media-play-btn");
|
||||
if (!btn) return;
|
||||
e.preventDefault();
|
||||
btn.disabled = true;
|
||||
var labelEl = btn.querySelector("span:last-child");
|
||||
if (labelEl) {
|
||||
labelEl.textContent = "Loading\u2026";
|
||||
} else {
|
||||
btn.textContent = "\u25b6 Loading\u2026";
|
||||
}
|
||||
|
||||
var hlsUrl = btn.dataset.hlsUrl;
|
||||
var isAudio = btn.dataset.audioOnly === "true";
|
||||
|
||||
// If HLS URL present and not audio, ensure hls.js is loaded first
|
||||
if (hlsUrl && !isAudio && _isHlsUrl(hlsUrl)) {
|
||||
_loadHls(function () {
|
||||
_activatePlayer(btn);
|
||||
});
|
||||
} else {
|
||||
_activatePlayer(btn);
|
||||
}
|
||||
});
|
||||
|
||||
document.addEventListener("keydown", function (e) {
|
||||
if (e.key !== "Enter") return;
|
||||
var btn = e.target.closest(".media-play-btn");
|
||||
if (!btn) return;
|
||||
btn.click();
|
||||
});
|
||||
|
||||
// ===========================================================================
|
||||
// 13. Plan review dialog
|
||||
// ===========================================================================
|
||||
|
||||
@@ -867,7 +867,7 @@ body { position: static; }
|
||||
.approval-tool { padding: 8px 12px; border-bottom: 1px solid var(--border); }
|
||||
.approval-tool:last-of-type { border-bottom: none; }
|
||||
.approval-tool .tool-name { color: var(--yellow); font-weight: 600; font-size: 11px; margin-bottom: 3px; }
|
||||
.approval-tool .tool-cmd { color: var(--fg-bright); white-space: pre-wrap; word-break: break-all; }
|
||||
.approval-tool .tool-cmd { color: var(--fg-bright); white-space: pre-wrap; word-break: break-all; max-height: 120px; overflow: hidden; }
|
||||
.approval-tool .tool-cmd .dollar { color: var(--green); }
|
||||
.approval-tool .tool-diff { white-space: pre-wrap; font-size: 12px; margin-top: 4px; }
|
||||
.approval-tool .tool-diff .diff-del { color: var(--red); }
|
||||
@@ -961,6 +961,128 @@ body { position: static; }
|
||||
letter-spacing: 0.03em;
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Media embed cards (MCP tool output with stream_url / results)
|
||||
========================================================================== */
|
||||
.media-embed {
|
||||
border-top: 1px solid var(--border);
|
||||
background: var(--code-bg);
|
||||
}
|
||||
.media-card {
|
||||
display: flex;
|
||||
gap: 12px;
|
||||
padding: 10px 12px;
|
||||
align-items: flex-start;
|
||||
}
|
||||
.media-card-thumb {
|
||||
width: 80px;
|
||||
height: 120px;
|
||||
object-fit: cover;
|
||||
border-radius: var(--radius-sm);
|
||||
background: var(--bg-surface);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.media-card-info {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
}
|
||||
.media-card-title {
|
||||
font-family: var(--font-display);
|
||||
font-size: 14px;
|
||||
font-weight: 600;
|
||||
color: var(--fg-bright);
|
||||
margin-bottom: 2px;
|
||||
}
|
||||
.media-card-meta {
|
||||
font-size: 11px;
|
||||
color: var(--fg-dim);
|
||||
margin-bottom: 6px;
|
||||
}
|
||||
.media-play-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
padding: 4px 12px;
|
||||
font-size: 12px;
|
||||
font-family: var(--font-mono);
|
||||
color: var(--accent);
|
||||
background: transparent;
|
||||
border: 1px solid var(--accent);
|
||||
border-radius: var(--radius-sm);
|
||||
cursor: pointer;
|
||||
transition: background 0.12s ease;
|
||||
}
|
||||
.media-play-btn:hover {
|
||||
background: rgba(229, 160, 66, 0.1);
|
||||
}
|
||||
.media-play-btn:focus-visible {
|
||||
outline: 2px solid var(--accent);
|
||||
outline-offset: 2px;
|
||||
}
|
||||
.media-play-btn:active {
|
||||
background: rgba(229, 160, 66, 0.2);
|
||||
}
|
||||
.media-play-btn:disabled {
|
||||
opacity: 0.5;
|
||||
cursor: wait;
|
||||
}
|
||||
.media-player {
|
||||
width: 100%;
|
||||
max-height: 480px;
|
||||
background: #000;
|
||||
border-radius: 0;
|
||||
}
|
||||
audio.media-player {
|
||||
max-height: 54px;
|
||||
}
|
||||
.media-player-error {
|
||||
color: var(--red);
|
||||
font-size: 12px;
|
||||
padding: 8px 12px;
|
||||
background: var(--code-bg);
|
||||
}
|
||||
.media-results-list {
|
||||
padding: 6px 12px;
|
||||
}
|
||||
.media-result-row {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
padding: 4px 0;
|
||||
align-items: center;
|
||||
font-size: 12px;
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
.media-result-row:last-child {
|
||||
border-bottom: none;
|
||||
}
|
||||
.media-result-thumb {
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
object-fit: cover;
|
||||
border-radius: var(--radius-sm);
|
||||
background: var(--bg-surface);
|
||||
flex-shrink: 0;
|
||||
}
|
||||
.media-result-title {
|
||||
color: var(--fg-bright);
|
||||
font-weight: 500;
|
||||
}
|
||||
.media-result-meta {
|
||||
color: var(--fg-dim);
|
||||
font-size: 11px;
|
||||
}
|
||||
.media-results-count {
|
||||
text-align: right;
|
||||
font-size: 10px;
|
||||
color: var(--fg-dim);
|
||||
padding: 4px 0;
|
||||
}
|
||||
@media (max-width: 480px) {
|
||||
.media-card { flex-direction: column; }
|
||||
.media-card-thumb { width: 100%; height: auto; max-height: 200px; }
|
||||
.media-player { max-height: 280px; }
|
||||
}
|
||||
|
||||
/* ==========================================================================
|
||||
Plan review dialog
|
||||
========================================================================== */
|
||||
@@ -1467,6 +1589,7 @@ body { position: static; }
|
||||
#health-indicator, #theme-toggle,
|
||||
#mcp-status, .msg-assistant tbody tr,
|
||||
.msg-assistant .img-placeholder,
|
||||
.media-play-btn,
|
||||
#new-ws-cancel, #new-ws-submit,
|
||||
#new-ws-box input, #new-ws-box select,
|
||||
.split-handle, .pane-action-btn,
|
||||
|
||||
Reference in New Issue
Block a user