mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
480a1426b3
* fix(session): fail-closed history-commit handoff (#981) The deleted-workstream discovery is now a terminal, ws_id-keyed latch: keyed conversation commits refuse admission once the durable parent is gone (convergence finalizers and force-abandon are exempt), history handoff refuses to mint a proof token so /history fails closed with a 503 instead of silently wiping the pane, and the SSE stream carries a workstream_gone resync reason. Discarded commits leave a forensic log of commit keys and roles, never content. Conversation rows gain a commit_key (migration 071): keyed saves are idempotent under retry, validated against the full commit identity, and refused when they would cross a workstream deletion. The prune orphan category now requires a NULL alias plus a two-hour updated grace, with cutoffs computed at discovery time and carried into both dialects' rechecks. The mid-turn interjection queue is owner-partitioned with no per-site mode flags: pops take the acting principal's and unowned rows, other participants' rows are structurally retained, and enforcement lives at queue admission plus the shared before_spawn gates. The retraction ledger is bounded by open pop windows: pops open a window atomically with the queue delete, restores close their ids atomically with the ledger consume, every other exit closes through one helper, and misses for unheld ids record nothing. The workstream-gone latch refuses unattended wakes at all three gates (watcher spawn, claim, delivery pre-pop), and the retry dispatcher regained its pre-envelope cancel/error convergence net. Persistence-state reporting derives through the session bound to each UI instead of a registry lookup by id that failed open to healthy during tombstone retention. The dashboard roster no longer re-inserts ghost entries from trailing activity events, the history tool-outcome scan tolerates interleaved non-turn rows, and the shared handoff-deadline handle owns its own retirement. Single-sourced across call sites: keyed-commit row values, attachment save wrappers, tail-truncation and conflict-resolution bodies for both storage dialects; worker-slot lifecycle field sets; the direct-commit admission frame; queued-row layout accessors; the string-aware comment stripper shared by every JS harness suite. Refs #981 #964 * fix(session): sweep handoff fixes to their sibling surfaces The interactive replay loop treated a system row as a tool-batch boundary, so every tool result after an interleaved row vanished from that pane while the coordinator rendered the same history correctly. Only a conversational turn ends the batch window now, matching the shared outcome index. Accepted user turns clear the composer's attachment chips on the same viewer policy that settles optimistic bubbles rather than on having matched a local bubble, so a workstream created with an upload no longer keeps a chip for an attachment the create dispatch already consumed. The coordinator's raced-Stop arm emits the stream-end hook it inherits alongside the idle state, leaving no unfinalized bubble or unflushed tool output. Ending a session surfaces a failure toast when the request never lands or answers with a non-JSON body. The per-second persistence reconcile now probes each session without blocking: a workstream whose generation and handoff locks are held is skipped until the next pass instead of contending the locks every commit needs. The one-shot repair that gates workstream creation at capacity keeps a definite probe — it has no next pass, and the sessions likeliest to be contended are the ones whose unresolved journals emptied its candidate list. Single-sourced: the attachment lane builds its conversation row through the shared commit-identity builder; the ordinary worker exit releases its slot through the lifecycle owner; both operator surfaces snapshot their counters through one non-consuming helper; the replay preamble loses its per-kind wrappers and its config hook; the browser harness suites share one brace walker; and each in-flight history attempt is one record carrying both its abort controller and its deadline. Refs #981 #964
837 lines
29 KiB
Python
837 lines
29 KiB
Python
"""Accepted model-turn provenance across dispatch, storage, and projections."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import dataclasses
|
|
import json
|
|
import logging
|
|
import re
|
|
from typing import TYPE_CHECKING, Any
|
|
from unittest.mock import MagicMock, patch
|
|
|
|
import httpx
|
|
import pytest
|
|
|
|
from tests._session_helpers import (
|
|
RecordingUI,
|
|
arm_session,
|
|
make_session,
|
|
replace_session_lane,
|
|
scripted_chat_client,
|
|
seam_provider,
|
|
)
|
|
from turnstone.console.coordinator_client import _serialize_messages
|
|
from turnstone.core.export import export_workstream
|
|
from turnstone.core.history_decoration import project_history_messages
|
|
from turnstone.core.model_registry import ModelConfig, ModelRegistry
|
|
from turnstone.core.model_turn import ModelLane, model_turn
|
|
from turnstone.core.providers import ModelCapabilities, StreamChunk, UsageInfo
|
|
from turnstone.core.session import ConversationPersistenceError, GenerationCancelled
|
|
from turnstone.core.storage._utils import _fork_turn_insert_row
|
|
from turnstone.core.trajectory import (
|
|
PROVENANCE_META_KEY,
|
|
EffectStatus,
|
|
Turn,
|
|
TurnProvenance,
|
|
turn_from_dict,
|
|
turn_to_dict,
|
|
)
|
|
|
|
if TYPE_CHECKING:
|
|
from collections.abc import Iterator
|
|
|
|
from turnstone.core.storage import StorageBackend
|
|
|
|
|
|
def _good_stream(text: str) -> list[StreamChunk]:
|
|
return [
|
|
StreamChunk(content_delta=text),
|
|
StreamChunk(
|
|
finish_reason="stop",
|
|
usage=UsageInfo(prompt_tokens=10, completion_tokens=2, total_tokens=12),
|
|
),
|
|
]
|
|
|
|
|
|
def _dying_stream(text: str) -> Iterator[StreamChunk]:
|
|
yield StreamChunk(content_delta=text)
|
|
raise httpx.ReadError("old binding died")
|
|
|
|
|
|
def _provenance(turn: Turn) -> dict[str, str | int]:
|
|
raw = turn.meta.extra.get(PROVENANCE_META_KEY)
|
|
assert isinstance(raw, dict)
|
|
return raw
|
|
|
|
|
|
def _log_has_field(record: logging.LogRecord, key: str, value: str | int) -> bool:
|
|
"""Accept either the console or JSON/dict structlog renderer.
|
|
|
|
Logging configuration is process-global, so a full-suite predecessor may
|
|
select a different renderer than this file sees in isolation — including
|
|
the colored console renderer, whose ANSI escapes would otherwise split
|
|
``key=value``. The event fields are the contract; their presentation
|
|
(renderer AND styling) is not.
|
|
"""
|
|
message = re.sub(r"\x1b\[[0-9;]*m", "", record.getMessage())
|
|
return any(
|
|
candidate in message
|
|
for candidate in (
|
|
f"{key}={value}",
|
|
f"'{key}': {value!r}",
|
|
f'"{key}": {json.dumps(value)}',
|
|
)
|
|
)
|
|
|
|
|
|
def _register_session_parent(session: Any) -> None:
|
|
"""Mirror production's parent-before-keyed-conversation ordering."""
|
|
from turnstone.core.storage import get_storage
|
|
|
|
storage = get_storage()
|
|
assert storage is not None
|
|
storage.register_workstream(session.ws_id, user_id=session._user_id)
|
|
|
|
|
|
def test_model_turn_stamps_one_immutable_serving_identity() -> None:
|
|
provider = seam_provider("accepted")
|
|
lane = ModelLane(
|
|
provider=provider,
|
|
client=MagicMock(),
|
|
model="backend-v2",
|
|
alias="assistant-fast",
|
|
registry_generation=17,
|
|
capabilities=ModelCapabilities(),
|
|
)
|
|
|
|
result = model_turn(
|
|
lane,
|
|
[Turn.user("hello")],
|
|
acting_principal_id="user-alice",
|
|
)
|
|
|
|
expected = {
|
|
"model_alias": "assistant-fast",
|
|
"backend_model_id": "backend-v2",
|
|
"registry_generation": 17,
|
|
"acting_principal_id": "user-alice",
|
|
}
|
|
assert result.provenance.to_meta() == expected
|
|
assert _provenance(result.turn) == expected
|
|
|
|
|
|
def test_model_turn_drain_retry_logs_kernel_axes_without_principal(
|
|
caplog: pytest.LogCaptureFixture,
|
|
) -> None:
|
|
provider = seam_provider("unused")
|
|
provider.create_streaming.side_effect = [
|
|
_dying_stream("discarded"),
|
|
_good_stream("accepted"),
|
|
]
|
|
lane = ModelLane(
|
|
provider=provider,
|
|
client=MagicMock(),
|
|
model="retry-kernel",
|
|
alias="retry-alias",
|
|
registry_generation=12,
|
|
capabilities=ModelCapabilities(),
|
|
)
|
|
|
|
with (
|
|
patch("turnstone.core.model_turn.time.sleep"),
|
|
caplog.at_level(logging.WARNING, logger="turnstone.core.model_turn"),
|
|
):
|
|
result = model_turn(
|
|
lane,
|
|
[Turn.user("hello")],
|
|
acting_principal_id="private-principal-id",
|
|
)
|
|
|
|
assert result.content == "accepted"
|
|
[retry] = [
|
|
record for record in caplog.records if "model_turn.drain_retry" in record.getMessage()
|
|
]
|
|
assert _log_has_field(retry, "alias", "retry-alias")
|
|
assert _log_has_field(retry, "model", "retry-kernel")
|
|
assert _log_has_field(retry, "registry_generation", 12)
|
|
assert "private-principal-id" not in retry.getMessage()
|
|
assert "acting_principal_id" not in retry.getMessage()
|
|
|
|
|
|
def test_creation_fallback_stamps_fallback_binding_and_principal(tmp_db: str) -> None:
|
|
registry = ModelRegistry(
|
|
models={
|
|
"primary": ModelConfig(
|
|
"primary", "http://primary/v1", "key", "primary-id", provider="openai-compatible"
|
|
),
|
|
"fallback": ModelConfig(
|
|
"fallback",
|
|
"http://fallback/v1",
|
|
"key",
|
|
"fallback-id",
|
|
provider="openai-compatible",
|
|
),
|
|
},
|
|
default="primary",
|
|
fallback=["fallback"],
|
|
)
|
|
session = make_session(
|
|
registry=registry,
|
|
model_alias="primary",
|
|
user_id="owner",
|
|
ui=RecordingUI(), # type: ignore[no-untyped-call]
|
|
)
|
|
_register_session_parent(session)
|
|
session._title_generated = True
|
|
session._primary_lane().client.chat.completions.create = MagicMock(
|
|
side_effect=ConnectionError("primary unavailable")
|
|
)
|
|
registry.get_client("fallback").chat.completions.create = scripted_chat_client(
|
|
{"content": "served by fallback"}
|
|
)
|
|
|
|
session.send("hello", acting_user_id="user-alice")
|
|
|
|
assert _provenance(session.messages[-1]) == {
|
|
"model_alias": "fallback",
|
|
"backend_model_id": "fallback-id",
|
|
"registry_generation": registry.generation,
|
|
"acting_principal_id": "user-alice",
|
|
}
|
|
|
|
|
|
def test_midstream_rebind_stamps_only_the_successful_replacement(
|
|
tmp_db: str,
|
|
caplog: pytest.LogCaptureFixture,
|
|
) -> None:
|
|
session = make_session(
|
|
model_alias="primary",
|
|
registry_generation=3,
|
|
user_id="owner",
|
|
ui=RecordingUI(), # type: ignore[no-untyped-call]
|
|
)
|
|
_register_session_parent(session)
|
|
provider = arm_session(
|
|
session,
|
|
_dying_stream("discarded"),
|
|
_good_stream("accepted"),
|
|
)
|
|
refresh_count = 0
|
|
|
|
def _refresh() -> None:
|
|
nonlocal refresh_count
|
|
refresh_count += 1
|
|
if refresh_count != 2:
|
|
return
|
|
replace_session_lane(session, model="replacement-id")
|
|
session._model_binding = dataclasses.replace(
|
|
session._model_binding,
|
|
registry_generation=9,
|
|
)
|
|
|
|
with (
|
|
patch.object(session, "_refresh_model_from_registry", side_effect=_refresh),
|
|
caplog.at_level(logging.DEBUG, logger="turnstone.core.session"),
|
|
):
|
|
session.send("hello", acting_user_id="private-principal-id")
|
|
|
|
assert provider.create_streaming.call_count == 2
|
|
assert _provenance(session.messages[-1]) == {
|
|
"model_alias": "primary",
|
|
"backend_model_id": "replacement-id",
|
|
"registry_generation": 9,
|
|
"acting_principal_id": "private-principal-id",
|
|
}
|
|
[retry] = [record for record in caplog.records if "stream.retry" in record.getMessage()]
|
|
assert _log_has_field(retry, "alias", "primary")
|
|
assert _log_has_field(retry, "model", "test-model")
|
|
assert _log_has_field(retry, "registry_generation", 3)
|
|
assert "private-principal-id" not in retry.getMessage()
|
|
assert "acting_principal_id" not in retry.getMessage()
|
|
|
|
[finished] = [record for record in caplog.records if "stream.finished" in record.getMessage()]
|
|
assert _log_has_field(finished, "alias", "primary")
|
|
assert _log_has_field(finished, "model", "replacement-id")
|
|
assert _log_has_field(finished, "registry_generation", 9)
|
|
assert "private-principal-id" not in finished.getMessage()
|
|
assert "acting_principal_id" not in finished.getMessage()
|
|
|
|
|
|
def test_headless_send_stamps_effective_owner_principal(tmp_db: str) -> None:
|
|
"""Scheduled/internal sends record the credential principal they use."""
|
|
session = make_session(
|
|
model_alias="headless",
|
|
registry_generation=6,
|
|
user_id="owner-principal",
|
|
ui=RecordingUI(), # type: ignore[no-untyped-call]
|
|
)
|
|
_register_session_parent(session)
|
|
arm_session(session, _good_stream("accepted"))
|
|
|
|
session.send("scheduled work")
|
|
|
|
assert _provenance(session.messages[-1]) == {
|
|
"model_alias": "headless",
|
|
"backend_model_id": "test-model",
|
|
"registry_generation": 6,
|
|
"acting_principal_id": "owner-principal",
|
|
}
|
|
|
|
|
|
def test_shared_workstream_rebind_cannot_relabel_inflight_turn(tmp_db: str) -> None:
|
|
session = make_session(
|
|
model_alias="shared",
|
|
registry_generation=4,
|
|
user_id="owner",
|
|
ui=RecordingUI(), # type: ignore[no-untyped-call]
|
|
)
|
|
_register_session_parent(session)
|
|
|
|
def _stream() -> Iterator[StreamChunk]:
|
|
# A second browser binds a new actor while Alice's response is in
|
|
# flight. The accepted result must retain the generation principal.
|
|
session.bind_acting_user("user-bob")
|
|
yield from _good_stream("alice's answer")
|
|
|
|
arm_session(session, _stream())
|
|
session.send("alice's question", acting_user_id="user-alice")
|
|
|
|
assert session._acting_user_id == "user-bob"
|
|
assert _provenance(session.messages[-1])["acting_principal_id"] == "user-alice"
|
|
|
|
|
|
def test_tool_rows_record_the_same_principal_as_their_assistant_turn(tmp_db: str) -> None:
|
|
"""Turn identity is symmetric across the roles one generation persists.
|
|
|
|
The assistant row carries the principal on its four-axis provenance
|
|
envelope; the TOOL rows its batch produced carry the same identity as a
|
|
plain sibling of their disposition — not a second four-axis tuple, since a
|
|
tool row is an effect receipt and its kernel axes would be empty. Both
|
|
read the generation's bound principal, so revocation can query tool rows
|
|
directly instead of joining each one back to its batch head.
|
|
"""
|
|
session = make_session(
|
|
model_alias="main",
|
|
registry_generation=5,
|
|
user_id="owner",
|
|
ui=RecordingUI(), # type: ignore[no-untyped-call]
|
|
)
|
|
_register_session_parent(session)
|
|
session._title_generated = True
|
|
session._primary_lane().client.chat.completions.create = scripted_chat_client(
|
|
{
|
|
"tool_calls": [{"id": "call-audit", "name": "read_only_probe", "arguments": "{}"}],
|
|
"finish_reason": "tool_calls",
|
|
},
|
|
{"content": "done"},
|
|
)
|
|
tool_meta: dict[str, str | None] = {}
|
|
|
|
def _record(_ws_id: str, role: str, _content: str, *_args: Any, **kwargs: Any) -> int:
|
|
if role == "tool":
|
|
tool_meta[str(kwargs.get("tool_call_id") or "")] = kwargs.get("meta")
|
|
return 1
|
|
|
|
with (
|
|
patch.object(
|
|
session,
|
|
"_execute_tools",
|
|
return_value=([("call-audit", "observed output")], None),
|
|
),
|
|
patch("turnstone.core.session.save_message", side_effect=_record),
|
|
):
|
|
session.send("run the probe", acting_user_id="user-alice")
|
|
|
|
assert _provenance(session.messages[-1])["acting_principal_id"] == "user-alice"
|
|
assert json.loads(tool_meta["call-audit"]) == {"acting_principal": "user-alice"}
|
|
|
|
|
|
def test_utility_completion_stamps_snapshotted_acting_principal(tmp_db: str) -> None:
|
|
"""History-visible utility output can never inherit a later actor rebind."""
|
|
session = make_session(user_id="owner")
|
|
session.bind_acting_user("user-alice")
|
|
provider = seam_provider("summary")
|
|
lane = replace_session_lane(
|
|
session,
|
|
provider=provider,
|
|
model="summary-kernel",
|
|
alias="summary-alias",
|
|
)
|
|
|
|
result = session._utility_completion([Turn.user("summarize")], lane=lane)
|
|
|
|
assert result.provenance == TurnProvenance(
|
|
model_alias="summary-alias",
|
|
backend_model_id="summary-kernel",
|
|
registry_generation=lane.registry_generation,
|
|
acting_principal_id="user-alice",
|
|
)
|
|
|
|
|
|
def test_token_calibration_isolated_across_primary_fallback_primary() -> None:
|
|
"""A -> B -> A restores A's ratio and A's own prompt-count anchor."""
|
|
session = make_session(model_alias="primary", registry_generation=7)
|
|
primary = session._primary_lane()
|
|
fallback = dataclasses.replace(
|
|
primary,
|
|
alias="fallback",
|
|
model="fallback-kernel",
|
|
registry_generation=11,
|
|
)
|
|
session.messages = [Turn.user("first")]
|
|
session._msg_tokens = [1]
|
|
|
|
session._activate_token_calibration(primary)
|
|
session._last_usage = {
|
|
"prompt_tokens": 50,
|
|
"completion_tokens": 5,
|
|
"total_tokens": 55,
|
|
}
|
|
session._update_token_table(
|
|
msgs=[{"role": "user", "content": "a" * 246}],
|
|
tool_def_chars=0,
|
|
provenance=TurnProvenance(
|
|
model_alias=primary.alias,
|
|
backend_model_id=primary.model,
|
|
registry_generation=primary.registry_generation,
|
|
),
|
|
)
|
|
primary_ratio = session._chars_per_token
|
|
assert primary_ratio == 5.0
|
|
|
|
session.messages.extend([Turn.assistant("primary answer"), Turn.user("second")])
|
|
session._msg_tokens.extend([5, 2])
|
|
session._activate_token_calibration(fallback)
|
|
session._last_usage = {
|
|
"prompt_tokens": 25,
|
|
"completion_tokens": 7,
|
|
"total_tokens": 32,
|
|
}
|
|
session._update_token_table(
|
|
msgs=[{"role": "user", "content": "b" * 46}],
|
|
tool_def_chars=0,
|
|
provenance=TurnProvenance(
|
|
model_alias=fallback.alias,
|
|
backend_model_id=fallback.model,
|
|
registry_generation=fallback.registry_generation,
|
|
),
|
|
)
|
|
assert session._chars_per_token == 2.0
|
|
|
|
session.messages.append(Turn.assistant("fallback answer"))
|
|
session._msg_tokens.append(7)
|
|
session._activate_token_calibration(primary)
|
|
|
|
assert session._chars_per_token == primary_ratio
|
|
expected = 50 + sum(session._msg_tokens[1:])
|
|
assert session._estimated_prompt_tokens() == expected
|
|
assert session._estimated_prompt_tokens() != 25 + sum(session._msg_tokens[3:])
|
|
|
|
|
|
def test_cancelled_partial_stamps_the_armed_fallback_lane_and_principal(
|
|
tmp_db: str,
|
|
) -> None:
|
|
"""A partial accepted on Stop is an assistant turn, not unattributed UI."""
|
|
session = make_session(
|
|
model_alias="primary",
|
|
registry_generation=3,
|
|
user_id="owner",
|
|
ui=RecordingUI(), # type: ignore[no-untyped-call]
|
|
)
|
|
_register_session_parent(session)
|
|
fallback_provider = seam_provider("unused", provider_name="fallback-provider")
|
|
fallback_lane = ModelLane(
|
|
provider=fallback_provider,
|
|
client=MagicMock(),
|
|
model="fallback-kernel",
|
|
alias="fallback-alias",
|
|
registry_generation=13,
|
|
capabilities=ModelCapabilities(),
|
|
)
|
|
|
|
def _cancel_from_fallback(consumer, *_args, **_kwargs):
|
|
consumer.begin_attempt(MagicMock(armed=True), None, fallback_lane)
|
|
consumer(StreamChunk(content_delta="partial answer"))
|
|
session.cancel()
|
|
raise GenerationCancelled()
|
|
|
|
session._title_generated = True
|
|
with patch.object(
|
|
session,
|
|
"_model_turn_with_fallback",
|
|
side_effect=_cancel_from_fallback,
|
|
):
|
|
session.send("hello", acting_user_id="user-alice")
|
|
|
|
assert _provenance(session.messages[-1]) == {
|
|
"model_alias": "fallback-alias",
|
|
"backend_model_id": "fallback-kernel",
|
|
"registry_generation": 13,
|
|
"acting_principal_id": "user-alice",
|
|
}
|
|
from turnstone.core.storage import get_storage
|
|
|
|
storage = get_storage()
|
|
assert storage is not None
|
|
durable = storage.load_message_turns(session.ws_id, checkpointed=False)
|
|
assert _provenance(durable[-1]) == _provenance(session.messages[-1])
|
|
|
|
|
|
def test_provenance_dict_bridge_is_strict_and_lossless() -> None:
|
|
raw = {
|
|
"model_alias": "main",
|
|
"backend_model_id": "kernel",
|
|
"registry_generation": 8,
|
|
"acting_principal_id": "alice",
|
|
}
|
|
msg = {"role": "assistant", "content": "ok", "_provenance": raw}
|
|
turn = turn_from_dict(msg)
|
|
assert _provenance(turn) == raw
|
|
assert turn_to_dict(turn) == msg
|
|
|
|
torn = turn_from_dict(
|
|
{
|
|
"role": "assistant",
|
|
"content": "bad",
|
|
"_provenance": {**raw, "registry_generation": "8"},
|
|
}
|
|
)
|
|
assert PROVENANCE_META_KEY not in torn.meta.extra
|
|
assert "_provenance" not in turn_to_dict(torn)
|
|
|
|
|
|
def test_storage_rehydrates_and_fork_preserves_assistant_provenance(
|
|
storage_backend: StorageBackend,
|
|
) -> None:
|
|
raw = {
|
|
"model_alias": "main",
|
|
"backend_model_id": "kernel",
|
|
"registry_generation": 8,
|
|
"acting_principal_id": "alice",
|
|
}
|
|
ws_id = "provenance-storage-roundtrip"
|
|
storage_backend.save_message(
|
|
ws_id,
|
|
"assistant",
|
|
"accepted",
|
|
meta=json.dumps({PROVENANCE_META_KEY: raw}),
|
|
)
|
|
|
|
turns = storage_backend.load_message_turns(ws_id, checkpointed=False)
|
|
assert len(turns) == 1
|
|
assert _provenance(turns[0]) == raw
|
|
loaded = storage_backend.load_messages(ws_id, repair=False)
|
|
assert loaded[0]["_provenance"] == raw
|
|
|
|
insert_row, attachment_ids = _fork_turn_insert_row(
|
|
turns[0],
|
|
"provenance-fork-destination",
|
|
"2026-08-09T00:00:00",
|
|
)
|
|
assert attachment_ids == []
|
|
assert json.loads(insert_row["meta"])[PROVENANCE_META_KEY] == raw
|
|
|
|
|
|
def test_storage_rehydrates_and_fork_preserves_tool_acting_principal(
|
|
storage_backend: StorageBackend,
|
|
) -> None:
|
|
"""A TOOL row answers "whose turn ran this effect" without a join.
|
|
|
|
Revocation and audit ask that question per effect. Deriving it from the
|
|
assistant row that opened the batch is a join that breaks exactly where it
|
|
matters — a cancelled batch whose receipts are synthesized separately — so
|
|
the row carries the identity itself, as a sibling of its disposition and
|
|
never as a fabricated four-axis model-attempt envelope.
|
|
"""
|
|
ws_id = "tool-principal-storage-roundtrip"
|
|
storage_backend.save_message(
|
|
ws_id,
|
|
"tool",
|
|
"probe output",
|
|
"read_only_probe",
|
|
tool_call_id="call-audit",
|
|
meta=json.dumps({"effect_status": "committed", "acting_principal": "user-alice"}),
|
|
)
|
|
|
|
turns = storage_backend.load_message_turns(ws_id, checkpointed=False)
|
|
assert len(turns) == 1
|
|
assert turns[0].effect_status is EffectStatus.COMMITTED
|
|
assert turns[0].meta.extra["acting_principal"] == "user-alice"
|
|
assert PROVENANCE_META_KEY not in turns[0].meta.extra
|
|
|
|
insert_row, attachment_ids = _fork_turn_insert_row(
|
|
turns[0],
|
|
"tool-principal-fork-destination",
|
|
"2026-08-10T00:00:00",
|
|
)
|
|
assert attachment_ids == []
|
|
assert json.loads(insert_row["meta"]) == {
|
|
"effect_status": "committed",
|
|
"acting_principal": "user-alice",
|
|
}
|
|
|
|
|
|
def test_tool_acting_principal_reaches_no_public_or_model_facing_payload(
|
|
storage_backend: StorageBackend,
|
|
) -> None:
|
|
"""The audit identity has no dict-bridge key, so no projection can carry it.
|
|
|
|
A principal-only envelope is the adversarial shape: were the tool branch in
|
|
``reconstruct_turns`` not to claim it, it would fall through to
|
|
``source_meta`` — which /history publishes as a turn's display ``meta``.
|
|
"""
|
|
ws_id = "tool-principal-private"
|
|
storage_backend.save_message(
|
|
ws_id,
|
|
"tool",
|
|
"probe output",
|
|
"read_only_probe",
|
|
tool_call_id="call-audit",
|
|
meta=json.dumps({"acting_principal": "private-user-id"}),
|
|
)
|
|
|
|
turns = storage_backend.load_message_turns(ws_id, checkpointed=False)
|
|
assert turns[0].meta.extra["acting_principal"] == "private-user-id"
|
|
assert "private-user-id" not in json.dumps(turn_to_dict(turns[0]))
|
|
|
|
loaded = storage_backend.load_messages(ws_id, repair=False)
|
|
assert "_source_meta" not in loaded[0]
|
|
assert "private-user-id" not in json.dumps(loaded)
|
|
|
|
history = project_history_messages(loaded)
|
|
assert "meta" not in history[0]
|
|
assert "private-user-id" not in json.dumps(history)
|
|
assert "private-user-id" not in json.dumps(_serialize_messages(loaded))
|
|
assert "private-user-id" not in json.dumps(
|
|
_serialize_messages(loaded, include_provider_content=True)
|
|
)
|
|
|
|
storage = MagicMock()
|
|
storage.get_workstream.return_value = {"state": "idle"}
|
|
storage.load_message_turns.return_value = turns
|
|
storage.get_attachments.return_value = []
|
|
assert b"private-user-id" not in export_workstream(storage, ws_id).data
|
|
|
|
|
|
def test_provider_bound_wire_never_carries_the_tool_acting_principal() -> None:
|
|
"""Sidecar meta stays sidecar: the lowered wire has no field for it."""
|
|
provider = seam_provider("done")
|
|
lane = ModelLane(
|
|
provider=provider,
|
|
client=MagicMock(),
|
|
model="backend-v2",
|
|
alias="assistant-fast",
|
|
registry_generation=17,
|
|
capabilities=ModelCapabilities(),
|
|
)
|
|
tool_turn = Turn.tool("call-audit", "probe output", effect_status=EffectStatus.COMMITTED)
|
|
tool_turn.meta.extra["acting_principal"] = "private-user-id"
|
|
|
|
model_turn(
|
|
lane,
|
|
[
|
|
Turn.user("run the probe"),
|
|
turn_from_dict(
|
|
{
|
|
"role": "assistant",
|
|
"tool_calls": [
|
|
{
|
|
"id": "call-audit",
|
|
"function": {"name": "read_only_probe", "arguments": "{}"},
|
|
}
|
|
],
|
|
}
|
|
),
|
|
tool_turn,
|
|
],
|
|
acting_principal_id="private-user-id",
|
|
)
|
|
|
|
wire = provider.create_streaming.call_args.kwargs["messages"]
|
|
assert any(message.get("role") == "tool" for message in wire)
|
|
assert "private-user-id" not in json.dumps(wire)
|
|
assert "acting_principal" not in json.dumps(wire)
|
|
|
|
|
|
def test_pending_and_ambiguous_ack_keep_one_exact_provenance_tuple(tmp_db: str) -> None:
|
|
"""A lost ACK cannot relabel or duplicate the accepted assistant row."""
|
|
session = make_session(
|
|
model_alias="main",
|
|
registry_generation=5,
|
|
user_id="owner",
|
|
ui=RecordingUI(), # type: ignore[no-untyped-call]
|
|
)
|
|
arm_session(session, _good_stream("accepted"))
|
|
durable_rows: list[dict[str, Any]] = []
|
|
ids_by_commit: dict[tuple[str, str], int] = {}
|
|
assistant_attempts = 0
|
|
|
|
def _ambiguous_save(ws_id: str, role: str, content: str, **kwargs: object) -> int:
|
|
nonlocal assistant_attempts
|
|
commit_key = kwargs.get("commit_key")
|
|
# Participant-join operator context is intentionally outside the
|
|
# conversation-row journal. It may precede the keyed user turn in a
|
|
# shared workstream and is irrelevant to this lost-ACK seam.
|
|
if not isinstance(commit_key, str) or not commit_key:
|
|
return 1
|
|
identity = (ws_id, commit_key)
|
|
if identity not in ids_by_commit:
|
|
ids_by_commit[identity] = len(ids_by_commit) + 1
|
|
row: dict[str, Any] = {
|
|
"role": role,
|
|
"content": content,
|
|
"_commit_key": commit_key,
|
|
}
|
|
raw_meta = kwargs.get("meta")
|
|
if role == "assistant":
|
|
assert isinstance(raw_meta, str)
|
|
row["_provenance"] = json.loads(raw_meta)[PROVENANCE_META_KEY]
|
|
durable_rows.append(row)
|
|
if role == "assistant":
|
|
assistant_attempts += 1
|
|
return 0
|
|
return ids_by_commit[identity]
|
|
|
|
with (
|
|
patch("turnstone.core.session.save_message", side_effect=_ambiguous_save),
|
|
pytest.raises(ConversationPersistenceError),
|
|
):
|
|
session.send("hello", acting_user_id="alice")
|
|
|
|
expected = {
|
|
"model_alias": "main",
|
|
"backend_model_id": "test-model",
|
|
"registry_generation": 5,
|
|
"acting_principal_id": "alice",
|
|
}
|
|
assert assistant_attempts == 1
|
|
physical_assistants = [row for row in durable_rows if row["role"] == "assistant"]
|
|
assert len(physical_assistants) == 1
|
|
assert physical_assistants[0]["_provenance"] == expected
|
|
|
|
# A storage outage leaves the immutable pending projection authoritative.
|
|
pending, _token = session.capture_history_handoff(lambda _overscan: [])
|
|
pending_assistants = [row for row in pending if row.get("role") == "assistant"]
|
|
assert len(pending_assistants) == 1
|
|
assert pending_assistants[0]["_provenance"] == expected
|
|
|
|
# Once the durable keyed row is visible it reconciles in place, still once
|
|
# and with the exact same request-time identity.
|
|
reconciled, _token = session.capture_history_handoff(lambda _overscan: durable_rows)
|
|
reconciled_assistants = [row for row in reconciled if row.get("role") == "assistant"]
|
|
assert len(reconciled_assistants) == 1
|
|
assert reconciled_assistants[0]["_provenance"] == expected
|
|
|
|
|
|
def test_public_and_model_facing_projections_do_not_expose_principal() -> None:
|
|
provenance = TurnProvenance(
|
|
model_alias="main",
|
|
backend_model_id="kernel",
|
|
registry_generation=8,
|
|
acting_principal_id="private-user-id",
|
|
)
|
|
turn = Turn.assistant("accepted")
|
|
turn.meta.extra[PROVENANCE_META_KEY] = provenance.to_meta()
|
|
internal = turn_to_dict(turn)
|
|
|
|
history = project_history_messages([internal])
|
|
assert history == [{"role": "assistant", "content": "accepted"}]
|
|
assert "private-user-id" not in json.dumps(history)
|
|
assert "_provenance" not in _serialize_messages([internal])[0]
|
|
assert "_provenance" not in _serialize_messages([internal], include_provider_content=True)[0]
|
|
|
|
storage = MagicMock()
|
|
storage.get_workstream.return_value = {"state": "idle"}
|
|
storage.load_message_turns.return_value = [turn]
|
|
storage.get_attachments.return_value = []
|
|
exported = export_workstream(storage, "ws")
|
|
payload = json.loads(exported.data)
|
|
assert payload == {"messages": [{"role": "assistant", "content": "accepted"}]}
|
|
assert b"private-user-id" not in exported.data
|
|
|
|
|
|
def test_history_projection_failure_returns_503_without_private_row_fields(
|
|
tmp_db: str,
|
|
) -> None:
|
|
"""No public decoration/projection failure can authorize raw history."""
|
|
from tests._coord_test_helpers import _fake_registry
|
|
from tests.test_coordinator_endpoints import (
|
|
_COORD_HEADERS,
|
|
_build_history_mgr,
|
|
_make_client,
|
|
)
|
|
from turnstone.core.storage import get_storage
|
|
|
|
storage = get_storage()
|
|
assert storage is not None
|
|
# A 200 history response is authoritative only when a concrete live
|
|
# session captures the durable rows and returns a handoff token. Use the
|
|
# endpoint suite's token-bearing session fixture; the generic coordinator
|
|
# stub intentionally has no history-handoff protocol and must now fail
|
|
# closed with 503.
|
|
manager = _build_history_mgr(storage)
|
|
workstream = manager.create(user_id="user-1")
|
|
private_principal = "PRIVATE-PRINCIPAL-SENTINEL"
|
|
private_reasoning = "PRIVATE-REASONING-SENTINEL"
|
|
private_signature = "PRIVATE-SIGNATURE-SENTINEL"
|
|
private_producer = "PRIVATE-PRODUCER-SENTINEL"
|
|
private_commit_key = "PRIVATE-COMMIT-KEY-SENTINEL"
|
|
raw = TurnProvenance(
|
|
model_alias="main",
|
|
backend_model_id="kernel",
|
|
registry_generation=8,
|
|
acting_principal_id=private_principal,
|
|
).to_meta()
|
|
storage.save_message(
|
|
workstream.id,
|
|
"assistant",
|
|
"accepted",
|
|
provider_data=json.dumps(
|
|
{
|
|
"producer": private_producer,
|
|
"blocks": [
|
|
{
|
|
"type": "reasoning_text",
|
|
"text": private_reasoning,
|
|
"signature": private_signature,
|
|
}
|
|
],
|
|
}
|
|
),
|
|
meta=json.dumps({PROVENANCE_META_KEY: raw}),
|
|
commit_key=private_commit_key,
|
|
)
|
|
client = _make_client(storage, coord_mgr=manager, registry=_fake_registry())
|
|
|
|
for projection_seam in (
|
|
"decorate_history_messages",
|
|
"extract_reasoning_for_history",
|
|
"project_history_messages",
|
|
):
|
|
with patch(
|
|
f"turnstone.core.history_decoration.{projection_seam}",
|
|
side_effect=RuntimeError("public projection unavailable"),
|
|
):
|
|
response = client.get(
|
|
f"/v1/api/workstreams/{workstream.id}/history",
|
|
headers=_COORD_HEADERS,
|
|
)
|
|
|
|
assert response.status_code == 503
|
|
assert response.json() == {"error": "History temporarily unavailable"}
|
|
assert "messages" not in response.json()
|
|
assert "cursor" not in response.json()
|
|
assert "handoff_token" not in response.json()
|
|
for private_value in (
|
|
private_principal,
|
|
private_reasoning,
|
|
private_signature,
|
|
private_producer,
|
|
private_commit_key,
|
|
"_provider_content",
|
|
"_producer",
|
|
"_provenance",
|
|
"_commit_key",
|
|
):
|
|
assert private_value not in response.text
|