Operator-context system turns (watch results, output-guard findings, idle
children, user interjections) carried their kind (_source) and a flattened
text content, but the structured per-kind fields were dropped at every
persist/deliver boundary — so the UI rendered every kind as one generic
operator bubble and the structured watch-result card was lost.
Wire the structured meta through as the single source of truth:
- Storage: new conversations.meta JSON column (migration 060); threaded
through save_message/save_messages_bulk (facade + protocol + both backends)
and rehydrated in reconstruct_turns onto Turn.meta.extra["source_meta"].
- Canonical: make_system_turn carries meta as one _source_meta dict;
turn_from_dict/turn_to_dict bridge it to/from Turn.meta.extra.
- Live + history: widen on_system_turn(content, source, meta) across all
impls + the SSE payload; surface _source_meta -> meta in the /history
projection. SDK HistoryEvent docs note the field.
- Producers derive both the model-facing content text AND the card from one
meta dict, so they cannot drift: render_output_guard_text, build_watch_
reminder carrying output, idle_children and user_interjection metadata.
- Frontend: addSystemContext / renderSystemTurn dispatch by source to the
watch-result, guard-finding, idle-children, and queued-message cards in
both the interactive and coordinator panes; every untrusted field renders
via textContent.
The meta is a leading-underscore key, stripped before the wire (sanitize_
messages and the native mid-conversation path copy only role+content), so the
per-provider wire payloads stay byte-identical. Additive column, no backfill:
operator turns predating it reload as plain text bubbles.
The by-reference content lane now materializes at the provider translator (the
C layer), not in the session. Each create_streaming / create_completion takes
a resolve_attachments callback and runs materialize_attachments() up front,
expanding {type:kind, attachment_id} placeholders to inline data-URI / document
parts by a content-addressed point-lookup the session hands down
(_resolve_attachments). _full_messages emits placeholders; the dict bridge
carries only placeholders.
RawContentBlock is removed — ContentBlock = TextBlock | AttachmentRef. A
resolved inline part is terminal (the wire payload / display output) and never
re-enters the canonical path, so turn_from_dict drops a stray inline image_url
rather than carrying bytes. resolve_attachment_parts / materialize_attachments
operate on the dict projection. Tool vision output rides by reference too
(_tool_content_by_reference): the turn carries placeholders, the bytes persist
content-addressed. The per-turn token estimate counts a by-ref image as one
fixed image budget; the document char budget lands at send (on resolution).
Wire harness byte-identical (the multipart fixture is a placeholder + a matching
resolver); full non-live suite green (7136).
Non-text content (user uploads, reloaded tool images) rides as AttachmentRef(id,kind)
in the canonical Turn — session.messages carries ids, never bytes. Each output
materializes it to inline data-URI/document parts by point-lookup on the content-
addressed store: the wire (ChatSession._lower_messages_to_wire, in _full_messages),
/history + export (reconstruct_messages resolves), and the per-turn token estimate
(a by-ref image costs one fixed image budget; the doc char budget lands at send).
reconstruct splits: reconstruct_turns = unresolved row→Turn (load_message_turns, the
resume path); reconstruct_messages = resolved dict facade. RawContentBlock is demoted
to the transient carrier for a resolved inline part on the dict↔Turn bridge.
ChatSession.messages flips from list[dict] to list[Turn] — the in-memory
canonical trajectory. Reads migrate to typed fields (turn.role, turn.text,
turn.tool_calls); appends and assignments go through turn_from_dict /
turns_from_dicts; the fork bulk-save and retry's multipart check read via
turn_to_dict. _full_messages lowers Turns→dicts at the wire boundary — the
fold/repair and provider translators still consume dicts until the next slice.
The token-accounting helpers accept a dict or a Turn.
Non-session consumers migrate too: coordinator_idle_observer and eval to typed
fields (mypy-enumerated), and server's last-assistant extractor via turn_to_dict
(an Any-typed call site mypy could not flag). An all-text multipart content
list (the unreadable-attachment placeholder path) now round-trips faithfully
through the adapter (single text block → str, multiple → list).
Tests that inspected session.messages as dicts read it through the
dicts_from_turns / turn_to_dict bridge; those that built it pass dicts through
turns_from_dicts / turn_from_dict. Byte-identical wire harness; full non-live
suite green (7130).
turn_from_dict / turn_to_dict losslessly bridge the OpenAI-like message dict
(plus its _-prefixed side channels) and the typed Turn, so the migration to
Turn can proceed one boundary at a time: a dict-producing layer can be read as
Turns, and a Turn-holding layer can hand dicts to a not-yet-migrated consumer.
The _ side channels become typed fields: _source→source, _provider_content
(+_producer)→native, _event_id→meta.event_id, _attachments_meta→meta.extra.
A transitional RawContentBlock carries image/document parts verbatim until the
by-reference AttachmentRef wiring (§2/§6) relocates byte-resolution to the
translator; text parts become TextBlock so .text/FTS stays faithful.
turn_to_dict(turn_from_dict(d)) == d for every shape reconstruct and the wire
path emit (test_trajectory). No consumers yet — wiring is the next slices.
Patrick Buckley