Commit Graph

4 Commits

Author SHA1 Message Date
Patrick Buckley 9a30530d41 feat(coord): surface child errors, isolate tool exceptions, add memory tool (#443)
* feat(coord): surface child errors, isolate tool exceptions, add memory tool

Closes four coordinator gaps identified during operator triage:

1. Child workstream errors now surface in inspect/wait. Worker-thread
   exception text is sanitized (URL userinfo masked, sk-/Bearer/ghp_/
   github_pat_/AKIA tokens redacted, capped at 1024 chars) and persisted
   to workstream_config.last_error before _emit_state("error") fires, so
   coord polling never sees state=error with a missing cause. The row
   is cleared on recovery transitions (idle/running) so a once-leaked
   exception body doesn't outlive the failure. inspect_workstream and
   wait_for_workstream return last_error for state=error rows; the
   wait surface prefers it over the assistant-tail walk.

2. Tool exceptions now return as tool_results with sibling-aware
   guidance. ChatSession._safe_prepare_tool wraps every per-call
   _prepare_tool invocation; a buggy preparer becomes an error item
   for that call only — sibling parallel tool_calls keep going,
   never orphaning the assistant message's tool_calls block.
   run_one's runtime exception path includes the exception class
   and a short note that other tool calls in the batch completed
   independently so the model can recover.

3. Memory tool exposed to coordinator with a coord-only scope.
   memory.json gains coordinator: true + interactive: true + per-kind
   kind_variants. Coord sessions see scope enum ["coordinator"] and
   an orchestration-flavored description; IC sessions see ["global",
   "workstream", "user"] and the existing flavor. Coord-scope rows
   are private to the coordinator session (children cannot read or
   write them), closing the cross-session prompt-injection lane that
   an adversarially-steered child would otherwise have. Coord
   visibility is also restricted to coord-scope only — coords no
   longer see global / workstream / user memories that belong to the
   user's interactive sessions.

4. Per-call exception isolation in tool batches. _safe_prepare_tool
   was previously the implicit shield; now it's an explicit method
   with documented invariants. KeyboardInterrupt / GenerationCancelled
   re-raise so the cooperative cancel path still works.

Other notable changes:
- LAST_ERROR_CONFIG_KEY + persist_last_error / clear_last_error /
  load_last_error / sanitize_error_text moved to turnstone.core.memory
  (the storage facade hub) — readers in coordinator_client.py import
  the constant.
- Memory scope tuples extracted to module constants
  _VALID_MEMORY_SCOPES and _IMPLICIT_SCOPE_WALK; seven inline
  duplicates collapsed.
- tools.py grows _apply_kind_variant for the per-kind tool surface;
  tools without kind_variants pass through unchanged (no spurious
  deep-copies).
- Session adds _coordinator_scope_id, _default_memory_scope,
  _implicit_scope_walk, and _record_fatal_error chokepoints so the
  worker-thread fatal path is one site rather than three.
- Removed duplicate on_error / on_state_change emits from
  session_routes.py and coordinator_adapter.py — session.send()'s
  _record_fatal_error owns the sequence now.

Tests: 4742 pass (no live), +30 net since the baseline. Ruff + mypy
clean on every modified production file.

* fix(coord): redact secrets in tool error paths via output_guard

Copilot review flagged two paths where ``str(exc)`` flowed back into
the model-facing tool_result without going through the credential-
redaction the new fatal-error path applies:

  - ``ChatSession._safe_prepare_tool``: a preparer-side exception
    becomes an error item whose ``error`` field embedded the raw
    exception text.
  - ``ChatSession._execute_tools.run_one``: a runtime tool exception
    became an ``Error executing X: <e>`` tool_result, again with
    the raw exception text.

Both now route through ``sanitize_error_text`` (sanitised log line +
sanitised tool_result), and ``sanitize_error_text`` itself was
refactored to delegate to ``output_guard.redact_credentials`` instead
of carrying its own parallel regex catalog — the audit log + post-tool
guard already use that pattern set, so the credential definition
stays in one place.

Also extended ``_RE_CONNECTION_STRING`` in ``output_guard`` to cover
``http(s)://user:pass@host`` so a misconfigured ``OPENAI_BASE_URL``
that lands in an httpx ``ConnectError.__str__`` is redacted by every
caller of ``redact_credentials`` (audit details, close-reason
persistence, last_error, the two tool error paths).  The
host (useful for triage) survives; only the password is replaced
with the standard ``[REDACTED:password]`` marker.

Tests: full suite (4745 pass), ruff + mypy clean.  Two new tests pin
the redaction behaviour in both tool error paths so a future refactor
can't drift back to leaking ``str(exc)`` verbatim.
2026-04-28 12:19:34 -07:00
Patrick Buckley 405baf7cb2 fix: memory list/search cross-workstream scope leak (#253)
* fix: scope-filter memory list/search to current workstream and user

Unscoped memory(action='list') and memory(action='search') returned all
memories across all workstreams. Now applies the same 3-query pattern
(global + current workstream + current user) used by system prompt
injection.

* fix: validate user scope on memory search/list for unauthenticated sessions

Adds _validate_scope guard to search and list prepare paths, matching
save/get/delete. Prevents explicit scope='user' from returning all
user-scoped memories when session is unauthenticated.

* fix: update _get_visible_memories references to _list_visible_memories

* fix: defense-in-depth guard for empty scope_id on search/list

Copilot review: if scope is 'user' or 'workstream' with empty
scope_id, the storage query returns all memories in that scope
across all users/workstreams.  The prepare step already validates
via _validate_scope, but add exec-level guard to reject scoped
queries with empty scope_id as defense-in-depth.
2026-03-30 04:43:49 -07:00
Patrick Buckley 8d88e6a7eb feat: add memory get action, reduce search/list preview to 200 chars (#247)
* feat: add memory get action, reduce search/list preview to 200 chars

search and list truncated memory content to 500 chars with no way to
read the full value.  Two changes:

- New 'get' action retrieves a single memory by name with complete
  untruncated content.  Searches scopes narrowest-first (workstream
  → user → global).
- search/list previews reduced from 500 to 200 chars now that get
  exists for full content.  Both append a hint:
  "Use memory(action='get', name='...') for full content."

Includes get_structured_memory_by_name wrapper in memory.py and
4 tests.

* Update turnstone/tools/memory.json

* fix: include 'get' in _prepare_memory docstring and invalid-action error
2026-03-29 20:53:48 -07:00
Patrick Buckley 723cad24bb feat: structured memory system — typed/scoped memories with BM25 rele… (#53)
* feat: structured memory system — typed/scoped memories with BM25 relevance and metacognitive prompting

Replace flat key-value memories table with structured_memories (migration 014).
Four memory types (user/project/feedback/reference), three scopes
(global/workstream/user). Consolidate remember/recall/forget into two tools:
memory (action-based: save/search/delete/list) and recall (conversation
history only).

BM25 relevance scoring (extracted to turnstone/core/bm25.py) selects top-5
memories for system message injection based on conversation context.
Metacognitive prompting injects ephemeral nudges after corrections, tool
denials, workstream resume, and completion signals.

Scope isolation enforced: system message injection and nudge counts filtered
to visible memories only (global + current workstream + authenticated user).
User scope requires authentication. Content capped at 32KB. ILIKE/LIKE
metacharacters escaped in both backends.

113 new tests (2053 total).

* fix: CI failure + copilot review feedback

- Fix time.monotonic() cooldown: use None sentinel instead of 0.0
  default (monotonic clock starts at boot, not epoch — fresh CI
  runners have uptime < 300s so cooldown check always triggered)
- Catch sa.exc.IntegrityError specifically in upsert instead of
  broad Exception (copilot review)
- Preserve existing description/type on upsert when caller doesn't
  explicitly set them (copilot review)
- Add last_accessed + access_count columns to schema/migration for
  future LRU/LFU eviction support
2026-03-13 21:21:09 -07:00