mirror of
https://github.com/turnstonelabs/turnstone.git
synced 2026-08-12 23:12:23 -06:00
Patrick Buckley
7053439e84
turnstone-eval was misnamed: it was a prompt optimizer, not a measurement harness. Split the 3252-line turnstone/eval.py into a strictly one-way dependency (optimizer -> eval-core; core never imports the optimizer): - turnstone/eval/core.py measurement substrate — everything up to and including _run_iteration: provider detection, NullUI, HeadlessSession, the test runner, score_run, aggregation, and neutral reporting. - turnstone/eval/cli.py new measure-only `turnstone-eval` — the old --no-optimize path promoted to the whole job (one _run_iteration call, then print the summary table). - turnstone/optimizer.py the UCB self-modify loop and its multi-agent pipeline (analyst/optimizer/observer/diversifier/tool optimizer), now `turnstone-optimizer`; imports from eval.core only. - turnstone/eval/__init__.py re-exports the core public API for back-compat (score_run, _match_action, _run_iteration, HeadlessSession). _apply_tool_overrides lives in core (HeadlessSession needs it) rather than alongside the other tree helpers, so the dependency stays one-way. Breaking change: `turnstone-eval` now measures; use `turnstone-optimizer` to optimize. Both code paths are behaviour-preserving — the moved function bodies are byte-identical.