Commit Graph

90 Commits

Author SHA1 Message Date
Vincent Koc c30322418e fix(openai): share GPT-Live browser broker ownership 2026-07-31 16:00:31 +08:00
synthclaw b9e2d9e81b fix(openai): restrict GPT-Live broker cleanup to plugin disable only
Root cause: the OpenAI plugin registered the process-wide Quicksilver
browser session broker cleanup as an unconditional runtime lifecycle
callback. Session reset/delete/restart cleanup would permanently stop
the shared broker, while talk.catalog continued to report the provider
as ready. All later GPT-Live browser session reservations failed until
process restart.

Fix: the cleanup callback now checks ctx.reason and only tears down
the broker when reason === "disable". Session reset/delete/restart
cleanup leaves the broker running.

Regression test: verifies cleanup is a no-op for reset/delete/restart
and returns a promise for disable.

Fixes openclaw/openclaw#116525
2026-07-31 00:04:45 -04:00
Peter Steinberger f78ba09120 feat(talk): support GPT-Live realtime voice over ChatGPT OAuth (#115226)
* feat(gateway): share the canonical browser-origin policy with plugins

Export resolveAcceptedBrowserOrigin through openclaw/plugin-sdk/webhook-request-guards
so browser-facing plugin routes reuse the Gateway's real origin contract instead of a
narrow allowedOrigins array check. Private LAN/Tailnet Control UI loads and the
Host-header fallback were previously rejected with 403 by plugin offer routes while the
Control UI itself worked.

Moves the loopback/forwarded-header helpers to net.ts (re-exported from auth.ts) so the
guard can delegate without importing gateway auth, and migrates the Codex realtime
broker onto the shared seam.

* feat(talk): let providers own agent delegation for realtime voice

Adds an optional runAgentConsult callback to the browser-session create request and
injects the existing embedded consult runtime from talk-client, bound to the same agent
and session key the GA tool path uses. Providers whose realtime protocol delegates work
through their own control channel (rather than GA function calls) can now reach the
OpenClaw agent without a client round-trip.

Threads the effective per-session model into browser-session capability resolution so a
request-level model override selects the right capability set, and propagates a caller
abort signal into consultRealtimeVoiceAgent so a superseded delegation stops its run.

* feat(openai): support GPT-Live realtime voice over ChatGPT OAuth

Implements OpenAI's quicksilver/frameless session natively for Talk browser sessions.
The Gateway creates the WebRTC call (multipart sdp+session to https://api.openai.com/v1/live)
and owns the sideband control socket, so the browser never holds upstream credentials;
delegation.created events run through the OpenClaw agent and stream back as speakable
context appends.

Verified end-to-end on 2026-07-28 against a ChatGPT Pro OAuth profile: call create 201
with an rtc_* id and answer SDP, sideband session.started, session.close teardown.
ChatGPT OAuth is preferred over a Platform API key because /v1/live access for platform
keys is waitlist-gated; the legacy chatgpt.com backend route returns 403 for every model
and protocol version and is not used.

Accepted models are gpt-live-1-codex and gpt-live-1-boulder-alpha; the voice allowlist is
the ten values the route actually accepts, since an invalid voice is rejected at call
creation and cannot be repaired afterwards (session.update reports immutable_field_update).

* docs: document GPT-Live Talk support and its route gotchas

Records the working route and auth, the accepted models and voices, the browser-only
scope, and the two traps that cost the most time: the chatgpt.com backend route returns
403 Voice session access denied for every model, and that same 403 is also what an
invalid voice returns, so it must not be read as an account entitlement block.

* fix(openai): resolve GPT-Live CI failures

* refactor(openai): own zod runtime dependency

* fix(openai): satisfy lint and live-shard gates for GPT-Live

Types the retry-delay finish callback as Error so the rejection reason is provably an
Error at the call site; the abort path already normalized a non-Error AbortSignal reason,
but the unknown parameter type hid that from static analysis.

Registers the new GPT-Live live test in the native-live-extensions-openai shard
expectation. The shard selector already picked the file up from the real tree; only the
hardcoded list in the tooling test lagged.

* fix(openai): clean up post-rebase capability resolver

* fix(openai): preserve GPT-Live delegation fragments

* fix(openai): close GPT-Live sideband handoff race

* fix(openai): accept UUID GPT-Live call ids

* style: apply oxfmt to GPT-Live sources and Talk docs

* style: format Talk docs after rebase

* fix(openai): keep GPT-Live transcript context across ignored delegations
2026-07-28 15:05:55 -04:00
Peter Steinberger 42515c4f07 fix(openai): prevent realtime transcript turns from reordering (#113723)
* fix(openai): preserve realtime transcript order

* chore: defer realtime note to release

* test(openai): align heartbeat prompt contract

* test(openai): align direct context contract
2026-07-25 08:11:09 -07:00
Peter Steinberger e7cba0e4d5 fix: provider dead exports no longer block changed checks (#108592)
* fix(ci): clean provider dead exports

* test(extensions): satisfy provider contract types

* refactor(openai): tighten provider runtime factory
2026-07-15 22:33:54 -07:00
Peter Steinberger 81941f2d68 test: enable noUncheckedIndexedAccess for extension tests (#105343) 2026-07-12 12:48:22 +01:00
Peter Steinberger ce562c3511 test(openai): align prerelease contracts (#105256) 2026-07-12 10:49:48 +01:00
NIO 527f8f0cbb fix(image-gen): bound image generation provider JSON response reads (#96495)
* fix(image-gen): bound image generation provider JSON response reads

Route success JSON reads through readProviderJsonResponse (16 MiB cap)
in openrouter, google, fal, minimax, openai, and vydra image generation
providers to prevent OOM from oversized or hostile endpoint responses.
Mirrors the response-limit campaign already applied to other provider paths.

AI-assisted.

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix(image-gen): size bounded JSON caps for inline image payloads

Signed-off-by: sallyom <somalley@redhat.com>

---------

Signed-off-by: sallyom <somalley@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: sallyom <somalley@redhat.com>
2026-06-26 07:08:30 -04:00
Peter Steinberger 96e5812426 docs: document medium extension sources 2026-06-04 21:33:54 -04:00
Peter Steinberger 304e2c83c0 chore(lint): enable stricter oxlint rules 2026-05-31 18:59:02 +01:00
Peter Steinberger 4c33aaa86c refactor: unify OpenAI provider identity (#88451)
* refactor: unify OpenAI provider identity

* refactor: move legacy oauth sidecar doctor helpers

* test: align OpenAI fixtures after rebase

* test: clean OpenAI provider unification

* fix: finish OpenAI provider cleanup

* fix: finish OpenAI cleanup follow-through

* fix: finish OpenAI CI cleanup
2026-05-31 00:29:44 +01:00
Peter Steinberger d92b3b5cc2 refactor: unify OpenAI provider identity
Refactor OpenAI provider identity so OpenAI remains the canonical provider for API-key and OAuth-backed flows while legacy openai-codex state is doctor/migration-only.

Keeps OpenAI Codex Responses as an API/transport class rather than a provider identity, moves auth aliases through providerAuthAliases, updates doctor repair sequencing for old auth/profile state, and refreshes tests/docs around the canonical OpenAI behavior.
2026-05-30 11:48:41 +02:00
Peter Steinberger bb46b79d3c refactor: internalize OpenClaw agent runtime (#85341)
* refactor: extract agent core package

Introduce packages/agent-core as the OpenClaw-owned home for reusable agent loop, harness, session, prompt, and runtime dependency contracts.

* refactor: extract shared llm runtime

Move provider model registries, stream wrappers, OAuth helpers, and LLM utilities into src/llm with plugin-sdk barrels instead of depending on the old embedded runtime layout.

* refactor: remove pi runtime internals

Rename remaining Pi-shaped agent surfaces to OpenClaw agent runtime names, delete obsolete Pi docs and package graph checks, and add the third-party notice for incorporated code.

* refactor: tighten agent session runtime

Make agent-core/runtime dependencies explicit, consolidate compaction and session transcript helpers, and move model/session helpers behind OpenClaw-owned contracts.

* refactor: remove static model and pi auth paths

Drop static model catalogs and Pi auth bridges, move model/provider facts to manifest-owned runtime contracts, and harden internal embedded-agent utilities.

* refactor: remove legacy provider compat paths

* docs: remove agent parity notes

* fix: skip provider wildcard metadata parsing

* refactor: share session extension sdk loading

* refactor: inline acpx proxy error formatter

* refactor: fold edit recovery into edit tool

* fix: accept extension batch separator

* test: align startup provider plugin expectations

* fix: restore provider-scoped release discovery

* test: align static asset packaging expectations

* fix: run static provider catalogs during scoped discovery

* fix: add provider entry catalogs for scoped live discovery

* fix: load lightweight provider catalog entries

* fix: refresh provider-scoped plugin metadata

* fix: keep provider catalog entries on release live path

* fix: keep static manifest models in release live checks

* fix: harden release model discovery

* fix: reduce OpenAI live cache probe reasoning

* fix: disable OpenAI cache probe reasoning

* ci: extend OpenAI gateway live timeout

* fix: extend live gateway model budget

* fix: stabilize release validation regressions

* fix: honor provider aliases in model rows

* fix: stabilize release validation lanes

* fix: stabilize release memory qa

* ci: stabilize release validation lanes

* ci: prefer ipv4 for live docker node calls

* fix: restore shared tool-call stream wrapper

* ci: remove legacy pi test shard alias

* fix: clean up embedded agent test drift

* fix: stabilize runtime alias status

* fix: clean up embedded agent ci drift

* fix: restore release ci invariants

* fix: clean up post-rebase runtime drift

* fix: restore release ci checks

* fix: restore release ci after rebase

* fix: remove stale pi runtime path

* test: align compaction runtime expectations

* test: update plugin prerelease expectations

* fix: handle claude live tool approvals

* fix: stabilize release validation gates

* fix: finish agent runtime import

* test: finish post-rebase agent runtime mocks

* fix: keep codex compaction native

* fix: stabilize codex app-server hook tests

* test: isolate codex diagnostic active run

* test: remove codex diagnostic completion race

# Conflicts:
#	extensions/codex/src/app-server/run-attempt.test.ts

* ci: fix full release manifest performance run id

* refactor: narrow llm plugin sdk boundary

* chore: drop generated google boundary stamps

* fix: repair rebase fallout

* fix: clean up rebased runtime references

* fix: decode codex jwt payloads as base64url

* fix: preserve shipped pi runtime alias

* fix: add scoped sdk virtual modules

* fix: decode llm codex oauth jwt as base64url

* fix: avoid stale vertex adc negative cache

* fix: harden tool arg decoding and codeql path

* fix: keep vertex adc negative checks live

* refactor: consolidate codex jwt and edit helpers

* fix: await codex oauth node runtime imports

* fix: preserve sdk tool and notice contracts

* fix: preserve shipped compat config boundaries

* fix: align codex oauth callback host

* fix: terminate agent-core loop streams on failure

* fix: keep codex oauth callback alive during fallback

* ci: include session tools in critical codeql scans

* fix: keep Cloudflare Anthropic provider auth header

* docs: redirect legacy pi runtime pages

* fix: honor bundled web provider compat discovery

* fix: protect session output spill files

* fix: keep legacy agent dir env blocked

* fix: contain auto-discovered skill symlinks

* fix: harden agent core sdk proxy surfaces

* fix: restore approval reaction sdk compat

* fix: keep live docker runs bounded

* fix: keep codex oauth redirect host aligned

* fix: resolve post-rebase agent runtime drift

* fix: redact anthropic oauth parse failures

* fix: preserve responses strict tool shaping

* fix: repair agent runtime rebase cleanup

* docs: redirect retired parity pages

* fix: bound auto-discovered resources to roots

* fix: repair post-rebase agent test drift

* fix: preserve bundled provider allowlist migration

* fix: preserve manifest-owned provider aliases

* fix: declare photon image dependency

* fix: keep provider headers out of proxy body

* fix: preserve shipped env aliases

* fix: refresh control ui i18n generated state

* fix: quote read fallback paths

* fix: preview edits through configured backend

* test: satisfy core test typecheck

* fix: preserve ZAI usage auth fallback

* test: repair codex diagnostic test

* fix: repair agent runtime rebase drift

* test: finish embedded runner import rename

* fix: repair agent runtime rebase integrations

* test: align compaction oauth fallback expectations

* fix: allow sdk-auth session models

* fix: update doctor tool schema import

* fix: preserve bedrock plugin region

* fix: stream harmony-like prose immediately

* ci: include session runtime in codeql shards

* fix: repair latest rebase integrations

* fix: honor explicit codex websocket transport

* fix: keep openai-compatible credentials provider-scoped

* fix: refresh sdk api baseline after rebase

* fix: route cli runtime aliases through openclaw harness

* test: rename stale harness mock expectation

* test: rename embedded agent overflow calls

* test: clean embedded auth test wording

* test: use openclaw stream types in deepinfra cache test

* fix: refresh sdk api baseline on latest main

* fix: honor bundled discovery compat allowlists

* fix: refresh sdk api baseline after latest rebase

* fix: remove stale rebase imports

* test: rename stale model catalog mock

* test: mock renamed doctor runtime modules

* fix: map canonical kimi env auth

* fix: use internal model registry in bench script

* fix: migrate deepinfra provider catalog entry

* fix: enforce builtin tool suppression

* fix: route compaction auth and proxy payloads safely

* refactor: prune unused llm registry leftovers

* test: update codex hooks session import

* test: fix model picker ci coverage

* test: align model picker auth mock types
2026-05-27 19:24:04 +01:00
Peter Steinberger 4f4d108639 chore(lint): remove underscore-dangle allow list (#83542)
* chore(lint): reduce underscore-dangle exceptions

* chore(lint): reduce more underscore exceptions

* chore(lint): remove underscore-dangle allow list

* fix(lint): repair underscore cleanup regressions

* test(lint): track version define suppression
2026-05-18 14:56:06 +01:00
Peter Steinberger e17dfb0014 test: guard extension mock helpers 2026-05-11 19:19:27 +01:00
Peter Steinberger 15cf49222f build: refresh deps and route testbox through crabbox 2026-05-11 03:41:00 +01:00
Peter Steinberger e8103c0153 test: tighten openai provider assertions 2026-05-11 01:06:51 +01:00
Peter Steinberger d273ae73c0 refactor: trim OpenClaw prompt guidance 2026-05-10 14:59:48 +01:00
Peter Steinberger 827b0de0ce refactor: reduce plugin sdk surface 2026-05-10 12:37:10 +01:00
Shakker 84c4a4fc5b test: tighten provider empty array assertions 2026-05-09 04:55:18 +01:00
Peter Steinberger 1ef85c7d4c test: make suites safe without isolation (#78834)
* test: make suites safe without isolation

* fix: narrow auth profile credential types

* test: inject channel module loader factory locally
2026-05-07 08:43:29 +01:00
pashpashpash 5bcc321343 Scope Codex heartbeat guidance to heartbeat turns (#76788)
* fix(codex): scope heartbeat guidance to collaboration mode

* fix heartbeat tool direct context

* test prompt heartbeat collaboration snapshots

* fix heartbeat changelog credit
2026-05-04 01:58:39 +09:00
Peter Steinberger f34b41f198 refactor: split plugin sdk test helpers 2026-04-28 01:14:19 +01:00
Peter Steinberger 90b6665ded refactor: move plugin api test helper to sdk 2026-04-28 00:24:54 +01:00
Peter Steinberger 0df6e5a473 refactor: expose plugin test helpers via sdk 2026-04-27 23:45:26 +01:00
Peter Steinberger 4336a7f3a9 refactor(plugin-sdk): narrow config runtime imports 2026-04-27 14:58:32 +01:00
Peter Steinberger d613c8e29b refactor(tts): resolve voice delivery from channel capabilities 2026-04-26 07:03:25 +01:00
Peter Steinberger 648422a6c1 fix(openai): send image edits as multipart uploads (#70657) 2026-04-23 20:48:55 +01:00
Vincent Koc abb32e39b5 fix(openai): refresh live prompt overlay config 2026-04-22 23:37:16 -07:00
Peter Steinberger 834fdc4832 docs: align documentation with current surfaces 2026-04-23 07:25:06 +01:00
Peter Steinberger aa94501f5f feat(openai): default images to gpt-image-2 2026-04-21 21:49:16 +01:00
Peter Steinberger 0da5e0e34e fix(openai): tighten gpt prompt contract 2026-04-21 06:14:54 +01:00
Peter Steinberger 22bff819ab fix: strengthen agent completion bias 2026-04-21 04:19:26 +01:00
Peter Steinberger 525e66e513 fix(openai): use tagged GPT-5 prompt contract 2026-04-21 02:45:17 +01:00
Peter Steinberger ab03d4e037 fix(openai): default GPT-5 prompt overlay 2026-04-21 02:36:16 +01:00
Peter Steinberger 5a289f5cad test: share openai plugin fixtures 2026-04-20 21:28:25 +01:00
Ayaan Zaidi 5c1d6feb33 test(plugins): fix sync register call sites 2026-04-17 10:14:00 +05:30
Vincent Koc 3587e0ef95 fix(codex): keep auth read diagnostics off stdout (#66451)
* fix(codex): keep auth read diagnostics off stdout

* docs(changelog): fix codex auth entry

* fix(codex): sanitize auth read diagnostics

* Update CHANGELOG.md
2026-04-14 11:13:57 +01:00
pashpashpash c848ebc8ce agents: split GPT-5 prompt and retry behavior (#65597)
* agents: split GPT-5 prompt and retry behavior

* agents: fix GPT-5 review follow-ups

* agents: address GPT-5 review follow-ups

* agents: avoid replaying side-effectful GPT retries

* agents: mark subagent control as mutating

* agents: fail closed on single-action retries

* commands: stabilize channel legacy doctor migration test

* agents: narrow single-action retry promise trigger
2026-04-12 18:52:22 -07:00
pashpashpash f5447aab88 OpenAI: strengthen heartbeat overlay guidance (#65148) 2026-04-13 06:47:40 +09:00
pashpashpash 323e37c862 openai: add heartbeat guidance to GPT-5 overlay (#65069)
* openai: add heartbeat guidance to GPT-5 overlay

* openai: soften heartbeat overlay wording

* openai: tighten heartbeat overlay wording

* openai: align heartbeat silence with ack contract
2026-04-12 09:58:07 +09:00
Eva 6aa63b4fdd agents: add openai provider-owned tool compat 2026-04-11 01:27:31 +01:00
Peter Steinberger aa55ba6316 test: remove duplicate openai image response keys 2026-04-10 23:11:01 +01:00
Peter Steinberger 3f6af907f7 test(codex): cover app-server command seams 2026-04-10 23:07:25 +01:00
Peter Steinberger 70e128e559 test: remove duplicate openai final url stubs 2026-04-10 22:46:51 +01:00
Peter Steinberger 49ec2f15c3 test: include openai guarded fetch final url 2026-04-10 22:45:42 +01:00
Tak Hoffman 43b91c0ab3 test: satisfy openai postJsonRequest mock shape 2026-04-10 16:44:45 -05:00
Tak Hoffman 98be3ab6de test: harden openai image generation stub 2026-04-10 16:41:50 -05:00
Peter Steinberger 9663343183 test(ci): align openai image edit assertion 2026-04-06 19:25:35 +01:00
Vincent Koc e7fe087677 fix(openai): normalize prompt overlay personality config 2026-04-06 17:24:51 +01:00