Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled

Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11),
free to diverge. Tree copied sans upstream .git; upstream remote added for
future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19.
Preserves docs/ARCHITECTURE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
2026-07-05 09:36:54 +00:00
parent 3216769225
commit bedb527145
21108 changed files with 6010766 additions and 0 deletions

21
packages/ai/LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 OpenClaw Foundation
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

28
packages/ai/README.md Normal file
View File

@@ -0,0 +1,28 @@
# `@openclaw/ai`
Reusable model API contracts, provider adapters, and streaming primitives from
OpenClaw. The package supports isolated runtime instances; importing it does not
register providers globally.
```ts
import { createLlmRuntime } from "@openclaw/ai";
import { registerBuiltInApiProviders } from "@openclaw/ai/providers";
const runtime = createLlmRuntime();
registerBuiltInApiProviders(runtime.registry);
```
Provider-neutral contracts, validation, diagnostics, and event streams are
available from the package root and focused subpaths such as
`@openclaw/ai/event-stream` and `@openclaw/ai/validation`. No second OpenClaw
runtime package is required.
Provider ids, credentials, model catalogs, retries, and failover remain
application concerns. OpenClaw supplies those policies around this package.
Host policy (request fetch guarding, secret redaction, strict-tool defaults,
diagnostics logging) can be injected with `configureAiTransportHost`; the
defaults are inert.
`@openclaw/ai/internal/*` subpaths exist for the OpenClaw application itself.
They carry no semver guarantee and can change or disappear in any release; do
not depend on them outside OpenClaw.

645
packages/ai/npm-shrinkwrap.json generated Normal file
View File

@@ -0,0 +1,645 @@
{
"name": "@openclaw/ai",
"version": "2026.6.11",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@openclaw/ai",
"version": "2026.6.11",
"license": "MIT",
"dependencies": {
"@anthropic-ai/sdk": "0.109.1",
"@google/genai": "2.10.0",
"@mistralai/mistralai": "2.4.0",
"openai": "6.45.0",
"partial-json": "0.1.7",
"typebox": "1.3.3"
},
"engines": {
"node": ">=22.19.0"
}
},
"node_modules/@anthropic-ai/sdk": {
"version": "0.109.1",
"resolved": "https://registry.npmjs.org/@anthropic-ai/sdk/-/sdk-0.109.1.tgz",
"integrity": "sha512-q9OnEKLr5H9nxSuXdgDgJhxfYMiE+AaUEBze2Gk91UcaaLnsN+Lx5fbCYywiqurU/APLdwv23x03Wm6WN3EBsg==",
"license": "MIT",
"dependencies": {
"json-schema-to-ts": "^3.1.1",
"standardwebhooks": "^1.0.0"
},
"bin": {
"anthropic-ai-sdk": "bin/cli"
},
"peerDependencies": {
"zod": "^3.25.0 || ^4.0.0"
},
"peerDependenciesMeta": {
"zod": {
"optional": true
}
}
},
"node_modules/@babel/runtime": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.7.tgz",
"integrity": "sha512-Nq8OhGWiZIZGV6hLHoyAKLLcJihP/xFeBMGJoUrxTX2psI8dCifzLhZISFb+VWS3wFMRDmCGw5R+dOySCqPLhw==",
"license": "MIT",
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@google/genai": {
"version": "2.10.0",
"resolved": "https://registry.npmjs.org/@google/genai/-/genai-2.10.0.tgz",
"integrity": "sha512-e4cFxj3tiuMtsgOT4G9c1hXyGJhg7/Buj7VVeBacRY3fRtkRZZ59Q3nuVp2xbq8BGQXLXCDB253qMhklMOeUDg==",
"hasInstallScript": true,
"license": "Apache-2.0",
"dependencies": {
"google-auth-library": "^10.3.0",
"p-retry": "^4.6.2",
"protobufjs": "^7.5.4",
"ws": "^8.18.0"
},
"engines": {
"node": ">=20.0.0"
},
"peerDependencies": {
"@modelcontextprotocol/sdk": "^1.25.2"
},
"peerDependenciesMeta": {
"@modelcontextprotocol/sdk": {
"optional": true
}
}
},
"node_modules/@mistralai/mistralai": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/@mistralai/mistralai/-/mistralai-2.4.0.tgz",
"integrity": "sha512-t6hCx242MTGolB76CI+17jDtPIe/bzLsMdUTMMoMn9Qo1h02N2G5jQYHmKDGU3X//OgR2wvngTD7tO6tPp5poQ==",
"license": "Apache-2.0",
"dependencies": {
"@opentelemetry/semantic-conventions": "^1.40.0",
"ws": "^8.18.0",
"zod": "^3.25.0 || ^4.0.0",
"zod-to-json-schema": "^3.25.0"
},
"peerDependencies": {
"@opentelemetry/api": "^1.9.0"
},
"peerDependenciesMeta": {
"@opentelemetry/api": {
"optional": true
}
}
},
"node_modules/@opentelemetry/semantic-conventions": {
"version": "1.41.1",
"resolved": "https://registry.npmjs.org/@opentelemetry/semantic-conventions/-/semantic-conventions-1.41.1.tgz",
"integrity": "sha512-/UhIkaZgPutTFmQ7RnIJGgDXZmtEJ7Dvi86xNTFWcnRxVRNk/aotsqDJYeEvDP+FSMB2SdW+pQzNMcWP0rwuNA==",
"license": "Apache-2.0",
"engines": {
"node": ">=14"
}
},
"node_modules/@protobufjs/aspromise": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/aspromise/-/aspromise-1.1.2.tgz",
"integrity": "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/base64": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/base64/-/base64-1.1.2.tgz",
"integrity": "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/codegen": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@protobufjs/codegen/-/codegen-2.0.5.tgz",
"integrity": "sha512-zgXFLzW3Ap33e6d0Wlj4MGIm6Ce8O89n/apUaGNB/jx+hw+ruWEp7EwGUshdLKVRCxZW12fp9r40E1mQrf/34g==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/eventemitter": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@protobufjs/eventemitter/-/eventemitter-1.1.1.tgz",
"integrity": "sha512-vW1GmwMZNnL+gMRaovlh9yZX74kc+TTU3FObkkurpMaRtBfLP3ldjS9KQWlwZgraRE0+dheEEoAxdzcJQ8eXZg==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/fetch": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@protobufjs/fetch/-/fetch-1.1.1.tgz",
"integrity": "sha512-GpptLrs57adMSuHi3VNj0mAF8dwh36LMaYF6XyJ6JMWlVsc+t42tm1HSEDmOs3A8fC9yyeisgLhsTVQokOZ0zw==",
"license": "BSD-3-Clause",
"dependencies": {
"@protobufjs/aspromise": "^1.1.1"
}
},
"node_modules/@protobufjs/float": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/@protobufjs/float/-/float-1.0.2.tgz",
"integrity": "sha512-Ddb+kVXlXst9d+R9PfTIxh1EdNkgoRe5tOX6t01f1lYWOvJnSPDBlG241QLzcyPdoNTsblLUdujGSE4RzrTZGQ==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/inquire": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/inquire/-/inquire-1.1.2.tgz",
"integrity": "sha512-pa0vFRuws4wkvaXKK1uXZMAwAX4/t8ANaJo45iw/oQHNQ9q5xUzwgFmVJGXiga2BeN+zpX7Vf9vmsiIa2J+MUw==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/path": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/@protobufjs/path/-/path-1.1.2.tgz",
"integrity": "sha512-6JOcJ5Tm08dOHAbdR3GrvP+yUUfkjG5ePsHYczMFLq3ZmMkAD98cDgcT2iA1lJ9NVwFd4tH/iSSoe44YWkltEA==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/pool": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@protobufjs/pool/-/pool-1.1.0.tgz",
"integrity": "sha512-0kELaGSIDBKvcgS4zkjz1PeddatrjYcmMWOlAuAPwAeccUrPHdUqo/J6LiymHHEiJT5NrF1UVwxY14f+fy4WQw==",
"license": "BSD-3-Clause"
},
"node_modules/@protobufjs/utf8": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/@protobufjs/utf8/-/utf8-1.1.1.tgz",
"integrity": "sha512-oOAWABowe8EAbMyWKM0tYDKi8Yaox52D+HWZhAIJqQXbqe0xI/GV7FhLWqlEKreMkfDjshR5FKgi3mnle0h6Eg==",
"license": "BSD-3-Clause"
},
"node_modules/@stablelib/base64": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/@stablelib/base64/-/base64-1.0.1.tgz",
"integrity": "sha512-1bnPQqSxSuc3Ii6MhBysoWCg58j97aUjuCSZrGSmDxNqtytIi0k8utUenAwTZN4V5mXXYGsVUI9zeBqy+jBOSQ==",
"license": "MIT"
},
"node_modules/@types/node": {
"version": "26.1.0",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.1.0.tgz",
"integrity": "sha512-O0A1G3xPGy4w7AgQdAQYUlQ+BKk2Oovw8eRpofyp5KdBZULnbe+WqaOVNrm705SHphCiG4XHsACrSmPu1f+Kgw==",
"license": "MIT",
"dependencies": {
"undici-types": "~8.3.0"
}
},
"node_modules/@types/retry": {
"version": "0.12.5",
"resolved": "https://registry.npmjs.org/@types/retry/-/retry-0.12.5.tgz",
"integrity": "sha512-3xSjTp3v03X/lSQLkczaN9UIEwJMoMCA1+Nb5HfbJEQWogdeQIyVtTvxPXDQjZ5zws8rFQfVfRdz03ARihPJgw==",
"license": "MIT"
},
"node_modules/agent-base": {
"version": "7.1.4",
"resolved": "https://registry.npmjs.org/agent-base/-/agent-base-7.1.4.tgz",
"integrity": "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==",
"license": "MIT",
"engines": {
"node": ">= 14"
}
},
"node_modules/base64-js": {
"version": "1.5.1",
"resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
"integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/bignumber.js": {
"version": "9.3.1",
"resolved": "https://registry.npmjs.org/bignumber.js/-/bignumber.js-9.3.1.tgz",
"integrity": "sha512-Ko0uX15oIUS7wJ3Rb30Fs6SkVbLmPBAKdlm7q9+ak9bbIeFf0MwuBsQV6z7+X768/cHsfg+WlysDWJcmthjsjQ==",
"license": "MIT",
"engines": {
"node": "*"
}
},
"node_modules/buffer-equal-constant-time": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/buffer-equal-constant-time/-/buffer-equal-constant-time-1.0.1.tgz",
"integrity": "sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA==",
"license": "BSD-3-Clause"
},
"node_modules/data-uri-to-buffer": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.1.tgz",
"integrity": "sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==",
"license": "MIT",
"engines": {
"node": ">= 12"
}
},
"node_modules/debug": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
"integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/ecdsa-sig-formatter": {
"version": "1.0.11",
"resolved": "https://registry.npmjs.org/ecdsa-sig-formatter/-/ecdsa-sig-formatter-1.0.11.tgz",
"integrity": "sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ==",
"license": "Apache-2.0",
"dependencies": {
"safe-buffer": "^5.0.1"
}
},
"node_modules/extend": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/extend/-/extend-3.0.2.tgz",
"integrity": "sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==",
"license": "MIT"
},
"node_modules/fast-sha256": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/fast-sha256/-/fast-sha256-1.3.0.tgz",
"integrity": "sha512-n11RGP/lrWEFI/bWdygLxhI+pVeo1ZYIVwvvPkW7azl/rOy+F3HYRZ2K5zeE9mmkhQppyv9sQFx0JM9UabnpPQ==",
"license": "Unlicense"
},
"node_modules/fetch-blob": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.2.0.tgz",
"integrity": "sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/jimmywarting"
},
{
"type": "paypal",
"url": "https://paypal.me/jimmywarting"
}
],
"license": "MIT",
"dependencies": {
"node-domexception": "^1.0.0",
"web-streams-polyfill": "^3.0.3"
},
"engines": {
"node": "^12.20 || >= 14.13"
}
},
"node_modules/formdata-polyfill": {
"version": "4.0.10",
"resolved": "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz",
"integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==",
"license": "MIT",
"dependencies": {
"fetch-blob": "^3.1.2"
},
"engines": {
"node": ">=12.20.0"
}
},
"node_modules/gaxios": {
"version": "7.1.5",
"resolved": "https://registry.npmjs.org/gaxios/-/gaxios-7.1.5.tgz",
"integrity": "sha512-5FZy72Rh8LhtjmvDrKkI+lVhrsQrVKVsItxMoDm5mNQE+xR0WVIIs+jzPSJgBvKVsLi24fZhXJIsNI0bihDzFg==",
"license": "Apache-2.0",
"dependencies": {
"extend": "^3.0.2",
"https-proxy-agent": "^7.0.1",
"node-fetch": "^3.3.2"
},
"engines": {
"node": ">=18"
}
},
"node_modules/gcp-metadata": {
"version": "8.1.2",
"resolved": "https://registry.npmjs.org/gcp-metadata/-/gcp-metadata-8.1.2.tgz",
"integrity": "sha512-zV/5HKTfCeKWnxG0Dmrw51hEWFGfcF2xiXqcA3+J90WDuP0SvoiSO5ORvcBsifmx/FoIjgQN3oNOGaQ5PhLFkg==",
"license": "Apache-2.0",
"dependencies": {
"gaxios": "^7.0.0",
"google-logging-utils": "^1.0.0",
"json-bigint": "^1.0.0"
},
"engines": {
"node": ">=18"
}
},
"node_modules/google-auth-library": {
"version": "10.9.0",
"resolved": "https://registry.npmjs.org/google-auth-library/-/google-auth-library-10.9.0.tgz",
"integrity": "sha512-xtvUqvINPhTaBm7nXqlYPcrMHJPm1lCNdSovxnKKhTm+4JsvQ+KGVYJViLoH9Yxu8w+T0Qv5HubzYT9BLrppJg==",
"license": "Apache-2.0",
"dependencies": {
"base64-js": "^1.3.0",
"ecdsa-sig-formatter": "^1.0.11",
"gaxios": "^7.1.4",
"gcp-metadata": "8.1.2",
"google-logging-utils": "1.1.3",
"jws": "^4.0.0"
},
"engines": {
"node": ">=18"
}
},
"node_modules/google-logging-utils": {
"version": "1.1.3",
"resolved": "https://registry.npmjs.org/google-logging-utils/-/google-logging-utils-1.1.3.tgz",
"integrity": "sha512-eAmLkjDjAFCVXg7A1unxHsLf961m6y17QFqXqAXGj/gVkKFrEICfStRfwUlGNfeCEjNRa32JEWOUTlYXPyyKvA==",
"license": "Apache-2.0",
"engines": {
"node": ">=14"
}
},
"node_modules/https-proxy-agent": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz",
"integrity": "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==",
"license": "MIT",
"dependencies": {
"agent-base": "^7.1.2",
"debug": "4"
},
"engines": {
"node": ">= 14"
}
},
"node_modules/json-bigint": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/json-bigint/-/json-bigint-1.0.0.tgz",
"integrity": "sha512-SiPv/8VpZuWbvLSMtTDU8hEfrZWg/mH/nV/b4o0CYbSxu1UIQPLdwKOCIyLQX+VIPO5vrLX3i8qtqFyhdPSUSQ==",
"license": "MIT",
"dependencies": {
"bignumber.js": "^9.0.0"
}
},
"node_modules/json-schema-to-ts": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/json-schema-to-ts/-/json-schema-to-ts-3.1.1.tgz",
"integrity": "sha512-+DWg8jCJG2TEnpy7kOm/7/AxaYoaRbjVB4LFZLySZlWn8exGs3A4OLJR966cVvU26N7X9TWxl+Jsw7dzAqKT6g==",
"license": "MIT",
"dependencies": {
"@babel/runtime": "^7.18.3",
"ts-algebra": "^2.0.0"
},
"engines": {
"node": ">=16"
}
},
"node_modules/jwa": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/jwa/-/jwa-2.0.1.tgz",
"integrity": "sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg==",
"license": "MIT",
"dependencies": {
"buffer-equal-constant-time": "^1.0.1",
"ecdsa-sig-formatter": "1.0.11",
"safe-buffer": "^5.0.1"
}
},
"node_modules/jws": {
"version": "4.0.1",
"resolved": "https://registry.npmjs.org/jws/-/jws-4.0.1.tgz",
"integrity": "sha512-EKI/M/yqPncGUUh44xz0PxSidXFr/+r0pA70+gIYhjv+et7yxM+s29Y+VGDkovRofQem0fs7Uvf4+YmAdyRduA==",
"license": "MIT",
"dependencies": {
"jwa": "^2.0.1",
"safe-buffer": "^5.0.1"
}
},
"node_modules/long": {
"version": "5.3.2",
"resolved": "https://registry.npmjs.org/long/-/long-5.3.2.tgz",
"integrity": "sha512-mNAgZ1GmyNhD7AuqnTG3/VQ26o760+ZYBPKjPvugO8+nLbYfX6TVpJPseBvopbdY+qpZ/lKUnmEc1LeZYS3QAA==",
"license": "Apache-2.0"
},
"node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT"
},
"node_modules/node-domexception": {
"name": "@nolyfill/domexception",
"version": "1.0.28",
"resolved": "https://registry.npmjs.org/@nolyfill/domexception/-/domexception-1.0.28.tgz",
"integrity": "sha512-tlc/FcYIv5i8RYsl2iDil4A0gOihaas1R5jPcIC4Zw3GhjKsVilw90aHcVlhZPTBLGBzd379S+VcnsDjd9ChiA==",
"license": "MIT",
"engines": {
"node": ">=12.4.0"
}
},
"node_modules/node-fetch": {
"version": "3.3.2",
"resolved": "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.2.tgz",
"integrity": "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==",
"license": "MIT",
"dependencies": {
"data-uri-to-buffer": "^4.0.0",
"fetch-blob": "^3.1.4",
"formdata-polyfill": "^4.0.10"
},
"engines": {
"node": "^12.20.0 || ^14.13.1 || >=16.0.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/node-fetch"
}
},
"node_modules/openai": {
"version": "6.45.0",
"resolved": "https://registry.npmjs.org/openai/-/openai-6.45.0.tgz",
"integrity": "sha512-5DQVNErssk0afNpTTHUm/qZPU4iKR9OYdNid8Ib4puq4gHNNvGWZht2zY4h9a8JMF949Ik6m8gQutllVPbjdnw==",
"license": "Apache-2.0",
"peerDependencies": {
"@aws-sdk/credential-provider-node": ">=3.972.0 <4",
"@smithy/hash-node": ">=4.3.0 <5",
"@smithy/signature-v4": ">=5.4.0 <6",
"ws": "^8.18.0",
"zod": "^3.25 || ^4.0"
},
"peerDependenciesMeta": {
"@aws-sdk/credential-provider-node": {
"optional": true
},
"@smithy/hash-node": {
"optional": true
},
"@smithy/signature-v4": {
"optional": true
},
"ws": {
"optional": true
},
"zod": {
"optional": true
}
}
},
"node_modules/p-retry": {
"version": "4.6.2",
"resolved": "https://registry.npmjs.org/p-retry/-/p-retry-4.6.2.tgz",
"integrity": "sha512-312Id396EbJdvRONlngUx0NydfrIQ5lsYu0znKVUzVvArzEIt08V1qhtyESbGVd1FGX7UKtiFp5uwKZdM8wIuQ==",
"license": "MIT",
"dependencies": {
"@types/retry": "0.12.0",
"retry": "^0.13.1"
},
"engines": {
"node": ">=8"
}
},
"node_modules/partial-json": {
"version": "0.1.7",
"resolved": "https://registry.npmjs.org/partial-json/-/partial-json-0.1.7.tgz",
"integrity": "sha512-Njv/59hHaokb/hRUjce3Hdv12wd60MtM9Z5Olmn+nehe0QDAsRtRbJPvJ0Z91TusF0SuZRIvnM+S4l6EIP8leA==",
"license": "MIT"
},
"node_modules/protobufjs": {
"version": "7.6.3",
"resolved": "https://registry.npmjs.org/protobufjs/-/protobufjs-7.6.3.tgz",
"integrity": "sha512-+k0vdJKNdW+Vu+dYe8tZA/VvQb6XKNWexC6URwBFXxNnjLJz9nQJCemGyNgRAWD+B7+nGNc9qMPGwcD7s4nzUw==",
"hasInstallScript": true,
"license": "BSD-3-Clause",
"dependencies": {
"@protobufjs/aspromise": "^1.1.2",
"@protobufjs/base64": "^1.1.2",
"@protobufjs/codegen": "^2.0.5",
"@protobufjs/eventemitter": "^1.1.1",
"@protobufjs/fetch": "^1.1.1",
"@protobufjs/float": "^1.0.2",
"@protobufjs/inquire": "^1.1.2",
"@protobufjs/path": "^1.1.2",
"@protobufjs/pool": "^1.1.0",
"@protobufjs/utf8": "^1.1.1",
"@types/node": ">=13.7.0",
"long": "^5.3.2"
},
"engines": {
"node": ">=12.0.0"
}
},
"node_modules/retry": {
"version": "0.13.1",
"resolved": "https://registry.npmjs.org/retry/-/retry-0.13.1.tgz",
"integrity": "sha512-XQBQ3I8W1Cge0Seh+6gjj03LbmRFWuoszgK9ooCpwYIrhhoO80pfq4cUkU5DkknwfOfFteRwlZ56PYOGYyFWdg==",
"license": "MIT",
"engines": {
"node": ">= 4"
}
},
"node_modules/safe-buffer": {
"version": "5.2.1",
"resolved": "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz",
"integrity": "sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/standardwebhooks": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/standardwebhooks/-/standardwebhooks-1.0.0.tgz",
"integrity": "sha512-BbHGOQK9olHPMvQNHWul6MYlrRTAOKn03rOe4A8O3CLWhNf4YHBqq2HJKKC+sfqpxiBY52pNeesD6jIiLDz8jg==",
"license": "MIT",
"dependencies": {
"@stablelib/base64": "^1.0.0",
"fast-sha256": "^1.3.0"
}
},
"node_modules/ts-algebra": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/ts-algebra/-/ts-algebra-2.0.0.tgz",
"integrity": "sha512-FPAhNPFMrkwz76P7cdjdmiShwMynZYN6SgOujD1urY4oNm80Ou9oMdmbR45LotcKOXoy7wSmHkRFE6Mxbrhefw==",
"license": "MIT"
},
"node_modules/typebox": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/typebox/-/typebox-1.3.3.tgz",
"integrity": "sha512-URXGUE31PJDQC+PtRMJeLdF4kmmOdFoVPikPCtV2oOIhUpNpppEdIz7W8bH8cFYPYHdDpaRvqwdegMTmHliudg==",
"license": "MIT"
},
"node_modules/undici-types": {
"version": "8.3.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-8.3.0.tgz",
"integrity": "sha512-j375ScV60dom+YkPFIfTLcOiPxkN/buHz5GobjLhixFuANaNs3C9l4GmrWqejgXWJ7BbJcFYpTEUkS1Ge8bpZQ==",
"license": "MIT"
},
"node_modules/web-streams-polyfill": {
"version": "3.3.3",
"resolved": "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz",
"integrity": "sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==",
"license": "MIT",
"engines": {
"node": ">= 8"
}
},
"node_modules/ws": {
"version": "8.21.0",
"resolved": "https://registry.npmjs.org/ws/-/ws-8.21.0.tgz",
"integrity": "sha512-Vsp28b7DRcimFQvrqu2Wek3z1iYxDCWqHYB8Qsnk/S4RfaCQzPGPyBNuVjJV3cd6UiKtUtp6sNM77gWvzcCH+g==",
"license": "MIT",
"engines": {
"node": ">=10.0.0"
},
"peerDependencies": {
"bufferutil": "^4.0.1",
"utf-8-validate": ">=5.0.2"
},
"peerDependenciesMeta": {
"bufferutil": {
"optional": true
},
"utf-8-validate": {
"optional": true
}
}
},
"node_modules/zod": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/zod/-/zod-4.4.3.tgz",
"integrity": "sha512-ytENFjIJFl2UwYglde2jchW2Hwm4GJFLDiSXWdTrJQBIN9Fcyp7n4DhxJEiWNAJMV1/BqWfW/kkg71UDcHJyTQ==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/colinhacks"
}
},
"node_modules/zod-to-json-schema": {
"version": "3.25.2",
"resolved": "https://registry.npmjs.org/zod-to-json-schema/-/zod-to-json-schema-3.25.2.tgz",
"integrity": "sha512-O/PgfnpT1xKSDeQYSCfRI5Gy3hPf91mKVDuYLUHZJMiDFptvP41MSnWofm8dnCm0256ZNfZIM7DSzuSMAFnjHA==",
"license": "ISC",
"peerDependencies": {
"zod": "^3.25.28 || ^4"
}
}
}
}

89
packages/ai/package.json Normal file
View File

@@ -0,0 +1,89 @@
{
"name": "@openclaw/ai",
"version": "2026.6.11",
"description": "Reusable model provider adapters and streaming runtime from OpenClaw",
"keywords": [
"ai",
"anthropic",
"google",
"llm",
"mistral",
"openai",
"streaming"
],
"homepage": "https://github.com/openclaw/openclaw#readme",
"bugs": {
"url": "https://github.com/openclaw/openclaw/issues"
},
"license": "MIT",
"repository": {
"type": "git",
"url": "git+https://github.com/openclaw/openclaw.git",
"directory": "packages/ai"
},
"files": [
"dist",
"npm-shrinkwrap.json",
"LICENSE",
"README.md"
],
"type": "module",
"main": "./dist/index.mjs",
"types": "./dist/index.d.mts",
"exports": {
".": {
"types": "./dist/index.d.mts",
"import": "./dist/index.mjs",
"default": "./dist/index.mjs"
},
"./providers": {
"types": "./dist/providers.d.mts",
"import": "./dist/providers.mjs",
"default": "./dist/providers.mjs"
},
"./diagnostics": {
"types": "./dist/diagnostics.d.mts",
"import": "./dist/diagnostics.mjs",
"default": "./dist/diagnostics.mjs"
},
"./event-stream": {
"types": "./dist/event-stream.d.mts",
"import": "./dist/event-stream.mjs",
"default": "./dist/event-stream.mjs"
},
"./types": {
"types": "./dist/types.d.mts",
"import": "./dist/types.mjs",
"default": "./dist/types.mjs"
},
"./validation": {
"types": "./dist/validation.d.mts",
"import": "./dist/validation.mjs",
"default": "./dist/validation.mjs"
},
"./internal/*": {
"types": "./dist/internal/*.d.mts",
"import": "./dist/internal/*.mjs",
"default": "./dist/internal/*.mjs"
}
},
"dependencies": {
"@anthropic-ai/sdk": "0.109.1",
"@google/genai": "2.10.0",
"@mistralai/mistralai": "2.4.0",
"openai": "6.45.0",
"partial-json": "0.1.7",
"typebox": "1.3.3"
},
"engines": {
"node": ">=22.19.0"
},
"publishConfig": {
"access": "public"
},
"openclaw": {
"release": {
"publishToNpm": true
}
}
}

View File

@@ -0,0 +1,76 @@
// LLM Runtime tests cover api registry behavior.
import { describe, expect, it, vi } from "vitest";
import {
createApiRegistry,
createAssistantMessageEventStream,
createLlmRuntime,
type Model,
} from "./index.js";
const TEST_SOURCE_ID = "test:llm-runtime-api-registry";
const emptyStream = () => createAssistantMessageEventStream();
const model = {
id: "test-model",
name: "Test Model",
api: "test-api",
provider: "test-provider",
baseUrl: "https://example.invalid",
input: ["text"],
reasoning: false,
contextWindow: 1000,
maxTokens: 100,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
} satisfies Model;
describe("LLM API registry", () => {
it("rejects mismatched model API calls", () => {
const registry = createApiRegistry();
registry.registerApiProvider(
{
api: "test-api",
stream: emptyStream,
streamSimple: emptyStream,
},
TEST_SOURCE_ID,
);
const provider = registry.getApiProvider("test-api");
expect(provider).toBeDefined();
expect(() => provider?.streamSimple({ ...model, api: "other-api" }, { messages: [] })).toThrow(
"Mismatched api: other-api expected test-api",
);
});
it("isolates providers between runtime instances", () => {
const first = createLlmRuntime();
const second = createLlmRuntime();
const streamSimple = vi.fn(() => createAssistantMessageEventStream());
first.registry.registerApiProvider({ api: "test-api", stream: streamSimple, streamSimple });
first.streamSimple(model, { messages: [] });
expect(streamSimple).toHaveBeenCalledOnce();
expect(() => second.streamSimple(model, { messages: [] })).toThrow(
"No API provider registered for api: test-api",
);
});
it("unregisters every provider owned by one source", () => {
const registry = createApiRegistry();
for (const api of ["test-api", "test-api-2"] as const) {
registry.registerApiProvider(
{
api,
stream: emptyStream,
streamSimple: emptyStream,
},
TEST_SOURCE_ID,
);
}
registry.unregisterApiProviders(TEST_SOURCE_ID);
expect(registry.getApiProviders()).toEqual([]);
});
});

View File

@@ -0,0 +1,119 @@
// LLM Runtime module implements api registry behavior.
import type {
Api,
AssistantMessageEventStreamContract,
Context,
Model,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "@openclaw/llm-core";
/** Runtime stream adapter signature stored in the API provider registry. */
export type ApiStreamFunction = (
model: Model,
context: Context,
options?: StreamOptions,
) => AssistantMessageEventStreamContract;
/** Runtime simple-stream adapter signature stored in the API provider registry. */
export type ApiStreamSimpleFunction = (
model: Model,
context: Context,
options?: SimpleStreamOptions,
) => AssistantMessageEventStreamContract;
/** Provider implementation registered by core or plugins for a specific model API. */
export interface ApiProvider<
TApi extends Api = Api,
TOptions extends StreamOptions = StreamOptions,
> {
/** Model API id this provider handles. */
api: TApi;
/** Full streaming adapter for callers that already own structured options. */
stream: StreamFunction<TApi, TOptions>;
/** Simple streaming adapter used by agent and plugin runtime defaults. */
streamSimple: StreamFunction<TApi, SimpleStreamOptions>;
}
/** Type-erased provider returned by a registry after API guards are installed. */
export interface RegisteredApiProvider {
api: Api;
stream: ApiStreamFunction;
streamSimple: ApiStreamSimpleFunction;
}
type RegisteredApiProviderEntry = {
provider: RegisteredApiProvider;
sourceId?: string;
};
function wrapStream<TApi extends Api, TOptions extends StreamOptions>(
api: TApi,
stream: StreamFunction<TApi, TOptions>,
): ApiStreamFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return stream(model as Model<TApi>, context, options as TOptions);
};
}
function wrapStreamSimple<TApi extends Api>(
api: TApi,
streamSimple: StreamFunction<TApi, SimpleStreamOptions>,
): ApiStreamSimpleFunction {
return (model, context, options) => {
if (model.api !== api) {
throw new Error(`Mismatched api: ${model.api} expected ${api}`);
}
return streamSimple(model as Model<TApi>, context, options);
};
}
/** Creates an isolated provider registry for one runtime or tenant. */
export function createApiRegistry() {
const providers = new Map<string, RegisteredApiProviderEntry>();
function registerApiProvider<TApi extends Api, TOptions extends StreamOptions>(
provider: ApiProvider<TApi, TOptions>,
/** Optional source id used to unregister all providers owned by one plugin/runtime. */
sourceId?: string,
): void {
providers.set(provider.api, {
provider: {
api: provider.api,
stream: wrapStream(provider.api, provider.stream),
streamSimple: wrapStreamSimple(provider.api, provider.streamSimple),
},
sourceId,
});
}
function getApiProvider(api: Api): RegisteredApiProvider | undefined {
return providers.get(api)?.provider;
}
function getApiProviders(): RegisteredApiProvider[] {
return Array.from(providers.values(), (entry) => entry.provider);
}
function unregisterApiProviders(sourceId: string): void {
for (const [api, entry] of providers.entries()) {
if (entry.sourceId === sourceId) {
providers.delete(api);
}
}
}
return {
registerApiProvider,
getApiProvider,
getApiProviders,
unregisterApiProviders,
clearApiProviders: () => providers.clear(),
};
}
export type ApiRegistry = ReturnType<typeof createApiRegistry>;

View File

@@ -0,0 +1,261 @@
// NEVER convert to top-level imports - breaks browser/Vite builds
let existsSync: typeof import("node:fs").existsSync | null = null;
let homedir: typeof import("node:os").homedir | null = null;
let join: typeof import("node:path").join | null = null;
type DynamicImport = (specifier: string) => Promise<unknown>;
type NodeBuiltinModule =
| typeof import("node:fs")
| typeof import("node:os")
| typeof import("node:path");
const dynamicImport: DynamicImport = (specifier) => import(specifier);
const NODE_FS_SPECIFIER = "node:fs";
const NODE_OS_SPECIFIER = "node:os";
const NODE_PATH_SPECIFIER = "node:path";
function loadNodeBuiltinModule(specifier: string): NodeBuiltinModule | null {
const getBuiltinModule = (typeof process !== "undefined" ? process : undefined) as
| (NodeJS.Process & { getBuiltinModule?: (id: string) => unknown })
| undefined;
if (typeof getBuiltinModule?.getBuiltinModule === "function") {
return getBuiltinModule.getBuiltinModule(specifier) as NodeBuiltinModule;
}
if (typeof require === "function") {
return require(specifier) as NodeBuiltinModule;
}
return null;
}
function loadNodeHelpersSync(): boolean {
try {
const fsModule = loadNodeBuiltinModule(NODE_FS_SPECIFIER) as typeof import("node:fs") | null;
const osModule = loadNodeBuiltinModule(NODE_OS_SPECIFIER) as typeof import("node:os") | null;
const pathModule = loadNodeBuiltinModule(NODE_PATH_SPECIFIER) as
| typeof import("node:path")
| null;
existsSync ??= fsModule?.existsSync ?? null;
homedir ??= osModule?.homedir ?? null;
join ??= pathModule?.join ?? null;
if (!existsSync || !homedir || !join) {
return false;
}
return true;
} catch {
return false;
}
}
// Eagerly load in Node.js/Bun environment only
if (typeof process !== "undefined" && (process.versions?.node || process.versions?.bun)) {
if (!loadNodeHelpersSync()) {
void dynamicImport(NODE_FS_SPECIFIER).then((m) => {
existsSync = (m as typeof import("node:fs")).existsSync;
});
void dynamicImport(NODE_OS_SPECIFIER).then((m) => {
homedir = (m as typeof import("node:os")).homedir;
});
void dynamicImport(NODE_PATH_SPECIFIER).then((m) => {
join = (m as typeof import("node:path")).join;
});
}
}
let procEnvCache: Map<string, string> | null = null;
function getProcessEnv(): NodeJS.ProcessEnv | undefined {
return typeof process === "undefined" ? undefined : process.env;
}
/**
* Fallback for https://github.com/oven-sh/bun/issues/27802
* Bun compiled binaries have an empty `process.env` inside sandbox
* environments on Linux. We can recover the env from `/proc/self/environ`.
*/
function getProcEnv(key: string): string | undefined {
if (typeof process === "undefined" || !process.versions?.bun) {
return undefined;
}
const env = getProcessEnv();
if (!env) {
return undefined;
}
// If process.env already has entries, the bug is not triggered.
if (Object.keys(env).length > 0) {
return undefined;
}
if (procEnvCache === null) {
procEnvCache = new Map();
try {
const { readFileSync } = require("node:fs") as typeof import("node:fs");
const data = readFileSync("/proc/self/environ", "utf-8");
for (const entry of data.split("\0")) {
const idx = entry.indexOf("=");
if (idx > 0) {
procEnvCache.set(entry.slice(0, idx), entry.slice(idx + 1));
}
}
} catch {
// /proc/self/environ may not be readable.
}
}
return procEnvCache.get(key);
}
function getEnvValue(key: string): string | undefined {
return getProcessEnv()?.[key] || getProcEnv(key);
}
let cachedVertexAdcCredentialsExists: true | null = null;
function hasVertexAdcCredentials(): boolean {
if (cachedVertexAdcCredentialsExists === null) {
if (!existsSync || !homedir || !join) {
const isNode =
typeof process !== "undefined" && (process.versions?.node || process.versions?.bun);
if (!isNode || !loadNodeHelpersSync()) {
return false;
}
}
const nodeExistsSync = existsSync;
const nodeHomedir = homedir;
const nodeJoin = join;
if (!nodeExistsSync || !nodeHomedir || !nodeJoin) {
return false;
}
// Check GOOGLE_APPLICATION_CREDENTIALS env var first (standard way)
const gacPath = getEnvValue("GOOGLE_APPLICATION_CREDENTIALS");
if (gacPath) {
cachedVertexAdcCredentialsExists = nodeExistsSync(gacPath) ? true : null;
} else {
// Fall back to default ADC path (lazy evaluation)
cachedVertexAdcCredentialsExists = nodeExistsSync(
nodeJoin(nodeHomedir(), ".config", "gcloud", "application_default_credentials.json"),
)
? true
: null;
}
}
return cachedVertexAdcCredentialsExists === true;
}
function getApiKeyEnvVars(provider: string): readonly string[] | undefined {
if (provider === "github-copilot") {
return ["COPILOT_GITHUB_TOKEN"];
}
// ANTHROPIC_OAUTH_TOKEN takes precedence over ANTHROPIC_API_KEY
if (provider === "anthropic") {
return ["ANTHROPIC_OAUTH_TOKEN", "ANTHROPIC_API_KEY"];
}
if (provider === "moonshot") {
return ["MOONSHOT_API_KEY", "KIMI_API_KEY"];
}
if (provider === "kimi" || provider === "kimi-coding") {
return ["KIMI_API_KEY", "KIMICODE_API_KEY"];
}
const envMap: Record<string, string> = {
openai: "OPENAI_API_KEY",
"azure-openai-responses": "AZURE_OPENAI_API_KEY",
deepseek: "DEEPSEEK_API_KEY",
google: "GEMINI_API_KEY",
"google-vertex": "GOOGLE_CLOUD_API_KEY",
groq: "GROQ_API_KEY",
cerebras: "CEREBRAS_API_KEY",
xai: "XAI_API_KEY",
openrouter: "OPENROUTER_API_KEY",
"vercel-ai-gateway": "AI_GATEWAY_API_KEY",
zai: "ZAI_API_KEY",
mistral: "MISTRAL_API_KEY",
minimax: "MINIMAX_API_KEY",
"minimax-cn": "MINIMAX_CN_API_KEY",
moonshotai: "MOONSHOT_API_KEY",
"moonshotai-cn": "MOONSHOT_API_KEY",
huggingface: "HF_TOKEN",
fireworks: "FIREWORKS_API_KEY",
together: "TOGETHER_API_KEY",
opencode: "OPENCODE_API_KEY",
"opencode-go": "OPENCODE_API_KEY",
"cloudflare-workers-ai": "CLOUDFLARE_API_KEY",
"cloudflare-ai-gateway": "CLOUDFLARE_API_KEY",
xiaomi: "XIAOMI_API_KEY",
"xiaomi-token-plan-cn": "XIAOMI_TOKEN_PLAN_CN_API_KEY",
"xiaomi-token-plan-ams": "XIAOMI_TOKEN_PLAN_AMS_API_KEY",
"xiaomi-token-plan-sgp": "XIAOMI_TOKEN_PLAN_SGP_API_KEY",
};
const envVar = envMap[provider];
return envVar ? [envVar] : undefined;
}
/**
* Find configured environment variables that can provide an API key for a provider.
*
* This only reports actual API key variables. It intentionally excludes ambient
* credential sources such as AWS profiles, AWS IAM credentials, and Google
* Application Default Credentials.
*/
export function findEnvKeys(provider: string): string[] | undefined {
const envVars = getApiKeyEnvVars(provider);
if (!envVars) {
return undefined;
}
const found = envVars.filter((envVar) => Boolean(getEnvValue(envVar)));
return found.length > 0 ? found : undefined;
}
/**
* Get API key for provider from known environment variables, e.g. OPENAI_API_KEY.
*
* Will not return API keys for providers that require OAuth tokens.
*/
export function getEnvApiKey(provider: string): string | undefined {
const envKeys = findEnvKeys(provider);
if (envKeys?.[0]) {
return getEnvValue(envKeys[0]);
}
// Vertex AI supports either an explicit API key or Application Default Credentials.
// Auth is configured via `gcloud auth application-default login`.
if (provider === "google-vertex") {
const hasCredentials = hasVertexAdcCredentials();
const hasProject = Boolean(
getEnvValue("GOOGLE_CLOUD_PROJECT") || getEnvValue("GCLOUD_PROJECT"),
);
const hasLocation = Boolean(getEnvValue("GOOGLE_CLOUD_LOCATION"));
if (hasCredentials && hasProject && hasLocation) {
return "<authenticated>";
}
}
if (provider === "amazon-bedrock") {
// Amazon Bedrock supports multiple credential sources:
// 1. AWS_PROFILE - named profile from ~/.aws/credentials
// 2. AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY - standard IAM keys
// 3. AWS_BEARER_TOKEN_BEDROCK - Bedrock bearer token
// 4. AWS_CONTAINER_CREDENTIALS_RELATIVE_URI - ECS task roles
// 5. AWS_CONTAINER_CREDENTIALS_FULL_URI - ECS task roles (full URI)
// 6. AWS_WEB_IDENTITY_TOKEN_FILE - IRSA (IAM Roles for Service Accounts)
if (
getEnvValue("AWS_PROFILE") ||
(getEnvValue("AWS_ACCESS_KEY_ID") && getEnvValue("AWS_SECRET_ACCESS_KEY")) ||
getEnvValue("AWS_BEARER_TOKEN_BEDROCK") ||
getEnvValue("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI") ||
getEnvValue("AWS_CONTAINER_CREDENTIALS_FULL_URI") ||
getEnvValue("AWS_WEB_IDENTITY_TOKEN_FILE")
) {
return "<authenticated>";
}
}
return undefined;
}

66
packages/ai/src/host.ts Normal file
View File

@@ -0,0 +1,66 @@
// Host policy ports for the reusable transport package. Fetch guarding,
// secret redaction, strict-tool policy, and diagnostics logging are owned by
// the embedding application (OpenClaw core installs its implementations via
// configureAiTransportHost); the library defaults below are inert so external
// consumers get safe, dependency-free behavior without wiring anything.
import type { Model } from "@openclaw/llm-core";
/** Strict-tool policy inputs for OpenAI-compatible routes. */
export interface OpenAIStrictToolSettingOptions {
transport?: "stream" | "websocket";
supportsStrictMode?: boolean;
}
/** Narrow host ports consumed by the built-in provider adapters. */
export interface AiTransportHost {
/**
* Builds a policy-guarded fetch for one model request.
* Returning undefined keeps the provider SDK's default fetch.
*/
buildModelFetch(
model: Model,
timeoutMs?: number,
options?: { sanitizeSse?: boolean },
): typeof fetch | undefined;
/** Redacts secrets inside structured tool-result payloads. */
redactSecrets<T>(value: T): T;
/** Redacts secret-bearing text in tool payload strings. */
redactToolPayloadText(text: string): string;
/**
* Resolves the host strict-tool default for OpenAI-compatible routes.
* undefined lets the request omit the strict flag entirely.
*/
resolveOpenAIStrictToolSetting(
model: Pick<Model, "provider" | "api" | "baseUrl" | "id"> & { compat?: unknown },
options?: OpenAIStrictToolSettingOptions,
): boolean | undefined;
/**
* Emits one transport diagnostic; build runs only when the host logs it and
* may return null to suppress the entry (e.g. de-duplication).
*/
logDebug(
subsystem: string,
build: () => { message: string; data?: Record<string, unknown> } | null,
): void;
}
const inertAiTransportHost: AiTransportHost = {
buildModelFetch: () => undefined,
redactSecrets: (value) => value,
redactToolPayloadText: (text) => text,
resolveOpenAIStrictToolSetting: (_model, options) =>
options?.supportsStrictMode ? false : undefined,
logDebug: () => {},
};
let activeAiTransportHost = inertAiTransportHost;
/** Installs host implementations for the transport policy ports. */
export function configureAiTransportHost(host: Partial<AiTransportHost>): void {
activeAiTransportHost = { ...inertAiTransportHost, ...host };
}
/** Returns the active transport host (inert defaults unless configured). */
export function getAiTransportHost(): AiTransportHost {
return activeAiTransportHost;
}

5
packages/ai/src/index.ts Normal file
View File

@@ -0,0 +1,5 @@
/** Reusable model API contracts, provider adapters, and streaming runtime. */
export * from "@openclaw/llm-core";
export * from "./api-registry.js";
export * from "./host.js";
export * from "./stream.js";

View File

@@ -0,0 +1,8 @@
export * from "../providers/anthropic.js";
export * from "../providers/anthropic-auth-headers.js";
export * from "../providers/anthropic-model-contract.js";
export * from "../providers/anthropic-refusal.js";
export * from "../providers/anthropic-server-fallback.js";
export * from "../providers/anthropic-thinking-replay.js";
export * from "../providers/anthropic-tool-projection.js";
export * from "../providers/anthropic-usage.js";

View File

@@ -0,0 +1,19 @@
// Process-default registry/runtime retained for the OpenClaw compatibility
// facade (src/llm). Deliberately not part of the public package API: external
// consumers create isolated runtimes via createLlmRuntime(); exporting these
// from the root barrel would reintroduce the mutable process-global registry.
import { createApiRegistry } from "../api-registry.js";
import { createLlmRuntime } from "../stream.js";
export const defaultApiRegistry = createApiRegistry();
export const defaultLlmRuntime = createLlmRuntime(defaultApiRegistry);
export const {
registerApiProvider,
getApiProvider,
getApiProviders,
unregisterApiProviders,
clearApiProviders,
} = defaultApiRegistry;
export const { stream, complete, streamSimple, completeSimple } = defaultLlmRuntime;

View File

@@ -0,0 +1,14 @@
export * from "../providers/agent-tools-parameter-schema.js";
export * from "../providers/azure-deployment-map.js";
export * from "../providers/azure-openai-responses-client-compat.js";
export * from "../providers/clean-for-gemini.js";
export * from "../providers/openai-completions.js";
export * from "../providers/openai-prompt-cache.js";
export * from "../providers/openai-reasoning-effort.js";
export * from "../providers/openai-responses.js";
export * from "../providers/openai-responses-stream-compat.js";
export * from "../providers/openai-stop-reason.js";
export * from "../providers/openai-tool-projection.js";
export * from "../providers/openai-tool-schema.js";
export * from "../providers/schema-keyword-strip.js";
export * from "../providers/tool-schema-json-projection.js";

View File

@@ -0,0 +1,15 @@
export * from "./default-runtime.js";
export * from "../env-api-keys.js";
export * from "../model-utils.js";
export * from "../session-resources.js";
export * from "../utils/deferred-event-buffer.js";
export * from "../utils/hash.js";
export * from "../utils/headers.js";
export * from "../utils/json-parse.js";
export * from "../utils/llm-request-activity.js";
export * from "../utils/oauth/openai-chatgpt-jwt.js";
export * from "../utils/overflow.js";
export * from "../utils/reasoning-tag-text-partitioner.js";
export * from "../utils/sanitize-unicode.js";
export * from "../utils/stream-first-event-timeout.js";
export * from "../utils/streaming-byte-guard.js";

View File

@@ -0,0 +1,5 @@
export * from "../providers/simple-options.js";
export * from "../providers/tool-result-text.js";
export * from "../providers/transform-messages.js";
export * from "../utils/prompt-cache-stability.js";
export * from "../utils/system-prompt-cache-boundary.js";

View File

@@ -0,0 +1,110 @@
// Provides model selection, usage, and thinking-level utility helpers.
import {
resolveClaudeFable5ModelIdentity,
resolveClaudeNativeThinkingLevelMap,
} from "@openclaw/llm-core";
import type { Api, Model, ModelThinkingLevel, Usage } from "./types.js";
/** Calculates and stores model cost fields from token usage and per-million pricing. */
export function calculateCost<TApi extends Api>(model: Model<TApi>, usage: Usage): Usage["cost"] {
usage.cost.input = (model.cost.input / 1000000) * usage.input;
usage.cost.output = (model.cost.output / 1000000) * usage.output;
usage.cost.cacheRead = (model.cost.cacheRead / 1000000) * usage.cacheRead;
usage.cost.cacheWrite = (model.cost.cacheWrite / 1000000) * usage.cacheWrite;
usage.cost.total =
usage.cost.input + usage.cost.output + usage.cost.cacheRead + usage.cost.cacheWrite;
return usage.cost;
}
const EXTENDED_THINKING_LEVELS: ModelThinkingLevel[] = [
"off",
"minimal",
"low",
"medium",
"high",
"xhigh",
"max",
];
function resolveThinkingLevelMap<TApi extends Api>(model: Model<TApi>) {
return model.api === "anthropic-messages"
? (resolveClaudeNativeThinkingLevelMap(model) ?? model.thinkingLevelMap)
: model.thinkingLevelMap;
}
/** Returns thinking levels exposed by a reasoning-capable model. */
export function getSupportedThinkingLevels<TApi extends Api>(
model: Model<TApi>,
): ModelThinkingLevel[] {
const fableContract =
model.api === "anthropic-messages" && resolveClaudeFable5ModelIdentity(model) !== undefined;
if (!model.reasoning && !fableContract) {
return ["off"];
}
const thinkingLevelMap = resolveThinkingLevelMap(model);
return EXTENDED_THINKING_LEVELS.filter((level) => {
const mapped = thinkingLevelMap?.[level];
if (mapped === null) {
return false;
}
if (level === "xhigh" || level === "max") {
return mapped !== undefined;
}
return true;
});
}
/** Clamps a requested thinking level to the closest supported level for a model. */
export function clampThinkingLevel<TApi extends Api>(
model: Model<TApi>,
level: ModelThinkingLevel,
): ModelThinkingLevel {
const availableLevels = getSupportedThinkingLevels(model);
if (availableLevels.includes(level)) {
return level;
}
const requestedIndex = EXTENDED_THINKING_LEVELS.indexOf(level);
if (requestedIndex === -1) {
return availableLevels[0] ?? "off";
}
// Explicit provider opt-outs are hard caps. Downgrade them before considering
// stronger levels so unsupported xhigh/max requests cannot increase cost.
const thinkingLevelMap = resolveThinkingLevelMap(model);
if ((level === "xhigh" || level === "max") && thinkingLevelMap?.[level] === null) {
for (let i = requestedIndex - 1; i >= 0; i--) {
const candidate = EXTENDED_THINKING_LEVELS[i];
if (availableLevels.includes(candidate)) {
return candidate;
}
}
}
// Prefer the next stronger available level, then walk down if the request was above the model cap.
for (let i = requestedIndex; i < EXTENDED_THINKING_LEVELS.length; i++) {
const candidate = EXTENDED_THINKING_LEVELS[i];
if (availableLevels.includes(candidate)) {
return candidate;
}
}
for (let i = requestedIndex - 1; i >= 0; i--) {
const candidate = EXTENDED_THINKING_LEVELS[i];
if (availableLevels.includes(candidate)) {
return candidate;
}
}
return availableLevels[0] ?? "off";
}
/** Compares model identity by provider and id. */
export function modelsAreEqual<TApi extends Api>(
a: Model<TApi> | null | undefined,
b: Model<TApi> | null | undefined,
): boolean {
if (!a || !b) {
return false;
}
return a.id === b.id && a.provider === b.provider;
}

View File

@@ -0,0 +1,6 @@
/** Lazy built-in protocol adapter registration. */
export {
BUILT_IN_API_PROVIDER_SOURCE_ID,
registerBuiltInApiProviders,
resetApiProviders,
} from "./providers/register-builtins.js";

View File

@@ -0,0 +1,971 @@
/**
* Normalizes model-facing tool parameter schemas across provider quirks.
* Handles local JSON Schema refs, OpenAPI nullable syntax, top-level unions,
* and provider-specific unsupported keyword stripping.
*/
import { isRecord as isSchemaRecord } from "@openclaw/normalization-core/record-coerce";
import { normalizeLowercaseStringOrEmpty } from "@openclaw/normalization-core/string-coerce";
import {
normalizeStringEntries,
uniqueValues,
} from "@openclaw/normalization-core/string-normalization";
import type { TSchema } from "typebox";
import { cleanSchemaForGemini } from "./clean-for-gemini.js";
import { stripUnsupportedSchemaKeywords } from "./schema-keyword-strip.js";
/**
* Narrow structural view of the host's model compat config. packages/ai must stay
* config-agnostic, so only tool-schema-relevant fields are modeled here; the host's
* ModelCompatConfig remains structurally assignable.
*/
export type ToolSchemaModelCompat = {
toolSchemaProfile?: string;
unsupportedToolSchemaKeywords?: string[];
omitEmptyArrayItems?: boolean;
};
/** Extracts the compat record whether callers pass a model (`{ compat }`) or the compat itself. */
export function extractToolSchemaModelCompat(
modelOrCompat: { compat?: unknown } | ToolSchemaModelCompat | undefined,
): ToolSchemaModelCompat | undefined {
if (!modelOrCompat || typeof modelOrCompat !== "object") {
return undefined;
}
if ("compat" in modelOrCompat) {
const compat = (modelOrCompat as { compat?: unknown }).compat;
return compat && typeof compat === "object" ? (compat as ToolSchemaModelCompat) : undefined;
}
return modelOrCompat as ToolSchemaModelCompat;
}
/** JSON Schema keywords this model/provider rejects in tool schemas. */
export function resolveUnsupportedToolSchemaKeywords(
modelOrCompat: { compat?: unknown } | ToolSchemaModelCompat | undefined,
): ReadonlySet<string> {
const keywords = extractToolSchemaModelCompat(modelOrCompat)?.unsupportedToolSchemaKeywords ?? [];
return new Set(
normalizeStringEntries(
keywords.filter((keyword): keyword is string => typeof keyword === "string"),
),
);
}
/** Whether empty `items: {}` on array schemas must be omitted for this model/provider. */
export function shouldOmitEmptyArrayItems(
modelOrCompat: { compat?: unknown } | ToolSchemaModelCompat | undefined,
): boolean {
return extractToolSchemaModelCompat(modelOrCompat)?.omitEmptyArrayItems === true;
}
export type ToolParameterSchemaOptions = {
modelProvider?: string;
modelId?: string;
modelCompat?: ToolSchemaModelCompat;
};
const MAX_TOOL_PARAMETER_SCHEMA_CACHE_ENTRIES_PER_SCHEMA = 8;
const toolParameterSchemaCache = new WeakMap<object, Array<{ key: string; value: TSchema }>>();
function resolveToolParameterSchemaCacheKey(
options: ToolParameterSchemaOptions | undefined,
): string {
const normalizedProvider = normalizeLowercaseStringOrEmpty(options?.modelProvider);
const normalizedModelId = normalizeLowercaseStringOrEmpty(options?.modelId);
const toolSchemaProfile = normalizeLowercaseStringOrEmpty(
options?.modelCompat?.toolSchemaProfile,
);
const unsupportedKeywords = Array.from(
resolveUnsupportedToolSchemaKeywords(options?.modelCompat),
).toSorted();
const omitEmptyArrayItems = shouldOmitEmptyArrayItems(options?.modelCompat);
return JSON.stringify([
normalizedProvider,
normalizedModelId,
toolSchemaProfile,
unsupportedKeywords,
omitEmptyArrayItems,
]);
}
function getCachedToolParameterSchema(schema: object, key: string): TSchema | undefined {
return toolParameterSchemaCache.get(schema)?.find((entry) => entry.key === key)?.value;
}
function rememberCachedToolParameterSchema(schema: object, key: string, value: TSchema): TSchema {
const entries = toolParameterSchemaCache.get(schema) ?? [];
toolParameterSchemaCache.set(
schema,
[{ key, value }, ...entries.filter((entry) => entry.key !== key)].slice(
0,
MAX_TOOL_PARAMETER_SCHEMA_CACHE_ENTRIES_PER_SCHEMA,
),
);
return value;
}
function isGeminiModelId(modelId: string): boolean {
return /(?:^|[/:])gemini(?:$|[-/:.])/.test(modelId);
}
function extractEnumValues(schema: unknown): unknown[] | undefined {
if (!schema || typeof schema !== "object") {
return undefined;
}
const record = schema as Record<string, unknown>;
if (Array.isArray(record.enum)) {
return record.enum;
}
if ("const" in record) {
return [record.const];
}
const variants = Array.isArray(record.anyOf)
? record.anyOf
: Array.isArray(record.oneOf)
? record.oneOf
: null;
if (variants) {
const values = variants.flatMap((variant) => {
const extracted = extractEnumValues(variant);
return extracted ?? [];
});
return values.length > 0 ? values : undefined;
}
return undefined;
}
function mergePropertySchemas(existing: unknown, incoming: unknown): unknown {
if (!existing) {
return incoming;
}
if (!incoming) {
return existing;
}
const existingEnum = extractEnumValues(existing);
const incomingEnum = extractEnumValues(incoming);
if (existingEnum || incomingEnum) {
const values = uniqueValues([...(existingEnum ?? []), ...(incomingEnum ?? [])]);
const merged: Record<string, unknown> = {};
for (const source of [existing, incoming]) {
if (!source || typeof source !== "object") {
continue;
}
const record = source as Record<string, unknown>;
for (const key of ["title", "description", "default"]) {
if (!(key in merged) && key in record) {
merged[key] = record[key];
}
}
}
const types = new Set(values.map((value) => typeof value));
if (types.size === 1) {
merged.type = Array.from(types)[0];
}
merged.enum = values;
return merged;
}
return existing;
}
type FlattenableVariantKey = "anyOf" | "oneOf";
type TopLevelConditionalKey = FlattenableVariantKey | "allOf";
function setOwnSchemaProperty(target: Record<string, unknown>, key: string, value: unknown): void {
Object.defineProperty(target, key, {
value,
enumerable: true,
configurable: true,
writable: true,
});
}
function hasTopLevelArrayKeyword(
schemaRecord: Record<string, unknown>,
key: TopLevelConditionalKey,
): boolean {
return Array.isArray(schemaRecord[key]);
}
function getFlattenableVariantKey(
schemaRecord: Record<string, unknown>,
): FlattenableVariantKey | null {
if (hasTopLevelArrayKeyword(schemaRecord, "anyOf")) {
return "anyOf";
}
if (hasTopLevelArrayKeyword(schemaRecord, "oneOf")) {
return "oneOf";
}
return null;
}
function getTopLevelConditionalKey(
schemaRecord: Record<string, unknown>,
): TopLevelConditionalKey | null {
return (
getFlattenableVariantKey(schemaRecord) ??
(hasTopLevelArrayKeyword(schemaRecord, "allOf") ? "allOf" : null)
);
}
function hasTopLevelObjectSchema(
schemaRecord: Record<string, unknown>,
conditionalKey: TopLevelConditionalKey | null,
): boolean {
return (
schemaRecord.type === "object" &&
isSchemaRecord(schemaRecord.properties) &&
conditionalKey === null
);
}
function isObjectLikeSchemaMissingType(
schemaRecord: Record<string, unknown>,
conditionalKey: TopLevelConditionalKey | null,
): boolean {
return (
!("type" in schemaRecord) &&
(isSchemaRecord(schemaRecord.properties) || Array.isArray(schemaRecord.required)) &&
conditionalKey === null
);
}
function isTypedObjectSchemaMissingValidProperties(
schemaRecord: Record<string, unknown>,
conditionalKey: TopLevelConditionalKey | null,
): boolean {
return (
schemaRecord.type === "object" &&
!isSchemaRecord(schemaRecord.properties) &&
conditionalKey === null
);
}
function isTrulyEmptySchema(schemaRecord: Record<string, unknown>): boolean {
return Object.keys(schemaRecord).length === 0;
}
function normalizeArraySchemasMissingItems(schema: unknown): unknown {
if (!isSchemaRecord(schema)) {
return schema;
}
let changed = false;
const nextSchema: Record<string, unknown> = { ...schema };
if (nextSchema.type === "array" && nextSchema.items === undefined) {
nextSchema.items = {};
changed = true;
}
const normalizeSchemaValue = (key: string): void => {
if (!(key in nextSchema)) {
return;
}
const value = nextSchema[key];
if (Array.isArray(value)) {
const normalized = value.map(normalizeArraySchemasMissingItems);
if (normalized.some((entry, index) => entry !== value[index])) {
nextSchema[key] = normalized;
changed = true;
}
return;
}
const normalized = normalizeArraySchemasMissingItems(value);
if (normalized !== value) {
nextSchema[key] = normalized;
changed = true;
}
};
for (const key of [
"items",
"contains",
"additionalProperties",
"propertyNames",
"not",
"if",
"then",
"else",
]) {
normalizeSchemaValue(key);
}
for (const key of ["anyOf", "oneOf", "allOf", "prefixItems"]) {
normalizeSchemaValue(key);
}
for (const key of [
"properties",
"patternProperties",
"dependentSchemas",
"$defs",
"definitions",
]) {
const value = nextSchema[key];
if (!isSchemaRecord(value)) {
continue;
}
let entriesChanged = false;
const normalizedEntries: Array<[string, unknown]> = Object.entries(value).map(
([entryKey, entryValue]) => {
const normalizedEntryValue = normalizeArraySchemasMissingItems(entryValue);
if (normalizedEntryValue !== entryValue) {
entriesChanged = true;
}
return [entryKey, normalizedEntryValue];
},
);
if (entriesChanged) {
nextSchema[key] = Object.fromEntries(normalizedEntries);
changed = true;
}
}
return changed ? nextSchema : schema;
}
function schemaAllowsArrayType(schema: Record<string, unknown>): boolean {
const type = schema.type;
return type === "array" || (Array.isArray(type) && type.includes("array"));
}
const ARRAY_ITEMS_SCHEMA_OBJECT_KEYS = new Set([
"additionalProperties",
"contains",
"else",
"if",
"items",
"not",
"propertyNames",
"then",
]);
const ARRAY_ITEMS_SCHEMA_ARRAY_KEYS = new Set(["allOf", "anyOf", "oneOf", "prefixItems"]);
const ARRAY_ITEMS_SCHEMA_MAP_KEYS = new Set([
"$defs",
"definitions",
"dependentSchemas",
"patternProperties",
"properties",
]);
function stripEmptyArrayItemsFromArraySchemas(schema: unknown): unknown {
if (Array.isArray(schema)) {
let changed = false;
const entries = schema.map((entry) => {
const next = stripEmptyArrayItemsFromArraySchemas(entry);
changed ||= next !== entry;
return next;
});
return changed ? entries : schema;
}
if (!isSchemaRecord(schema)) {
return schema;
}
let changed = false;
const entries = Object.entries(schema).flatMap(([key, value]) => {
if (
key === "items" &&
schemaAllowsArrayType(schema) &&
isSchemaRecord(value) &&
isTrulyEmptySchema(value)
) {
changed = true;
return [];
}
if (ARRAY_ITEMS_SCHEMA_OBJECT_KEYS.has(key)) {
const next = stripEmptyArrayItemsFromArraySchemas(value);
changed ||= next !== value;
return [[key, next] as const];
}
if (ARRAY_ITEMS_SCHEMA_ARRAY_KEYS.has(key) && Array.isArray(value)) {
const next = stripEmptyArrayItemsFromArraySchemas(value);
changed ||= next !== value;
return [[key, next] as const];
}
if (ARRAY_ITEMS_SCHEMA_MAP_KEYS.has(key) && isSchemaRecord(value)) {
let mapChanged = false;
const next = Object.fromEntries(
Object.entries(value).map(([entryKey, entryValue]) => {
const entryNext = stripEmptyArrayItemsFromArraySchemas(entryValue);
mapChanged ||= entryNext !== entryValue;
return [entryKey, entryNext] as const;
}),
);
changed ||= mapChanged;
return [[key, mapChanged ? next : value] as const];
}
return [[key, value] as const];
});
return changed ? Object.fromEntries(entries) : schema;
}
type SchemaDefs = {
$defs: Map<string, unknown>;
definitions: Map<string, unknown>;
};
function copySchemaMeta(from: Record<string, unknown>, to: Record<string, unknown>): void {
for (const key of ["title", "description", "default"] as const) {
if (key in from && from[key] !== undefined) {
to[key] = from[key];
}
}
}
function extendSchemaDefs(
defs: SchemaDefs | undefined,
schema: Record<string, unknown>,
): SchemaDefs | undefined {
const defsEntry =
schema.$defs && typeof schema.$defs === "object" && !Array.isArray(schema.$defs)
? (schema.$defs as Record<string, unknown>)
: undefined;
const legacyDefsEntry =
schema.definitions &&
typeof schema.definitions === "object" &&
!Array.isArray(schema.definitions)
? (schema.definitions as Record<string, unknown>)
: undefined;
if (!defsEntry && !legacyDefsEntry) {
return defs;
}
const next: SchemaDefs = defs
? {
$defs: new Map(defs.$defs),
definitions: new Map(defs.definitions),
}
: {
$defs: new Map<string, unknown>(),
definitions: new Map<string, unknown>(),
};
if (defsEntry) {
for (const [key, value] of Object.entries(defsEntry)) {
next.$defs.set(key, value);
}
}
if (legacyDefsEntry) {
for (const [key, value] of Object.entries(legacyDefsEntry)) {
next.definitions.set(key, value);
}
}
return next;
}
function decodeJsonPointerSegment(segment: string): string {
return segment.replaceAll("~1", "/").replaceAll("~0", "~");
}
function resolveJsonPointerPath(value: unknown, segments: string[]): unknown {
let current = value;
for (const segment of segments) {
if (!current || typeof current !== "object") {
return undefined;
}
const key = decodeJsonPointerSegment(segment);
if (Array.isArray(current)) {
const index = Number(key);
if (!Number.isInteger(index) || index < 0 || index >= current.length) {
return undefined;
}
current = current[index];
continue;
}
const record = current as Record<string, unknown>;
if (!Object.hasOwn(record, key)) {
return undefined;
}
current = record[key];
}
return current;
}
function resolveLocalJsonPointer(rootDocument: unknown, ref: string): unknown {
if (!ref.startsWith("#/")) {
return undefined;
}
return resolveJsonPointerPath(rootDocument, ref.slice(2).split("/"));
}
const SCHEMA_MAP_KEYS = new Set([
"$defs",
"definitions",
"dependentSchemas",
"patternProperties",
"properties",
]);
const SCHEMA_OBJECT_KEYS = new Set([
"additionalProperties",
"contains",
"else",
"if",
"items",
"not",
"propertyNames",
"then",
]);
const SCHEMA_ARRAY_KEYS = new Set(["allOf", "anyOf", "items", "oneOf", "prefixItems"]);
const SCHEMA_LITERAL_KEYS = new Set(["const", "default", "enum", "examples"]);
function tryResolveLocalRef(
ref: string,
defs: SchemaDefs | undefined,
rootDocument: unknown,
): unknown {
const match = ref.match(/^#\/(\$defs|definitions)\/([^/]+)(?:\/(.*))?$/);
if (match && defs) {
const namespace = match[1] === "$defs" ? defs.$defs : defs.definitions;
const name = decodeJsonPointerSegment(match[2] ?? "");
const resolved = name ? namespace.get(name) : undefined;
if (resolved !== undefined) {
const remainingPath = match[3] ? match[3].split("/") : [];
return resolveJsonPointerPath(resolved, remainingPath);
}
}
return resolveLocalJsonPointer(rootDocument, ref);
}
function inlineLocalSchemaRefsWithDefs(
schema: unknown,
defs: SchemaDefs | undefined,
refStack: Set<string> | undefined,
state: { unresolvedLocalRefs: boolean },
rootDocument: unknown,
): unknown {
if (!schema || typeof schema !== "object") {
return schema;
}
if (Array.isArray(schema)) {
return schema.map((entry) =>
inlineLocalSchemaRefsWithDefs(entry, defs, refStack, state, rootDocument),
);
}
const obj = schema as Record<string, unknown>;
const nextDefs = extendSchemaDefs(defs, obj);
const refValue = typeof obj.$ref === "string" ? obj.$ref : undefined;
if (refValue) {
if (refStack?.has(refValue)) {
return {};
}
const resolved = tryResolveLocalRef(refValue, nextDefs, rootDocument);
if (resolved === undefined) {
if (refValue.startsWith("#/")) {
state.unresolvedLocalRefs = true;
}
return { ...obj };
}
const nextRefStack = refStack ? new Set(refStack) : new Set<string>();
nextRefStack.add(refValue);
const inlined = inlineLocalSchemaRefsWithDefs(
resolved,
nextDefs,
nextRefStack,
state,
rootDocument,
);
if (!inlined || typeof inlined !== "object" || Array.isArray(inlined)) {
return inlined;
}
const result: Record<string, unknown> = { ...(inlined as Record<string, unknown>) };
copySchemaMeta(obj, result);
if (obj.nullable === true) {
result.nullable = true;
}
return result;
}
const result: Record<string, unknown> = {};
for (const [key, value] of Object.entries(obj)) {
if (key === "$defs" || key === "definitions" || key === "components") {
continue;
}
if (SCHEMA_LITERAL_KEYS.has(key)) {
setOwnSchemaProperty(result, key, value);
continue;
}
if (SCHEMA_MAP_KEYS.has(key) && isSchemaRecord(value)) {
setOwnSchemaProperty(
result,
key,
Object.fromEntries(
Object.entries(value).map(([entryKey, entryValue]) => [
entryKey,
inlineLocalSchemaRefsWithDefs(entryValue, nextDefs, refStack, state, rootDocument),
]),
),
);
continue;
}
if (SCHEMA_OBJECT_KEYS.has(key) && isSchemaRecord(value)) {
setOwnSchemaProperty(
result,
key,
inlineLocalSchemaRefsWithDefs(value, nextDefs, refStack, state, rootDocument),
);
continue;
}
if (SCHEMA_ARRAY_KEYS.has(key) && Array.isArray(value)) {
setOwnSchemaProperty(
result,
key,
value.map((entry) =>
inlineLocalSchemaRefsWithDefs(entry, nextDefs, refStack, state, rootDocument),
),
);
continue;
}
setOwnSchemaProperty(result, key, value);
}
if (state.unresolvedLocalRefs) {
if ("$defs" in obj) {
result.$defs = obj.$defs;
}
if ("definitions" in obj) {
result.definitions = obj.definitions;
}
if ("components" in obj) {
result.components = obj.components;
}
}
return result;
}
/** Inline local $ref pointers so providers receive self-contained tool schemas. */
function inlineLocalToolSchemaRefs(schema: unknown): TSchema {
if (!schema || typeof schema !== "object") {
return schema as TSchema;
}
const defs = extendSchemaDefs(undefined, schema as Record<string, unknown>);
return inlineLocalSchemaRefsWithDefs(
schema,
defs,
undefined,
{
unresolvedLocalRefs: false,
},
schema,
) as TSchema;
}
const OPENAPI_SCHEMA_ANNOTATION_KEYS = new Set([
"discriminator",
"externalDocs",
"readOnly",
"writeOnly",
"xml",
"example",
]);
function appendNullSchemaType(type: unknown): unknown {
if (type === "null") {
return type;
}
if (typeof type === "string") {
return [type, "null"];
}
if (Array.isArray(type)) {
return type.includes("null") ? type : [...type, "null"];
}
return type;
}
function isNullSchemaLike(schema: unknown): boolean {
if (!isSchemaRecord(schema)) {
return false;
}
if (schema.type === "null") {
return true;
}
if (Array.isArray(schema.type) && schema.type.includes("null")) {
return true;
}
if ("const" in schema && schema.const === null) {
return true;
}
return Array.isArray(schema.enum) && schema.enum.includes(null);
}
function hasOpenApiComposition(schema: Record<string, unknown>): boolean {
return ["allOf", "anyOf", "oneOf"].some((key) => Array.isArray(schema[key]));
}
function schemaCompositionAlreadyAllowsNull(schema: Record<string, unknown>): boolean {
return (
(Array.isArray(schema.anyOf) && schema.anyOf.some(isNullSchemaLike)) ||
(Array.isArray(schema.oneOf) && schema.oneOf.some(isNullSchemaLike))
);
}
function wrapNullableComposedSchema(schema: Record<string, unknown>): Record<string, unknown> {
if (schemaCompositionAlreadyAllowsNull(schema)) {
return schema;
}
const wrapped: Record<string, unknown> = {
anyOf: [schema, { type: "null" }],
};
copySchemaMeta(schema, wrapped);
return wrapped;
}
function normalizeOpenApiSchemaKeywords(schema: unknown): unknown {
if (Array.isArray(schema)) {
let changed = false;
const normalized = schema.map((entry) => {
const next = normalizeOpenApiSchemaKeywords(entry);
changed ||= next !== entry;
return next;
});
return changed ? normalized : schema;
}
if (!isSchemaRecord(schema)) {
return schema;
}
let changed = false;
const nullable = schema.nullable === true;
const normalized: Record<string, unknown> = {};
for (const [key, value] of Object.entries(schema)) {
if (key === "nullable" || OPENAPI_SCHEMA_ANNOTATION_KEYS.has(key)) {
changed = true;
continue;
}
if (SCHEMA_LITERAL_KEYS.has(key)) {
normalized[key] = value;
continue;
}
if (SCHEMA_MAP_KEYS.has(key) && isSchemaRecord(value)) {
let mapChanged = false;
const next = Object.fromEntries(
Object.entries(value).map(([entryKey, entryValue]) => {
const nextEntry = normalizeOpenApiSchemaKeywords(entryValue);
mapChanged ||= nextEntry !== entryValue;
return [entryKey, nextEntry];
}),
);
normalized[key] = mapChanged ? next : value;
changed ||= mapChanged;
continue;
}
if (key === "components") {
normalized[key] = value;
continue;
}
if (SCHEMA_OBJECT_KEYS.has(key) && isSchemaRecord(value)) {
const next = normalizeOpenApiSchemaKeywords(value);
normalized[key] = next;
changed ||= next !== value;
continue;
}
if (SCHEMA_ARRAY_KEYS.has(key) && Array.isArray(value)) {
const next = value.map(normalizeOpenApiSchemaKeywords);
normalized[key] = next;
changed ||= next.some((entry, index) => entry !== value[index]);
continue;
}
normalized[key] = value;
}
if (nullable) {
if (hasOpenApiComposition(normalized)) {
return wrapNullableComposedSchema(normalized);
}
if ("type" in normalized) {
const nextType = appendNullSchemaType(normalized.type);
if (nextType !== normalized.type) {
normalized.type = nextType;
}
}
if (Array.isArray(normalized.enum) && !normalized.enum.includes(null)) {
normalized.enum = [...normalized.enum, null];
}
}
return changed || nullable ? normalized : schema;
}
function normalizeToolParameterSchemaUncached(
schema: unknown,
options?: ToolParameterSchemaOptions,
): TSchema {
const inlinedSchema = normalizeOpenApiSchemaKeywords(inlineLocalToolSchemaRefs(schema));
const schemaRecord =
inlinedSchema && typeof inlinedSchema === "object"
? (inlinedSchema as Record<string, unknown>)
: undefined;
if (!schemaRecord) {
return inlinedSchema as TSchema;
}
// Provider quirks:
// - Gemini rejects several JSON Schema keywords, so we scrub those.
// - OpenAI rejects function tool schemas unless the *top-level* is `type: "object"`.
// (TypeBox root unions compile to `{ anyOf: [...] }` without `type`).
// - Anthropic expects full JSON Schema draft 2020-12 compliance.
// - xAI rejects validation-constraint keywords (minLength, maxLength, etc.) outright.
//
// Normalize once here so callers can always pass `tools` through unchanged.
const normalizedProvider = normalizeLowercaseStringOrEmpty(options?.modelProvider);
const normalizedModelId = normalizeLowercaseStringOrEmpty(options?.modelId);
const normalizedToolSchemaProfile = normalizeLowercaseStringOrEmpty(
options?.modelCompat?.toolSchemaProfile,
);
const isGeminiProvider =
normalizedProvider.includes("google") ||
normalizedProvider.includes("gemini") ||
isGeminiModelId(normalizedModelId) ||
normalizedToolSchemaProfile === "gemini";
const isAnthropicProvider = normalizedProvider.includes("anthropic");
const unsupportedToolSchemaKeywords = resolveUnsupportedToolSchemaKeywords(options?.modelCompat);
const omitEmptyArrayItems = shouldOmitEmptyArrayItems(options?.modelCompat);
function applyProviderCleaning(s: unknown): TSchema {
const normalizedSchema = normalizeArraySchemasMissingItems(s);
const arrayItemsCompatibleSchema = omitEmptyArrayItems
? stripEmptyArrayItemsFromArraySchemas(normalizedSchema)
: normalizedSchema;
if (isGeminiProvider && !isAnthropicProvider) {
const geminiCompatibleSchema = cleanSchemaForGemini(arrayItemsCompatibleSchema);
return unsupportedToolSchemaKeywords.size > 0
? (stripUnsupportedSchemaKeywords(
geminiCompatibleSchema,
unsupportedToolSchemaKeywords,
) as TSchema)
: geminiCompatibleSchema;
}
if (unsupportedToolSchemaKeywords.size > 0) {
return stripUnsupportedSchemaKeywords(
arrayItemsCompatibleSchema,
unsupportedToolSchemaKeywords,
) as TSchema;
}
return arrayItemsCompatibleSchema as TSchema;
}
const conditionalKey = getTopLevelConditionalKey(schemaRecord);
const flattenableVariantKey = getFlattenableVariantKey(schemaRecord);
if (hasTopLevelObjectSchema(schemaRecord, conditionalKey)) {
return applyProviderCleaning(schemaRecord);
}
if (isObjectLikeSchemaMissingType(schemaRecord, conditionalKey)) {
return applyProviderCleaning({
...schemaRecord,
type: "object",
properties: isSchemaRecord(schemaRecord.properties) ? schemaRecord.properties : {},
});
}
if (isTypedObjectSchemaMissingValidProperties(schemaRecord, conditionalKey)) {
return applyProviderCleaning({ ...schemaRecord, properties: {} });
}
if (!flattenableVariantKey) {
if (isTrulyEmptySchema(schemaRecord)) {
// Handle the proven MCP no-parameter case: a truly empty schema object.
return applyProviderCleaning({ type: "object", properties: {} });
}
if (conditionalKey === "allOf") {
// Top-level `allOf` is not safely flattenable with the same heuristics we
// use for unions. Keep it explicit rather than silently rewriting it.
return applyProviderCleaning(inlinedSchema);
}
return applyProviderCleaning(inlinedSchema);
}
const variants = schemaRecord[flattenableVariantKey] as unknown[];
const mergedProperties: Record<string, unknown> = {};
const requiredCounts = new Map<string, number>();
let objectVariants = 0;
for (const entry of variants) {
if (!entry || typeof entry !== "object") {
continue;
}
const props = (entry as { properties?: unknown }).properties;
if (!props || typeof props !== "object") {
continue;
}
objectVariants += 1;
for (const [key, value] of Object.entries(props as Record<string, unknown>)) {
if (!(key in mergedProperties)) {
mergedProperties[key] = value;
continue;
}
mergedProperties[key] = mergePropertySchemas(mergedProperties[key], value);
}
const required = Array.isArray((entry as { required?: unknown }).required)
? (entry as { required: unknown[] }).required
: [];
for (const key of required) {
if (typeof key !== "string") {
continue;
}
requiredCounts.set(key, (requiredCounts.get(key) ?? 0) + 1);
}
}
const baseRequired = Array.isArray(schemaRecord.required)
? schemaRecord.required.filter((key) => typeof key === "string")
: undefined;
const mergedRequired =
baseRequired && baseRequired.length > 0
? baseRequired
: objectVariants > 0
? Array.from(requiredCounts.entries())
.filter(([, count]) => count === objectVariants)
.map(([key]) => key)
: undefined;
const nextSchema: Record<string, unknown> = { ...schemaRecord };
const flattenedSchema = {
type: "object",
...(typeof nextSchema.title === "string" ? { title: nextSchema.title } : {}),
...(typeof nextSchema.description === "string" ? { description: nextSchema.description } : {}),
properties:
Object.keys(mergedProperties).length > 0 ? mergedProperties : (schemaRecord.properties ?? {}),
...(mergedRequired && mergedRequired.length > 0 ? { required: mergedRequired } : {}),
additionalProperties:
"additionalProperties" in schemaRecord ? schemaRecord.additionalProperties : true,
};
// Flatten union schemas into a single object schema:
// - Gemini doesn't allow top-level `type` together with `anyOf`.
// - OpenAI rejects schemas without top-level `type: "object"`.
// - Anthropic accepts proper JSON Schema with constraints.
// Merging properties preserves useful enums like `action` while keeping schemas portable.
return applyProviderCleaning(flattenedSchema);
}
/** Return a provider-compatible JSON schema for a model-facing tool. */
export function normalizeToolParameterSchema(
schema: unknown,
options?: ToolParameterSchemaOptions,
): TSchema {
if (!schema || typeof schema !== "object") {
return normalizeToolParameterSchemaUncached(schema, options);
}
const cacheKey = resolveToolParameterSchemaCacheKey(options);
const cached = getCachedToolParameterSchema(schema, cacheKey);
if (cached) {
return cached;
}
return rememberCachedToolParameterSchema(
schema,
cacheKey,
normalizeToolParameterSchemaUncached(schema, options),
);
}

View File

@@ -0,0 +1,38 @@
type AnthropicAuthModel = {
provider?: string;
authHeader?: boolean;
headers?: Record<string, string>;
};
export function usesFoundryBearerAuth(model: AnthropicAuthModel): boolean {
return (
model.provider === "microsoft-foundry" &&
(model.authHeader === true || hasBearerAuthorizationHeader(model.headers))
);
}
function hasBearerAuthorizationHeader(headers?: Record<string, string>): boolean {
if (!headers) {
return false;
}
return Object.entries(headers).some(
([key, value]) => key.toLowerCase() === "authorization" && /^bearer\s+\S+/i.test(value.trim()),
);
}
export function omitFoundryBearerCredentialHeaders(
headers?: Record<string, string>,
): Record<string, string> | undefined {
if (!headers) {
return undefined;
}
const next: Record<string, string> = {};
for (const [key, value] of Object.entries(headers)) {
const lower = key.toLowerCase();
if (lower === "authorization" || lower === "x-api-key" || lower === "api-key") {
continue;
}
next[key] = value;
}
return Object.keys(next).length > 0 ? next : undefined;
}

View File

@@ -0,0 +1,98 @@
// Model-bound thinking cannot be exposed or replayed after a model switch.
import { resolveClaudeFable5ModelIdentity, resolveClaudeModelIdentity } from "@openclaw/llm-core";
import { normalizeLowercaseStringOrEmpty } from "@openclaw/normalization-core/string-coerce";
export {
resolveClaudeFable5ModelIdentity,
resolveClaudeModelIdentity,
resolveClaudeNativeThinkingLevelMap,
supportsClaudeAdaptiveThinking,
supportsClaudeNativeMaxEffort,
supportsClaudeNativeXhighEffort,
} from "@openclaw/llm-core";
type ReplayModelRef = {
provider?: string;
api?: string;
modelId?: string;
responseModelId?: string;
modelParams?: Record<string, unknown>;
};
function normalizeModelId(modelId?: string): string {
const normalized = normalizeLowercaseStringOrEmpty(modelId);
const unprefixed = normalized.startsWith("anthropic/")
? normalized.slice("anthropic/".length)
: normalized;
return unprefixed.replace(/[._\s]+/g, "-");
}
function normalizeApi(api?: string): string {
const normalized = normalizeLowercaseStringOrEmpty(api);
return normalized === "openclaw-anthropic-messages-transport" ? "anthropic-messages" : normalized;
}
function hasConcreteResponseModel(ref: ReplayModelRef): boolean {
const responseModelId = normalizeModelId(ref.responseModelId);
// Deployment APIs may echo the requested alias. Only a different response
// model proves the backing identity and overrides configured metadata.
return responseModelId.length > 0 && responseModelId !== normalizeModelId(ref.modelId);
}
export function usesClaudeFable5MessagesContract(model: {
id?: string;
params?: Record<string, unknown>;
api?: string;
}): boolean {
return (
normalizeApi(model.api) === "anthropic-messages" &&
resolveClaudeFable5ModelIdentity(model) !== undefined
);
}
export function requiresClaudeAdaptiveThinking(model: {
id?: string;
params?: Record<string, unknown>;
api?: string;
}): boolean {
if (normalizeApi(model.api) !== "anthropic-messages") {
return false;
}
const modelId = resolveClaudeModelIdentity(model);
return (
resolveClaudeFable5ModelIdentity(model) !== undefined ||
/(?:^|-)claude-mythos-preview(?=$|[^a-z0-9])/.test(modelId)
);
}
function resolveReplayFableIdentity(ref: ReplayModelRef): string | undefined {
if (normalizeApi(ref.api) !== "anthropic-messages") {
return undefined;
}
if (hasConcreteResponseModel(ref)) {
return resolveClaudeFable5ModelIdentity({ id: ref.responseModelId });
}
return resolveClaudeFable5ModelIdentity({ id: ref.modelId, params: ref.modelParams });
}
export function resolveModelBoundThinkingReplayMode(params: {
source: ReplayModelRef;
target: ReplayModelRef;
}): "default" | "preserve" | "drop" {
const sourceApi = normalizeApi(params.source.api);
const targetApi = normalizeApi(params.target.api);
const sourceIdentity = resolveReplayFableIdentity(params.source);
const targetIdentity = resolveReplayFableIdentity(params.target);
const sameRoute =
normalizeLowercaseStringOrEmpty(params.source.provider) ===
normalizeLowercaseStringOrEmpty(params.target.provider) &&
sourceApi === targetApi &&
normalizeModelId(params.source.modelId) === normalizeModelId(params.target.modelId);
if (!sourceIdentity && !targetIdentity) {
return "default";
}
if (!sourceIdentity && !hasConcreteResponseModel(params.source) && targetIdentity && sameRoute) {
return "preserve";
}
const sameModel = sourceApi === targetApi && sourceIdentity === targetIdentity;
return sameModel ? "preserve" : "drop";
}

View File

@@ -0,0 +1,55 @@
import type { AssistantMessageDiagnostic } from "../types.js";
type AnthropicRefusalOutput = {
stopReason: string;
errorMessage?: string;
diagnostics?: AssistantMessageDiagnostic[];
};
type AnthropicRefusalDetails = {
category: string | null;
explanation: string | null;
};
function readNullableString(value: unknown): string | null {
return typeof value === "string" && value.trim() ? value.trim() : null;
}
function readAnthropicRefusalDetails(value: unknown): AnthropicRefusalDetails {
if (!value || typeof value !== "object") {
return { category: null, explanation: null };
}
const details = value as Record<string, unknown>;
return {
category: readNullableString(details.category),
explanation: readNullableString(details.explanation),
};
}
function formatAnthropicRefusalMessage(details: AnthropicRefusalDetails): string {
const category = details.category ? ` (category: ${details.category})` : "";
const explanation = details.explanation ? `: ${details.explanation}` : ".";
return `Anthropic refusal${category}${explanation}`;
}
export function applyAnthropicRefusal(
output: AnthropicRefusalOutput,
stopDetails: unknown,
provider: string,
): void {
const details = readAnthropicRefusalDetails(stopDetails);
output.stopReason = "error";
output.errorMessage = formatAnthropicRefusalMessage(details);
output.diagnostics = [
...(output.diagnostics ?? []),
{
type: "provider_refusal",
timestamp: Date.now(),
details: {
provider,
category: details.category,
explanation: details.explanation,
},
},
];
}

View File

@@ -0,0 +1,83 @@
import type { AssistantMessageDiagnostic } from "../types.js";
/** Anthropic beta that re-serves safety refusals on an allowed fallback model. */
export const ANTHROPIC_SERVER_SIDE_FALLBACK_BETA = "server-side-fallback-2026-06-01";
// Anthropic documents claude-opus-4-8 as the allowed fallback for claude-fable-5.
export const CLAUDE_FABLE_5_FALLBACK_MODEL = "claude-opus-4-8";
// Fallback-served turns bill at the serving model's rates.
export const CLAUDE_FABLE_5_FALLBACK_MODEL_COST = {
input: 5,
output: 25,
cacheRead: 0.5,
cacheWrite: 6.25,
} as const;
export function buildAnthropicServerSideFallbacks(): Array<{ model: string }> {
return [{ model: CLAUDE_FABLE_5_FALLBACK_MODEL }];
}
export type AnthropicFallbackBoundary = {
fromModel: string | null;
toModel: string | null;
};
function readBoundaryModel(value: unknown): string | null {
if (!value || typeof value !== "object") {
return null;
}
const model = (value as { model?: unknown }).model;
return typeof model === "string" && model.trim() ? model : null;
}
/** Reads a `fallback` content block marking where one model's output gives way to the next. */
export function readAnthropicFallbackBoundary(block: unknown): AnthropicFallbackBoundary | null {
if (!block || typeof block !== "object") {
return null;
}
const record = block as { type?: unknown; from?: unknown; to?: unknown };
if (record.type !== "fallback") {
return null;
}
return {
fromModel: readBoundaryModel(record.from),
toModel: readBoundaryModel(record.to),
};
}
/**
* Drops pre-fallback thinking/tool calls while preserving the text prefix that
* the serving model continued. Dropped tool calls must never execute or replay.
*/
export function applyAnthropicFallbackBoundary(params: {
output: {
content: Array<{ type: string }>;
responseModel?: string;
diagnostics?: AssistantMessageDiagnostic[];
};
boundary: AnthropicFallbackBoundary;
provider: string;
}): void {
const { output, boundary } = params;
const survivors = output.content.filter((block) => block.type === "text");
for (const survivor of survivors) {
delete (survivor as { textSignature?: string }).textSignature;
}
output.content.splice(0, output.content.length, ...survivors);
if (boundary.toModel) {
output.responseModel = boundary.toModel;
}
output.diagnostics = [
...(output.diagnostics ?? []),
{
type: "provider_fallback",
timestamp: Date.now(),
details: {
provider: params.provider,
fromModel: boundary.fromModel,
toModel: boundary.toModel,
},
},
];
}

View File

@@ -0,0 +1,58 @@
type ReplayMessage = {
role?: unknown;
content?: unknown;
toolCallId?: unknown;
};
export const ANTHROPIC_OMITTED_REASONING_TEXT = "[assistant reasoning omitted]";
function asReplayMessage(value: unknown): ReplayMessage | undefined {
return value && typeof value === "object" ? (value as ReplayMessage) : undefined;
}
/**
* Anthropic tool results continue the preceding assistant turn. Preserve that
* turn's signed thinking even when the next request disables new thinking.
*/
export function findActiveAnthropicToolTurnAssistantIndex(messages: readonly unknown[]): number {
const toolResultIds = new Set<string>();
let index = messages.length - 1;
while (index >= 0) {
const message = asReplayMessage(messages[index]);
if (message?.role !== "toolResult") {
break;
}
if (typeof message.toolCallId === "string") {
toolResultIds.add(message.toolCallId);
}
index -= 1;
}
if (toolResultIds.size === 0) {
return -1;
}
const assistant = asReplayMessage(messages[index]);
if (assistant?.role !== "assistant" || !Array.isArray(assistant.content)) {
return -1;
}
const toolCallIds = new Set<string>();
for (const block of assistant.content) {
if (!block || typeof block !== "object") {
continue;
}
const record = block as { type?: unknown; id?: unknown };
if (
(record.type === "toolCall" ||
record.type === "tool_use" ||
record.type === "function_call") &&
typeof record.id === "string"
) {
toolCallIds.add(record.id);
}
}
return [...toolResultIds].every((toolCallId) => toolCallIds.has(toolCallId)) ? index : -1;
}

View File

@@ -0,0 +1,163 @@
import { isRecord } from "@openclaw/normalization-core/record-coerce";
import { projectRuntimeToolInputSchema } from "./tool-schema-json-projection.js";
type AnthropicToolDescriptor = {
readonly name: string;
readonly description: string;
readonly parameters: unknown;
};
type AnthropicProjectedTool = {
readonly originalName: string;
readonly wireName: string;
readonly description?: string;
readonly inputSchema: {
readonly type: "object";
readonly properties: Record<string, unknown>;
readonly required: string[];
};
};
export type AnthropicToolProjection = {
readonly inputToolCount: number;
readonly unavailableOriginalNames: ReadonlySet<string>;
readonly tools: readonly AnthropicProjectedTool[];
};
type AnthropicParallelToolChoice = {
readonly disable_parallel_tool_use?: boolean;
};
export type AnthropicProjectedToolChoice =
| ({ readonly type: "auto" } & AnthropicParallelToolChoice)
| ({ readonly type: "any" } & AnthropicParallelToolChoice)
| { readonly type: "none" }
| ({ readonly type: "tool"; readonly name: string } & AnthropicParallelToolChoice);
function isProviderSupportedViolation(violation: string): boolean {
return violation.endsWith(".$dynamicRef") || violation.endsWith(".$dynamicAnchor");
}
/** Snapshots direct/custom tool descriptors before Anthropic payload construction. */
export function projectAnthropicTools(
tools: readonly AnthropicToolDescriptor[],
toWireName: (name: string) => string,
): AnthropicToolProjection {
const projectedTools: AnthropicProjectedTool[] = [];
const unavailableOriginalNames = new Set<string>();
for (const tool of tools) {
let projectedTool: AnthropicProjectedTool;
let originalName: string | undefined;
try {
const name = tool.name;
originalName = name;
if (!name) {
continue;
}
const schemaProjection = projectRuntimeToolInputSchema(tool.parameters, `${name}.parameters`);
if (
!isRecord(schemaProjection.schema) ||
schemaProjection.violations.some((violation) => !isProviderSupportedViolation(violation))
) {
unavailableOriginalNames.add(name);
continue;
}
const properties = schemaProjection.schema.properties;
const required = schemaProjection.schema.required;
if (
(properties !== undefined && properties !== null && !isRecord(properties)) ||
(required !== undefined &&
required !== null &&
(!Array.isArray(required) || required.some((entry) => typeof entry !== "string")))
) {
unavailableOriginalNames.add(name);
continue;
}
let description: string | undefined;
try {
description = typeof tool.description === "string" ? tool.description : undefined;
} catch {
// Description is optional; keep the usable tool schema.
}
const wireName = toWireName(name);
projectedTool = {
originalName: name,
wireName,
...(description ? { description } : {}),
inputSchema: {
type: "object",
properties: (properties ?? {}) as Record<string, unknown>,
required: (required ?? []) as string[],
},
};
} catch {
// Direct/custom tool arrays can bypass the runtime quarantine.
if (originalName) {
unavailableOriginalNames.add(originalName);
}
continue;
}
const conflictingTool = projectedTools.find(
(entry) => entry.wireName === projectedTool.wireName,
);
if (conflictingTool && conflictingTool.originalName !== projectedTool.originalName) {
throw new Error(
`Anthropic tool names "${conflictingTool.originalName}" and "${projectedTool.originalName}" both map to "${projectedTool.wireName}"`,
);
}
projectedTools.push(projectedTool);
}
return {
inputToolCount: tools.length,
unavailableOriginalNames,
tools: projectedTools,
};
}
/** Keeps forced Anthropic tool choices aligned with the projected wire names. */
export function reconcileAnthropicToolChoice(
choice: AnthropicProjectedToolChoice,
projection: AnthropicToolProjection,
): AnthropicProjectedToolChoice | undefined {
if (projection.inputToolCount === 0) {
return choice;
}
if (choice.type === "tool") {
const requestedName = choice.name;
const originalMatch = projection.tools.find((tool) => tool.originalName === requestedName);
if (originalMatch) {
return { ...choice, name: originalMatch.wireName };
}
if (projection.unavailableOriginalNames.has(requestedName)) {
throw new Error(
`Anthropic tool_choice requested unavailable tool "${requestedName}" after schema conversion`,
);
}
const matchedTool = projection.tools.find((tool) => tool.wireName === requestedName);
if (!matchedTool) {
throw new Error(
`Anthropic tool_choice requested unavailable tool "${requestedName}" after schema conversion`,
);
}
return { ...choice, name: matchedTool.wireName };
}
if (projection.tools.length === 0) {
if (choice.type === "auto") {
return undefined;
}
if (choice.type === "any") {
throw new Error(
"Anthropic tool_choice requires a tool, but no tools survived schema conversion",
);
}
}
return choice;
}
/** Maps Claude Code wire names without trusting every direct/custom descriptor. */
export function resolveOriginalAnthropicToolName(
name: string,
projection: AnthropicToolProjection | undefined,
): string {
return projection?.tools.find((tool) => tool.wireName === name)?.originalName ?? name;
}

View File

@@ -0,0 +1,77 @@
import { describe, expect, it } from "vitest";
import { readLastAnthropicIterationUsage } from "./anthropic-usage.js";
describe("readLastAnthropicIterationUsage", () => {
it.each(["message", "compaction", "advisor_message"])(
"reads the final %s iteration as the context snapshot",
(type) => {
expect(
readLastAnthropicIterationUsage({
iterations: [
{
type: "message",
input_tokens: 1,
output_tokens: 2,
cache_read_input_tokens: 3,
cache_creation_input_tokens: 4,
},
{
type,
input_tokens: 12,
output_tokens: 15_104,
cache_read_input_tokens: 148_862,
cache_creation_input_tokens: 0,
},
],
}),
).toEqual({
state: "valid",
usage: {
contextPromptTokens: 148_874,
totalTokens: 163_978,
},
});
},
);
it("reports absent iterations separately from malformed iterations", () => {
expect(readLastAnthropicIterationUsage({ input_tokens: 1 })).toEqual({ state: "absent" });
});
it("does not reuse an earlier iteration when the final iteration is malformed", () => {
expect(
readLastAnthropicIterationUsage({
iterations: [
{
type: "message",
input_tokens: 12,
output_tokens: 15_104,
cache_read_input_tokens: 148_862,
cache_creation_input_tokens: 0,
},
{
type: "message",
input_tokens: "malformed",
output_tokens: 1,
cache_read_input_tokens: 0,
cache_creation_input_tokens: 0,
},
],
}),
).toEqual({ state: "invalid" });
});
it("rejects a final iteration with incomplete cache usage", () => {
expect(
readLastAnthropicIterationUsage({
iterations: [
{
type: "message",
input_tokens: 12,
output_tokens: 15_104,
},
],
}),
).toEqual({ state: "invalid" });
});
});

View File

@@ -0,0 +1,83 @@
type AnthropicUsagePayload = {
input_tokens?: unknown;
output_tokens?: unknown;
cache_read_input_tokens?: unknown;
cache_creation_input_tokens?: unknown;
iterations?: unknown;
};
export type AnthropicPromptUsageSnapshot = {
input: number;
cacheRead: number;
cacheWrite: number;
};
export type AnthropicIterationUsageSnapshot = {
contextPromptTokens: number;
totalTokens: number;
};
export type AnthropicIterationUsageResult =
| { state: "absent" }
| { state: "invalid" }
| { state: "valid"; usage: AnthropicIterationUsageSnapshot };
export function readAnthropicUsageTokenCount(value: unknown): number | undefined {
return typeof value === "number" && Number.isFinite(value) && value >= 0 ? value : undefined;
}
export function readAnthropicPromptUsageSnapshot(
usage: AnthropicUsagePayload,
): AnthropicPromptUsageSnapshot | undefined {
const input = readAnthropicUsageTokenCount(usage.input_tokens);
const cacheRead =
usage.cache_read_input_tokens == null
? 0
: readAnthropicUsageTokenCount(usage.cache_read_input_tokens);
const cacheWrite =
usage.cache_creation_input_tokens == null
? 0
: readAnthropicUsageTokenCount(usage.cache_creation_input_tokens);
if (input === undefined || cacheRead === undefined || cacheWrite === undefined) {
return undefined;
}
return { input, cacheRead, cacheWrite };
}
export function readLastAnthropicIterationUsage(
usage: AnthropicUsagePayload,
): AnthropicIterationUsageResult {
if (usage.iterations == null) {
return { state: "absent" };
}
if (!Array.isArray(usage.iterations) || usage.iterations.length === 0) {
return { state: "invalid" };
}
// Anthropic documents the final iteration as the true context window.
// Top-level cache fields remain cumulative billing totals across iterations.
const iteration = usage.iterations.at(-1);
if (!iteration || typeof iteration !== "object" || Array.isArray(iteration)) {
return { state: "invalid" };
}
const record = iteration as AnthropicUsagePayload;
const input = readAnthropicUsageTokenCount(record.input_tokens);
const cacheRead = readAnthropicUsageTokenCount(record.cache_read_input_tokens);
const cacheWrite = readAnthropicUsageTokenCount(record.cache_creation_input_tokens);
const outputTokens = readAnthropicUsageTokenCount(record.output_tokens);
if (
input === undefined ||
cacheRead === undefined ||
cacheWrite === undefined ||
outputTokens === undefined
) {
return { state: "invalid" };
}
const contextPromptTokens = input + cacheRead + cacheWrite;
return {
state: "valid",
usage: {
contextPromptTokens,
totalTokens: contextPromptTokens + outputTokens,
},
};
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,30 @@
// Azure deployment map tests cover model-to-deployment resolution.
import { describe, expect, it } from "vitest";
import {
parseAzureDeploymentNameMap,
resolveAzureDeploymentNameFromMap,
} from "./azure-deployment-map.js";
describe("Azure deployment name map", () => {
it("preserves equals signs inside deployment names", () => {
const map = parseAzureDeploymentNameMap("gpt-5=deployment=blue, ignored, gpt-4 = prod = east ");
expect(map.get("gpt-5")).toBe("deployment=blue");
expect(map.get("gpt-4")).toBe("prod = east");
expect(
resolveAzureDeploymentNameFromMap({
modelId: "gpt-5",
deploymentMap: "gpt-5=deployment=blue",
}),
).toBe("deployment=blue");
});
it("falls back to the model id when the map has no usable entry", () => {
expect(
resolveAzureDeploymentNameFromMap({
modelId: "gpt-5",
deploymentMap: "other=deployment,missing-value=",
}),
).toBe("gpt-5");
});
});

View File

@@ -0,0 +1,32 @@
/** Parses AZURE_OPENAI_DEPLOYMENT_MAP-style model=deployment entries. */
export function parseAzureDeploymentNameMap(value: string | undefined): Map<string, string> {
const map = new Map<string, string>();
if (!value) {
return map;
}
for (const entry of value.split(",")) {
const trimmed = entry.trim();
if (!trimmed) {
continue;
}
const separator = trimmed.indexOf("=");
if (separator <= 0) {
continue;
}
const modelId = trimmed.slice(0, separator).trim();
const deploymentName = trimmed.slice(separator + 1).trim();
if (!modelId || !deploymentName) {
continue;
}
map.set(modelId, deploymentName);
}
return map;
}
/** Resolves the Azure deployment name for a model id, falling back to the model id. */
export function resolveAzureDeploymentNameFromMap(params: {
modelId: string;
deploymentMap?: string;
}): string {
return parseAzureDeploymentNameMap(params.deploymentMap).get(params.modelId) || params.modelId;
}

View File

@@ -0,0 +1,29 @@
export function isTraditionalAzureOpenAIHost(hostname: string): boolean {
return (
hostname.endsWith(".openai.azure.com") || hostname.endsWith(".cognitiveservices.azure.com")
);
}
export function isOpenAICompatibleAzureResponsesBaseUrl(baseUrl: string): boolean {
let url: URL;
try {
url = new URL(baseUrl);
} catch {
return false;
}
if (isTraditionalAzureOpenAIHost(url.hostname)) {
return false;
}
const hostname = url.hostname.toLowerCase();
const isFoundryHost =
hostname.endsWith(".services.ai.azure.com") ||
hostname.endsWith(".api.cognitive.microsoft.com");
if (!isFoundryHost) {
return false;
}
const normalizedPath = url.pathname.replace(/\/+$/, "");
return normalizedPath === "/openai/v1" || normalizedPath.endsWith("/openai/v1");
}

View File

@@ -0,0 +1,73 @@
import { describe, expect, it } from "vitest";
import type { Model } from "../types.js";
import { testing } from "./azure-openai-responses.js";
const azureResponsesModel = {
id: "gpt-5.5",
name: "GPT-5.5",
api: "azure-openai-responses",
provider: "azure",
baseUrl: "https://example.openai.azure.com/openai/v1",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 200000,
maxTokens: 8192,
} satisfies Model<"azure-openai-responses">;
describe("azure-openai-responses", () => {
it("keeps traditional Azure OpenAI hosts on the AzureOpenAI client path", () => {
const config = testing.resolveAzureConfig(azureResponsesModel, {
azureResourceName: "example",
azureApiVersion: "v1",
});
expect(config).toEqual({
baseUrl: "https://example.openai.azure.com/openai/v1",
apiVersion: "v1",
});
expect(testing.isOpenAICompatibleAzureResponsesBaseUrl(config.baseUrl)).toBe(false);
expect(
testing.isOpenAICompatibleAzureResponsesBaseUrl(
"https://example.cognitiveservices.azure.com/openai/v1",
),
).toBe(false);
});
it("uses the OpenAI-compatible client path for Foundry /openai/v1 endpoints", () => {
expect(
testing.isOpenAICompatibleAzureResponsesBaseUrl(
"https://project.services.ai.azure.com/api/projects/demo/openai/v1",
),
).toBe(true);
expect(
testing.isOpenAICompatibleAzureResponsesBaseUrl(
"https://project.services.ai.azure.com/openai/v1",
),
).toBe(true);
expect(
testing.isOpenAICompatibleAzureResponsesBaseUrl(
"https://eastus.api.cognitive.microsoft.com/openai/v1",
),
).toBe(true);
});
it("does not treat non-v1 custom endpoints as OpenAI-compatible Responses bases", () => {
expect(
testing.isOpenAICompatibleAzureResponsesBaseUrl(
"https://project.services.ai.azure.com/api/projects/demo",
),
).toBe(false);
});
it("keeps private or APIM Azure OpenAI-compatible paths on the AzureOpenAI client path", () => {
expect(testing.isOpenAICompatibleAzureResponsesBaseUrl("https://aoai.internal/openai/v1")).toBe(
false,
);
expect(
testing.isOpenAICompatibleAzureResponsesBaseUrl(
"https://gateway.example.com/proxy/openai/v1",
),
).toBe(false);
});
});

View File

@@ -0,0 +1,252 @@
// Azure OpenAI Responses provider adapts Azure deployments to Responses API streams.
import OpenAI, { AzureOpenAI } from "openai";
import type { ResponseCreateParamsStreaming } from "openai/resources/responses/responses.js";
import { getEnvApiKey } from "../env-api-keys.js";
import { getAiTransportHost } from "../host.js";
import type {
Context,
Model,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { resolveAzureDeploymentNameFromMap } from "./azure-deployment-map.js";
import { isOpenAICompatibleAzureResponsesBaseUrl } from "./azure-openai-responses-client-compat.js";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.js";
import {
applyCommonResponsesParams,
convertResponsesMessages,
createResponsesAssistantOutput,
resolveResponsesReasoningEffort,
runResponsesStreamLifecycle,
} from "./openai-responses-shared.js";
import { buildBaseOptions } from "./simple-options.js";
const DEFAULT_AZURE_API_VERSION = "v1";
const AZURE_TOOL_CALL_PROVIDERS = new Set(["openai", "opencode", "azure-openai-responses"]);
function resolveDeploymentName(
model: Model<"azure-openai-responses">,
options?: AzureOpenAIResponsesOptions,
): string {
if (options?.azureDeploymentName) {
return options.azureDeploymentName;
}
return resolveAzureDeploymentNameFromMap({
modelId: model.id,
deploymentMap: process.env.AZURE_OPENAI_DEPLOYMENT_NAME_MAP,
});
}
function formatAzureOpenAIError(error: unknown): string {
if (error instanceof Error) {
const status = (error as Error & { status?: unknown }).status;
const statusCode = typeof status === "number" ? status : undefined;
if (statusCode !== undefined) {
return `Azure OpenAI API error (${statusCode}): ${error.message}`;
}
return error.message;
}
try {
return JSON.stringify(error);
} catch {
return String(error);
}
}
// Azure OpenAI Responses-specific options
export interface AzureOpenAIResponsesOptions extends StreamOptions {
reasoningEffort?: "minimal" | "low" | "medium" | "high" | "xhigh";
reasoningSummary?: "auto" | "detailed" | "concise" | null;
azureApiVersion?: string;
azureResourceName?: string;
azureBaseUrl?: string;
azureDeploymentName?: string;
}
/**
* Generate function for Azure OpenAI Responses API
*/
export const streamAzureOpenAIResponses: StreamFunction<
"azure-openai-responses",
AzureOpenAIResponsesOptions
> = (
model: Model<"azure-openai-responses">,
context: Context,
options?: AzureOpenAIResponsesOptions,
) => {
const stream = new AssistantMessageEventStream();
const output = createResponsesAssistantOutput(model, "azure-openai-responses");
// Start async processing
void runResponsesStreamLifecycle({
stream,
model,
output,
options,
createClient: () => {
const apiKey = options?.apiKey || getEnvApiKey(model.provider) || "";
return createClient(model, apiKey, options);
},
buildParams: () => buildParams(model, context, options, resolveDeploymentName(model, options)),
formatError: formatAzureOpenAIError,
});
return stream;
};
export const streamSimpleAzureOpenAIResponses: StreamFunction<
"azure-openai-responses",
SimpleStreamOptions
> = (model: Model<"azure-openai-responses">, context: Context, options?: SimpleStreamOptions) => {
const apiKey = options?.apiKey || getEnvApiKey(model.provider);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const base = buildBaseOptions(model, options, apiKey);
const reasoningEffort = resolveResponsesReasoningEffort(model, options?.reasoning);
return streamAzureOpenAIResponses(model, context, {
...base,
reasoningEffort: reasoningEffort === "max" ? "xhigh" : reasoningEffort,
} satisfies AzureOpenAIResponsesOptions);
};
function normalizeAzureBaseUrl(baseUrl: string): string {
const trimmed = baseUrl.trim().replace(/\/+$/, "");
let url: URL;
try {
url = new URL(trimmed);
} catch {
throw new Error(`Invalid Azure OpenAI base URL: ${baseUrl}`);
}
const isAzureHost =
url.hostname.endsWith(".openai.azure.com") ||
url.hostname.endsWith(".cognitiveservices.azure.com");
const normalizedPath = url.pathname.replace(/\/+$/, "");
// Ensure Azure hosts have /openai/v1 as base path so the AzureOpenAI SDK
// can append /deployments/<model>/... and ?api-version=v1 correctly.
if (
isAzureHost &&
(normalizedPath === "" || normalizedPath === "/" || normalizedPath === "/openai")
) {
url.pathname = "/openai/v1";
url.search = "";
}
return url.toString().replace(/\/+$/, "");
}
function buildDefaultBaseUrl(resourceName: string): string {
return `https://${resourceName}.openai.azure.com/openai/v1`;
}
function resolveAzureConfig(
model: Model<"azure-openai-responses">,
options?: AzureOpenAIResponsesOptions,
): { baseUrl: string; apiVersion: string } {
const apiVersion =
options?.azureApiVersion || process.env.AZURE_OPENAI_API_VERSION || DEFAULT_AZURE_API_VERSION;
const baseUrl =
options?.azureBaseUrl?.trim() || process.env.AZURE_OPENAI_BASE_URL?.trim() || undefined;
const resourceName = options?.azureResourceName || process.env.AZURE_OPENAI_RESOURCE_NAME;
let resolvedBaseUrl = baseUrl;
if (!resolvedBaseUrl && resourceName) {
resolvedBaseUrl = buildDefaultBaseUrl(resourceName);
}
if (!resolvedBaseUrl && model.baseUrl) {
resolvedBaseUrl = model.baseUrl;
}
if (!resolvedBaseUrl) {
throw new Error(
"Azure OpenAI base URL is required. Set AZURE_OPENAI_BASE_URL or AZURE_OPENAI_RESOURCE_NAME, or pass azureBaseUrl, azureResourceName, or model.baseUrl.",
);
}
return {
baseUrl: normalizeAzureBaseUrl(resolvedBaseUrl),
apiVersion,
};
}
function createClient(
model: Model<"azure-openai-responses">,
apiKeyInput: string,
options?: AzureOpenAIResponsesOptions,
) {
let apiKey = apiKeyInput;
if (!apiKey) {
if (!process.env.AZURE_OPENAI_API_KEY) {
throw new Error(
"Azure OpenAI API key is required. Set AZURE_OPENAI_API_KEY environment variable or pass it as an argument.",
);
}
apiKey = process.env.AZURE_OPENAI_API_KEY;
}
const headers = { ...model.headers };
if (options?.headers) {
Object.assign(headers, options.headers);
}
const { baseUrl, apiVersion } = resolveAzureConfig(model, options);
const guardedFetch = getAiTransportHost().buildModelFetch({ ...model, baseUrl });
if (isOpenAICompatibleAzureResponsesBaseUrl(baseUrl)) {
return new OpenAI({
apiKey,
dangerouslyAllowBrowser: true,
defaultHeaders: headers,
baseURL: baseUrl,
fetch: guardedFetch,
});
}
return new AzureOpenAI({
apiKey,
apiVersion,
dangerouslyAllowBrowser: true,
defaultHeaders: headers,
baseURL: baseUrl,
fetch: guardedFetch,
});
}
function buildParams(
model: Model<"azure-openai-responses">,
context: Context,
options: AzureOpenAIResponsesOptions | undefined,
deploymentName: string,
) {
const messages = convertResponsesMessages(model, context, AZURE_TOOL_CALL_PROVIDERS);
const params: ResponseCreateParamsStreaming = {
model: deploymentName,
input: messages,
stream: true,
prompt_cache_key:
options?.cacheRetention === "none"
? undefined
: clampOpenAIPromptCacheKey(options?.promptCacheKey ?? options?.sessionId),
};
applyCommonResponsesParams(params, model, context, options);
return params;
}
export const testing = {
isOpenAICompatibleAzureResponsesBaseUrl,
normalizeAzureBaseUrl,
resolveAzureConfig,
};

View File

@@ -0,0 +1,15 @@
import type { CacheRetention } from "../types.js";
/**
* Resolve cache retention preference.
* Defaults to "short" and uses OPENCLAW_CACHE_RETENTION for backward compatibility.
*/
export function resolveCacheRetention(cacheRetention?: CacheRetention): CacheRetention {
if (cacheRetention) {
return cacheRetention;
}
if (typeof process !== "undefined" && process.env.OPENCLAW_CACHE_RETENTION === "long") {
return "long";
}
return "short";
}

View File

@@ -0,0 +1,228 @@
// Gemini schema cleaner tests cover OpenAPI-compatible tool schema cleanup for
// Gemini-backed providers before schemas are sent upstream.
import { describe, expect, it } from "vitest";
import { cleanSchemaForGemini } from "./clean-for-gemini.js";
describe("cleanSchemaForGemini", () => {
it("coerces null properties to an empty object", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: null,
}) as { type?: unknown; properties?: unknown };
expect(cleaned.type).toBe("object");
expect(cleaned.properties).toStrictEqual({});
});
it("coerces non-object properties to an empty object", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: "invalid",
}) as { properties?: unknown };
expect(cleaned.properties).toStrictEqual({});
});
it("coerces array properties to an empty object", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: [],
}) as { properties?: unknown };
expect(cleaned.properties).toStrictEqual({});
});
it("filters required fields that are not in properties", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
action: { type: "string" },
amount: { type: "number" },
},
required: ["action", "amount", "token"],
}) as { required?: string[] };
expect(cleaned.required).toEqual(["action", "amount"]);
});
it("preserves required when all fields exist in properties", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
action: { type: "string" },
amount: { type: "number" },
},
required: ["action", "amount"],
}) as { required?: string[] };
expect(cleaned.required).toEqual(["action", "amount"]);
});
it("removes required entirely when no fields match properties", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
action: { type: "string" },
},
required: ["missing_a", "missing_b"],
}) as { required?: string[] };
expect(cleaned.required).toBeUndefined();
});
it("removes required from object schemas when properties is absent", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
required: ["a", "b"],
}) as { required?: string[] };
expect(cleaned.required).toBeUndefined();
});
it("leaves required as-is for non-object schemas when properties is absent", () => {
const cleaned = cleanSchemaForGemini({
type: "array",
required: ["a", "b"],
}) as { required?: string[] };
expect(cleaned.required).toEqual(["a", "b"]);
});
it("filters required in nested object properties", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
config: {
type: "object",
properties: {
name: { type: "string" },
},
required: ["name", "ghost"],
},
},
}) as { properties?: { config?: { required?: string[] } } };
expect(cleaned.properties?.config?.required).toEqual(["name"]);
});
it("does not treat inherited keys as declared properties", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
name: { type: "string" },
},
required: ["toString", "name"],
}) as { required?: string[] };
expect(cleaned.required).toEqual(["name"]);
});
it("coerces nested null properties while preserving valid siblings", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
bad: {
type: "object",
properties: null,
},
good: {
type: "string",
},
},
}) as {
properties?: {
bad?: { properties?: unknown };
good?: { type?: unknown };
};
};
expect(cleaned.properties?.bad?.properties).toStrictEqual({});
expect(cleaned.properties?.good?.type).toBe("string");
});
it("strips empty required arrays", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
name: { type: "string" },
},
required: [],
}) as Record<string, unknown>;
expect(cleaned).not.toHaveProperty("required");
expect(cleaned.type).toBe("object");
});
it("preserves non-empty required arrays", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
name: { type: "string" },
},
required: ["name"],
}) as Record<string, unknown>;
expect(cleaned.required).toEqual(["name"]);
});
it("strips empty required arrays in nested schemas", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
nested: {
type: "object",
properties: {
optional: { type: "string" },
},
required: [],
},
},
required: ["nested"],
}) as { properties?: { nested?: Record<string, unknown> }; required?: string[] };
expect(cleaned.required).toEqual(["nested"]);
expect(cleaned.properties?.nested).not.toHaveProperty("required");
});
it("strips the not keyword from schemas", () => {
// `not` is outside the OpenAPI 3.0 subset accepted by Gemini-backed
// providers and triggers upstream HTTP 400s if left in tool schemas.
const cleaned = cleanSchemaForGemini({
type: "object",
not: { const: true },
properties: {
name: { type: "string" },
},
}) as Record<string, unknown>;
expect(cleaned).not.toHaveProperty("not");
expect(cleaned.type).toBe("object");
expect(cleaned.properties).toEqual({ name: { type: "string" } });
});
it("collapses type arrays by stripping null entries", () => {
// Type arrays like ["string", "null"] must collapse to a scalar OpenAPI
// type for Gemini compatibility.
const cleaned = cleanSchemaForGemini({
type: ["string", "null"],
description: "nullable field",
}) as Record<string, unknown>;
expect(cleaned.type).toBe("string");
expect(cleaned.description).toBe("nullable field");
});
it("collapses type arrays in nested property schemas", () => {
const cleaned = cleanSchemaForGemini({
type: "object",
properties: {
agentId: {
type: ["string", "null"],
description: "Agent id",
},
},
}) as { properties?: { agentId?: Record<string, unknown> } };
expect(cleaned.properties?.agentId?.type).toBe("string");
});
});

View File

@@ -0,0 +1,458 @@
// Cloud Code Assist API rejects a subset of JSON Schema keywords.
// This module scrubs/normalizes tool schemas to keep Gemini happy.
import type { TSchema } from "typebox";
// Keywords that Cloud Code Assist API rejects (not compliant with their JSON Schema subset)
export const GEMINI_UNSUPPORTED_SCHEMA_KEYWORDS = new Set([
"patternProperties",
"additionalProperties",
"$schema",
"$id",
"$ref",
"$defs",
"definitions",
// Non-standard (OpenAPI) keyword; Claude validators reject it.
"examples",
// Cloud Code Assist appears to validate tool schemas more strictly/quirkily than
// draft 2020-12 in practice; these constraints frequently trigger 400s.
"minLength",
"maxLength",
"minimum",
"maximum",
"multipleOf",
"pattern",
"format",
"minItems",
"maxItems",
"uniqueItems",
"minProperties",
"maxProperties",
// JSON Schema composition keywords not supported by OpenAPI 3.0 subset.
// `const` is handled separately (converted to enum) in the cleaning loop,
// but `not` has no safe equivalent and must be stripped.
"not",
]);
const SCHEMA_META_KEYS = ["description", "title", "default"] as const;
function copySchemaMeta(from: Record<string, unknown>, to: Record<string, unknown>): void {
for (const key of SCHEMA_META_KEYS) {
if (key in from && from[key] !== undefined) {
to[key] = from[key];
}
}
}
// Check if an anyOf/oneOf array contains only literal values that can be flattened.
// TypeBox Type.Literal generates { const: "value", type: "string" }.
// Some schemas may use { enum: ["value"], type: "string" }.
// Both patterns are flattened to { type: "string", enum: ["a", "b", ...] }.
function tryFlattenLiteralAnyOf(variants: unknown[]): { type: string; enum: unknown[] } | null {
if (variants.length === 0) {
return null;
}
const allValues: unknown[] = [];
let commonType: string | null = null;
for (const variant of variants) {
if (!variant || typeof variant !== "object") {
return null;
}
const v = variant as Record<string, unknown>;
let literalValue: unknown;
if ("const" in v) {
literalValue = v.const;
} else if (Array.isArray(v.enum) && v.enum.length === 1) {
literalValue = v.enum[0];
} else {
return null;
}
const variantType = typeof v.type === "string" ? v.type : null;
if (!variantType) {
return null;
}
if (commonType === null) {
commonType = variantType;
} else if (commonType !== variantType) {
return null;
}
allValues.push(literalValue);
}
if (commonType && allValues.length > 0) {
return { type: commonType, enum: allValues };
}
return null;
}
function isNullSchema(variant: unknown): boolean {
if (!variant || typeof variant !== "object" || Array.isArray(variant)) {
return false;
}
const record = variant as Record<string, unknown>;
if ("const" in record && record.const === null) {
return true;
}
if (Array.isArray(record.enum) && record.enum.length === 1) {
return record.enum[0] === null;
}
const typeValue = record.type;
if (typeValue === "null") {
return true;
}
if (Array.isArray(typeValue) && typeValue.length === 1 && typeValue[0] === "null") {
return true;
}
return false;
}
function stripNullVariants(variants: unknown[]): {
variants: unknown[];
stripped: boolean;
} {
if (variants.length === 0) {
return { variants, stripped: false };
}
const nonNull = variants.filter((variant) => !isNullSchema(variant));
return {
variants: nonNull,
stripped: nonNull.length !== variants.length,
};
}
type SchemaDefs = Map<string, unknown>;
function extendSchemaDefs(
defs: SchemaDefs | undefined,
schema: Record<string, unknown>,
): SchemaDefs | undefined {
const defsEntry =
schema.$defs && typeof schema.$defs === "object" && !Array.isArray(schema.$defs)
? (schema.$defs as Record<string, unknown>)
: undefined;
const legacyDefsEntry =
schema.definitions &&
typeof schema.definitions === "object" &&
!Array.isArray(schema.definitions)
? (schema.definitions as Record<string, unknown>)
: undefined;
if (!defsEntry && !legacyDefsEntry) {
return defs;
}
const next = defs ? new Map(defs) : new Map<string, unknown>();
if (defsEntry) {
for (const [key, value] of Object.entries(defsEntry)) {
next.set(key, value);
}
}
if (legacyDefsEntry) {
for (const [key, value] of Object.entries(legacyDefsEntry)) {
next.set(key, value);
}
}
return next;
}
function decodeJsonPointerSegment(segment: string): string {
return segment.replaceAll("~1", "/").replaceAll("~0", "~");
}
function tryResolveLocalRef(ref: string, defs: SchemaDefs | undefined): unknown {
if (!defs) {
return undefined;
}
const match = ref.match(/^#\/(?:\$defs|definitions)\/(.+)$/);
if (!match) {
return undefined;
}
const name = decodeJsonPointerSegment(match[1] ?? "");
if (!name) {
return undefined;
}
return defs.get(name);
}
function simplifyUnionVariants(params: { obj: Record<string, unknown>; variants: unknown[] }): {
variants: unknown[];
simplified?: unknown;
} {
const { obj, variants } = params;
const { variants: nonNullVariants, stripped } = stripNullVariants(variants);
const flattened = tryFlattenLiteralAnyOf(nonNullVariants);
if (flattened) {
const result: Record<string, unknown> = {
type: flattened.type,
enum: flattened.enum,
};
copySchemaMeta(obj, result);
return { variants: nonNullVariants, simplified: result };
}
if (stripped && nonNullVariants.length === 1) {
const lone = nonNullVariants[0];
if (lone && typeof lone === "object" && !Array.isArray(lone)) {
const result: Record<string, unknown> = {
...(lone as Record<string, unknown>),
};
copySchemaMeta(obj, result);
return { variants: nonNullVariants, simplified: result };
}
return { variants: nonNullVariants, simplified: lone };
}
return { variants: stripped ? nonNullVariants : variants };
}
// Gemini rejects object schemas whose `required` entries do not exist in `properties`.
function sanitizeRequiredFields(schema: Record<string, unknown>): Record<string, unknown> {
if (!Array.isArray(schema.required)) {
return schema;
}
if (
!schema.properties ||
typeof schema.properties !== "object" ||
Array.isArray(schema.properties)
) {
if (schema.type === "object") {
delete schema.required;
}
return schema;
}
const properties = schema.properties as Record<string, unknown>;
const required = schema.required.filter(
(key): key is string => typeof key === "string" && Object.hasOwn(properties, key),
);
if (required.length > 0) {
schema.required = required;
} else {
delete schema.required;
}
return schema;
}
function cleanSchemaForGeminiWithDefs(
schema: unknown,
defs: SchemaDefs | undefined,
refStack: Set<string> | undefined,
): unknown {
if (!schema || typeof schema !== "object") {
return schema;
}
if (Array.isArray(schema)) {
return schema.map((item) => cleanSchemaForGeminiWithDefs(item, defs, refStack));
}
const obj = schema as Record<string, unknown>;
const nextDefs = extendSchemaDefs(defs, obj);
const refValue = typeof obj.$ref === "string" ? obj.$ref : undefined;
if (refValue) {
if (refStack?.has(refValue)) {
return {};
}
const resolved = tryResolveLocalRef(refValue, nextDefs);
if (resolved) {
const nextRefStack = refStack ? new Set(refStack) : new Set<string>();
nextRefStack.add(refValue);
const cleaned = cleanSchemaForGeminiWithDefs(resolved, nextDefs, nextRefStack);
if (!cleaned || typeof cleaned !== "object" || Array.isArray(cleaned)) {
return cleaned;
}
const result: Record<string, unknown> = {
...(cleaned as Record<string, unknown>),
};
copySchemaMeta(obj, result);
return result;
}
const result: Record<string, unknown> = {};
copySchemaMeta(obj, result);
return result;
}
const hasAnyOf = "anyOf" in obj && Array.isArray(obj.anyOf);
const hasOneOf = "oneOf" in obj && Array.isArray(obj.oneOf);
let cleanedAnyOf = hasAnyOf
? (obj.anyOf as unknown[]).map((variant) =>
cleanSchemaForGeminiWithDefs(variant, nextDefs, refStack),
)
: undefined;
let cleanedOneOf = hasOneOf
? (obj.oneOf as unknown[]).map((variant) =>
cleanSchemaForGeminiWithDefs(variant, nextDefs, refStack),
)
: undefined;
if (hasAnyOf) {
const simplified = simplifyUnionVariants({ obj, variants: cleanedAnyOf ?? [] });
cleanedAnyOf = simplified.variants;
if ("simplified" in simplified) {
return simplified.simplified;
}
}
if (hasOneOf) {
const simplified = simplifyUnionVariants({ obj, variants: cleanedOneOf ?? [] });
cleanedOneOf = simplified.variants;
if ("simplified" in simplified) {
return simplified.simplified;
}
}
const cleaned: Record<string, unknown> = {};
for (const [key, value] of Object.entries(obj)) {
if (GEMINI_UNSUPPORTED_SCHEMA_KEYWORDS.has(key)) {
continue;
}
if (key === "const") {
cleaned.enum = [value];
continue;
}
// Google's schema validator rejects `"required": []` — omit empty arrays.
if (key === "required" && Array.isArray(value) && value.length === 0) {
continue;
}
if (key === "type" && (hasAnyOf || hasOneOf)) {
continue;
}
if (
key === "type" &&
Array.isArray(value) &&
value.every((entry) => typeof entry === "string")
) {
const types = value.filter((entry) => entry !== "null");
cleaned.type = types.length === 1 ? types[0] : types;
continue;
}
if (key === "properties") {
if (value && typeof value === "object" && !Array.isArray(value)) {
const props = value as Record<string, unknown>;
cleaned[key] = Object.fromEntries(
Object.entries(props).map(([k, v]) => [
k,
cleanSchemaForGeminiWithDefs(v, nextDefs, refStack),
]),
);
} else {
// Guard malformed schemas (e.g. properties: null) that can trigger
// downstream Object.* crashes in strict provider validators.
cleaned[key] = {};
}
} else if (key === "items" && value) {
if (Array.isArray(value)) {
cleaned[key] = value.map((entry) =>
cleanSchemaForGeminiWithDefs(entry, nextDefs, refStack),
);
} else if (typeof value === "object") {
cleaned[key] = cleanSchemaForGeminiWithDefs(value, nextDefs, refStack);
} else {
cleaned[key] = value;
}
} else if (key === "anyOf" && Array.isArray(value)) {
cleaned[key] =
cleanedAnyOf ??
value.map((variant) => cleanSchemaForGeminiWithDefs(variant, nextDefs, refStack));
} else if (key === "oneOf" && Array.isArray(value)) {
cleaned[key] =
cleanedOneOf ??
value.map((variant) => cleanSchemaForGeminiWithDefs(variant, nextDefs, refStack));
} else if (key === "allOf" && Array.isArray(value)) {
cleaned[key] = value.map((variant) =>
cleanSchemaForGeminiWithDefs(variant, nextDefs, refStack),
);
} else {
cleaned[key] = value;
}
}
// Cloud Code Assist API rejects anyOf/oneOf in nested schemas even after
// simplifyUnionVariants runs above. Flatten remaining unions as a fallback:
// pick the common type or use the first variant's type so the tool
// declaration is accepted by Google's validation layer.
if (cleaned.anyOf && Array.isArray(cleaned.anyOf)) {
const flattened = flattenUnionFallback(cleaned, cleaned.anyOf);
if (flattened) {
return sanitizeRequiredFields(flattened);
}
}
if (cleaned.oneOf && Array.isArray(cleaned.oneOf)) {
const flattened = flattenUnionFallback(cleaned, cleaned.oneOf);
if (flattened) {
return sanitizeRequiredFields(flattened);
}
}
return sanitizeRequiredFields(cleaned);
}
/**
* Last-resort flattening for anyOf/oneOf arrays that could not be simplified
* by `simplifyUnionVariants`. Picks a representative type so the schema is
* accepted by Google's restricted JSON Schema validation.
*/
function flattenUnionFallback(
obj: Record<string, unknown>,
variants: unknown[],
): Record<string, unknown> | undefined {
const objects = variants.filter(
(v): v is Record<string, unknown> => Boolean(v) && typeof v === "object",
);
if (objects.length === 0) {
return undefined;
}
const types = new Set(objects.map((v) => v.type).filter(Boolean));
if (objects.length === 1) {
const merged: Record<string, unknown> = { ...objects[0] };
copySchemaMeta(obj, merged);
return merged;
}
if (types.size === 1) {
const merged: Record<string, unknown> = { type: Array.from(types)[0] };
copySchemaMeta(obj, merged);
return merged;
}
const first = objects[0];
if (first?.type) {
const merged: Record<string, unknown> = { type: first.type };
copySchemaMeta(obj, merged);
return merged;
}
const merged: Record<string, unknown> = {};
copySchemaMeta(obj, merged);
return merged;
}
export function cleanSchemaForGemini(schema: unknown): TSchema {
if (!schema || typeof schema !== "object") {
return schema as TSchema;
}
if (Array.isArray(schema)) {
return schema.map(cleanSchemaForGemini) as TSchema;
}
const defs = extendSchemaDefs(undefined, schema as Record<string, unknown>);
return cleanSchemaForGeminiWithDefs(schema, defs, undefined) as TSchema;
}

View File

@@ -0,0 +1,22 @@
// Cloudflare provider metadata describes Cloudflare-hosted model capabilities.
import type { Model } from "../types.js";
export function isCloudflareProvider(provider: string): boolean {
return provider === "cloudflare-workers-ai" || provider === "cloudflare-ai-gateway";
}
/** Substitute `{VAR}` placeholders in a Cloudflare baseUrl from process.env. */
export function resolveCloudflareBaseUrl(model: Model): string {
const url = model.baseUrl;
if (!url.includes("{")) {
return url;
}
const baseUrl = url.replace(/\{([A-Z_][A-Z0-9_]*)\}/g, (_match, name: string) => {
const value = process.env[name];
if (!value) {
throw new Error(`${name} is required for provider ${model.provider} but is not set.`);
}
return value;
});
return baseUrl;
}

View File

@@ -0,0 +1,38 @@
// GitHub Copilot header helpers build request headers for Copilot-backed providers.
import type { Message } from "../types.js";
// Copilot expects X-Initiator to indicate whether the request is user-initiated
// or agent-initiated (e.g. follow-up after assistant/tool messages).
export function inferCopilotInitiator(messages: Message[]): "user" | "agent" {
const last = messages[messages.length - 1];
return last && last.role !== "user" ? "agent" : "user";
}
// Copilot requires Copilot-Vision-Request header when sending images
export function hasCopilotVisionInput(messages: Message[]): boolean {
return messages.some((msg) => {
if (msg.role === "user" && Array.isArray(msg.content)) {
return msg.content.some((c) => c.type === "image");
}
if (msg.role === "toolResult" && Array.isArray(msg.content)) {
return msg.content.some((c) => c.type === "image");
}
return false;
});
}
export function buildCopilotDynamicHeaders(params: {
messages: Message[];
hasImages: boolean;
}): Record<string, string> {
const headers: Record<string, string> = {
"X-Initiator": inferCopilotInitiator(params.messages),
"Openai-Intent": "conversation-edits",
};
if (params.hasImages) {
headers["Copilot-Vision-Request"] = "true";
}
return headers;
}

View File

@@ -0,0 +1,403 @@
// Google shared conversion tests cover runtime-to-Google payload conversion.
import { describe, expect, it } from "vitest";
import type { Context, Tool } from "../types.js";
import { convertMessages, convertTools } from "./google-shared.js";
import {
asRecord,
expectConvertedRoles,
getFirstToolParameters,
makeGeminiCliAssistantMessage,
makeGeminiCliModel,
makeGoogleAssistantMessage,
makeModel,
} from "./google-shared.test-helpers.js";
type GoogleSharedTestModel = ReturnType<typeof makeModel> | ReturnType<typeof makeGeminiCliModel>;
const convertMessagesForTest = convertMessages as unknown as (
model: GoogleSharedTestModel,
context: Context,
) => ReturnType<typeof convertMessages>;
function requireRecordProperty(
record: Record<string, unknown>,
key: string,
): Record<string, unknown> {
const value = record[key];
if (!value || typeof value !== "object" || Array.isArray(value)) {
throw new Error(`expected object property ${key}`);
}
return value as Record<string, unknown>;
}
describe("google-shared convertTools", () => {
it("preserves parameters when type is missing", () => {
const tools = [
{
name: "noType",
description: "Tool with properties but no type",
parameters: {
properties: {
action: { type: "string" },
},
required: ["action"],
},
},
] as unknown as Tool[];
const converted = convertTools(tools);
const params = getFirstToolParameters(
converted as Parameters<typeof getFirstToolParameters>[0],
);
expect(params.type).toBeUndefined();
expect(params.properties).toEqual({
action: { type: "string" },
});
expect(params.required).toEqual(["action"]);
});
it("keeps unsupported JSON Schema keywords intact", () => {
const tools = [
{
name: "example",
description: "Example tool",
parameters: {
type: "object",
patternProperties: {
"^x-": { type: "string" },
},
additionalProperties: false,
properties: {
mode: {
type: "string",
const: "fast",
},
options: {
anyOf: [{ type: "string" }, { type: "number" }],
},
list: {
type: "array",
items: {
type: "string",
const: "item",
},
},
},
required: ["mode"],
},
},
] as unknown as Tool[];
const converted = convertTools(tools);
const params = getFirstToolParameters(
converted as Parameters<typeof getFirstToolParameters>[0],
);
const properties = asRecord(params.properties);
const mode = asRecord(properties.mode);
const options = asRecord(properties.options);
const list = asRecord(properties.list);
const items = asRecord(list.items);
expect(params.patternProperties).toEqual({ "^x-": { type: "string" } });
expect(params.additionalProperties).toBe(false);
expect(mode.const).toBe("fast");
expect(options.anyOf).toEqual([{ type: "string" }, { type: "number" }]);
expect(items.const).toBe("item");
expect(params.required).toEqual(["mode"]);
});
it("keeps supported schema fields", () => {
const tools = [
{
name: "settings",
description: "Settings tool",
parameters: {
type: "object",
properties: {
config: {
type: "object",
properties: {
retries: { type: "number", minimum: 1 },
tags: {
type: "array",
items: { type: "string" },
},
},
required: ["retries"],
},
},
required: ["config"],
},
},
] as unknown as Tool[];
const converted = convertTools(tools);
const params = getFirstToolParameters(
converted as Parameters<typeof getFirstToolParameters>[0],
);
const config = asRecord(asRecord(params.properties).config);
const configProps = asRecord(config.properties);
const retries = asRecord(configProps.retries);
const tags = asRecord(configProps.tags);
const items = asRecord(tags.items);
expect(params.type).toBe("object");
expect(config.type).toBe("object");
expect(retries.minimum).toBe(1);
expect(tags.type).toBe("array");
expect(items.type).toBe("string");
expect(config.required).toEqual(["retries"]);
expect(params.required).toEqual(["config"]);
});
});
describe("google-shared convertMessages", () => {
function expectConsecutiveMessagesNotMerged(params: {
modelId: string;
first: string;
second: string;
}) {
const model = makeModel(params.modelId);
const context = {
messages: [
{
role: "user",
content: params.first,
},
{
role: "user",
content: params.second,
},
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
expect(contents).toHaveLength(2);
expect(contents[0].role).toBe("user");
expect(contents[1].role).toBe("user");
expect(contents[0].parts).toHaveLength(1);
expect(contents[1].parts).toHaveLength(1);
}
it("keeps thinking blocks when provider/model match", () => {
const model = makeModel("gemini-1.5-pro");
const context = {
messages: [
makeGoogleAssistantMessage(model.id, [
{
type: "thinking",
thinking: "hidden",
thinkingSignature: "c2ln",
},
]),
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
expect(contents).toHaveLength(1);
expect(contents[0].role).toBe("model");
const part = asRecord(contents[0].parts?.[0]);
expect(part.thought).toBe(true);
expect(part.thoughtSignature).toBe("c2ln");
});
it("keeps thought signatures for Claude models", () => {
const model = makeModel("claude-3-opus");
const context = {
messages: [
makeGoogleAssistantMessage(model.id, [
{
type: "thinking",
thinking: "structured",
thinkingSignature: "c2ln",
},
]),
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
const parts = contents?.[0]?.parts ?? [];
expect(parts).toHaveLength(1);
const part = asRecord(parts[0]);
expect(part.thought).toBe(true);
expect(part.thoughtSignature).toBe("c2ln");
});
it("does not merge consecutive user messages for Gemini", () => {
expectConsecutiveMessagesNotMerged({
modelId: "gemini-1.5-pro",
first: "Hello",
second: "How are you?",
});
});
it("does not merge consecutive user messages for non-Gemini Google models", () => {
expectConsecutiveMessagesNotMerged({
modelId: "claude-3-opus",
first: "First",
second: "Second",
});
});
it("does not merge consecutive model messages for Gemini", () => {
const model = makeModel("gemini-1.5-pro");
const context = {
messages: [
{
role: "user",
content: "Hello",
},
makeGoogleAssistantMessage(model.id, [{ type: "text", text: "Hi there!" }]),
makeGoogleAssistantMessage(model.id, [{ type: "text", text: "How can I help?" }]),
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
expectConvertedRoles(contents, ["user", "model", "model"]);
expect(contents[1].parts).toHaveLength(1);
expect(contents[2].parts).toHaveLength(1);
});
it("handles user message after tool result without model response in between", () => {
const model = makeModel("gemini-1.5-pro");
const context = {
messages: [
{
role: "user",
content: "Use a tool",
},
makeGoogleAssistantMessage(model.id, [
{
type: "toolCall",
id: "call_1",
name: "myTool",
arguments: { arg: "value" },
},
]),
{
role: "toolResult",
toolCallId: "call_1",
toolName: "myTool",
content: [{ type: "text", text: "Tool result" }],
isError: false,
timestamp: 0,
},
{
role: "user",
content: "Now do something else",
},
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
expect(contents).toHaveLength(4);
expect(contents[0].role).toBe("user");
expect(contents[1].role).toBe("model");
expect(contents[2].role).toBe("user");
expect(contents[3].role).toBe("user");
const toolResponsePart = contents[2].parts?.find(
(part) => typeof part === "object" && part !== null && "functionResponse" in part,
);
const toolResponse = asRecord(toolResponsePart);
expect(requireRecordProperty(toolResponse, "functionResponse").name).toBe("myTool");
expect(contents[3].role).toBe("user");
});
it("ensures function call comes after user turn, not after model turn", () => {
const model = makeModel("gemini-1.5-pro");
const context = {
messages: [
{
role: "user",
content: "Hello",
},
makeGoogleAssistantMessage(model.id, [{ type: "text", text: "Hi!" }]),
makeGoogleAssistantMessage(model.id, [
{
type: "toolCall",
id: "call_1",
name: "myTool",
arguments: {},
},
]),
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
expectConvertedRoles(contents, ["user", "model", "model", "user"]);
const toolCallPart = contents[2].parts?.find(
(part) => typeof part === "object" && part !== null && "functionCall" in part,
);
const toolCall = asRecord(toolCallPart);
expect(requireRecordProperty(toolCall, "functionCall").name).toBe("myTool");
});
it("strips tool call and response ids for google-gemini-cli", () => {
const model = makeGeminiCliModel("gemini-3-flash");
const context = {
messages: [
{
role: "user",
content: "Use a tool",
},
makeGeminiCliAssistantMessage(model.id, [
{
type: "toolCall",
id: "call_1",
name: "myTool",
arguments: { arg: "value" },
thoughtSignature: "dGVzdA==",
},
]),
{
role: "toolResult",
toolCallId: "call_1",
toolName: "myTool",
content: [{ type: "text", text: "Tool result" }],
isError: false,
timestamp: 0,
},
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
const parts = contents.flatMap((content) => content.parts ?? []);
const toolCallPart = parts.find(
(part) => typeof part === "object" && part !== null && "functionCall" in part,
);
const toolResponsePart = parts.find(
(part) => typeof part === "object" && part !== null && "functionResponse" in part,
);
const toolCall = asRecord(toolCallPart);
const toolResponse = asRecord(toolResponsePart);
expect(asRecord(toolCall.functionCall).id).toBeUndefined();
expect(asRecord(toolResponse.functionResponse).id).toBeUndefined();
});
it("serializes structured tool results into function responses", () => {
const model = makeModel("gemini-1.5-pro");
const context = {
messages: [
{
role: "toolResult",
toolCallId: "call_1",
toolName: "session_status",
content: [{ type: "json", payload: { sessionKey: "current", status: "ok" } }],
isError: false,
timestamp: 0,
},
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
const toolResponsePart = contents[0]?.parts?.find(
(part) => typeof part === "object" && part !== null && "functionResponse" in part,
);
expect(toolResponsePart).toBeDefined();
const toolResponse = requireRecordProperty(asRecord(toolResponsePart), "functionResponse");
expect(asRecord(toolResponse.response).output).toBe(
'{"type":"json","payload":{"sessionKey":"current","status":"ok"}}',
);
});
});

View File

@@ -0,0 +1,87 @@
import type { Part } from "@google/genai";
import { describe, expect, it } from "vitest";
import type { Context, Model } from "../types.js";
import { convertMessages } from "./google-shared.js";
import { makeGoogleAssistantMessage } from "./google-shared.test-helpers.js";
const convertMessagesForTest = convertMessages as unknown as (
model: Model<"google-generative-ai">,
context: Context,
) => ReturnType<typeof convertMessages>;
const makeVisionModel = (id: string): Model<"google-generative-ai"> =>
({
id,
name: id,
api: "google-generative-ai",
provider: "google",
baseUrl: "https://example.invalid",
reasoning: false,
input: ["text", "image"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1,
maxTokens: 1,
}) as Model<"google-generative-ai">;
function countFunctionResponses(parts: readonly Part[] | undefined): number {
return (parts ?? []).filter((p) => p.functionResponse != null).length;
}
function functionResponseNames(parts: readonly Part[] | undefined): string[] {
return (parts ?? []).flatMap((part) =>
part.functionResponse?.name ? [part.functionResponse.name] : [],
);
}
describe("google-shared convertMessages — parallel tool results with an image (Gemini < 3)", () => {
it.each([
["image first", ["screenshot", "weather"]],
["image last", ["weather", "screenshot"]],
] as const)(
"keeps the response run immediate and retains a deferred %s result",
(_label, resultOrder) => {
const model = makeVisionModel("gemini-2.5-flash");
const toolResults = {
screenshot: {
role: "toolResult",
toolCallId: "call_1",
toolName: "screenshot",
content: [{ type: "image", mimeType: "image/png", data: "AAAA" }],
isError: false,
timestamp: 0,
},
weather: {
role: "toolResult",
toolCallId: "call_2",
toolName: "weather",
content: [{ type: "text", text: "Sunny, 21C" }],
isError: false,
timestamp: 0,
},
} as const;
const context = {
messages: [
{ role: "user", content: "Screenshot the page and check the weather." },
makeGoogleAssistantMessage(model.id, [
{ type: "toolCall", id: "call_1", name: "screenshot", arguments: {} },
{ type: "toolCall", id: "call_2", name: "weather", arguments: {} },
]),
...resultOrder.map((name) => toolResults[name]),
],
} as unknown as Context;
const contents = convertMessagesForTest(model, context);
expect(contents.map((content) => content.role)).toEqual(["user", "model", "user", "user"]);
expect(functionResponseNames(contents[2].parts)).toEqual(resultOrder);
expect(contents[3]).toEqual({
role: "user",
parts: [
{ text: "Tool result image:" },
{ inlineData: { mimeType: "image/png", data: "AAAA" } },
],
});
expect(contents.slice(3).some((c) => countFunctionResponses(c.parts) > 0)).toBe(false);
},
);
});

View File

@@ -0,0 +1,100 @@
// Google provider test helpers assert converted message and stream payloads.
import { expect } from "vitest";
import type { Model } from "../types.js";
function makeZeroUsageSnapshot() {
return {
inputTokens: 0,
outputTokens: 0,
cacheReadInputTokens: 0,
cacheCreationInputTokens: 0,
reasoningTokens: 0,
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
total: 0,
},
};
}
export const asRecord = (value: unknown): Record<string, unknown> => {
if (!value || typeof value !== "object" || Array.isArray(value)) {
throw new Error("expected record");
}
return value as Record<string, unknown>;
};
type ConvertedTools = ReadonlyArray<{
functionDeclarations?: ReadonlyArray<{
parametersJsonSchema?: unknown;
parameters?: unknown;
}>;
}>;
export const getFirstToolParameters = (converted: ConvertedTools): Record<string, unknown> => {
const functionDeclaration = asRecord(converted?.[0]?.functionDeclarations?.[0]);
return asRecord(functionDeclaration.parametersJsonSchema ?? functionDeclaration.parameters);
};
export const makeModel = (id: string): Model<"google-generative-ai"> =>
({
id,
name: id,
api: "google-generative-ai",
provider: "google",
baseUrl: "https://example.invalid",
reasoning: false,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1,
maxTokens: 1,
}) as Model<"google-generative-ai">;
export const makeGeminiCliModel = (id: string): Model<"google-gemini-cli"> =>
({
id,
name: id,
api: "google-gemini-cli",
provider: "google-gemini-cli",
baseUrl: "https://example.invalid",
reasoning: false,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 1,
maxTokens: 1,
}) as Model<"google-gemini-cli">;
export function makeGoogleAssistantMessage(model: string, content: unknown) {
return {
role: "assistant",
content,
api: "google-generative-ai",
provider: "google",
model,
usage: makeZeroUsageSnapshot(),
stopReason: "stop",
timestamp: 0,
};
}
export function makeGeminiCliAssistantMessage(model: string, content: unknown) {
return {
role: "assistant",
content,
api: "google-gemini-cli",
provider: "google-gemini-cli",
model,
usage: makeZeroUsageSnapshot(),
stopReason: "stop",
timestamp: 0,
};
}
export function expectConvertedRoles(contents: Array<{ role?: string }>, expectedRoles: string[]) {
expect(contents).toHaveLength(expectedRoles.length);
for (const [index, role] of expectedRoles.entries()) {
expect(contents[index]?.role).toBe(role);
}
}

View File

@@ -0,0 +1,252 @@
// Google shared provider tests cover response conversion and finish reasons.
import { FinishReason, type GenerateContentResponse } from "@google/genai";
import { describe, expect, it } from "vitest";
import type { AssistantMessage, Model } from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "../utils/system-prompt-cache-boundary.js";
import {
buildGoogleGenerateContentParams,
consumeGoogleGenerateContentStream,
} from "./google-shared.js";
const model: Model<"google-generative-ai"> = {
id: "gemini-test",
name: "Gemini Test",
api: "google-generative-ai",
provider: "google",
baseUrl: "",
reasoning: true,
input: ["text"],
cost: {
input: 1,
output: 2,
cacheRead: 0.25,
cacheWrite: 0,
},
contextWindow: 128_000,
maxTokens: 8_192,
};
function createOutput(): AssistantMessage {
return {
role: "assistant",
content: [],
api: model.api,
provider: model.provider,
model: model.id,
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
total: 0,
},
},
stopReason: "stop",
timestamp: 0,
};
}
async function* chunks(items: GenerateContentResponse[]) {
yield* items;
}
describe("consumeGoogleGenerateContentStream", () => {
it("projects text, thinking, tool calls, response id, and usage into one stream", async () => {
const output = createOutput();
const stream = new AssistantMessageEventStream();
const events: string[] = [];
const collect = (async () => {
for await (const event of stream) {
events.push(event.type);
}
})();
await consumeGoogleGenerateContentStream({
chunks: chunks([
{
responseId: "response-1",
candidates: [
{
content: {
parts: [
{ text: "thinking", thought: true, thoughtSignature: "dGhpbms=" },
{ text: "hello" },
{ functionCall: { name: "lookup", args: { query: "cats" } } },
],
},
},
],
} as GenerateContentResponse,
{
candidates: [{ finishReason: FinishReason.STOP }],
usageMetadata: {
promptTokenCount: 10,
cachedContentTokenCount: 2,
candidatesTokenCount: 3,
thoughtsTokenCount: 4,
totalTokenCount: 17,
},
} as GenerateContentResponse,
]),
model,
output,
stream,
nextToolCallId: (name) => `generated-${name}`,
});
await collect;
expect(events).toEqual([
"start",
"thinking_start",
"thinking_delta",
"thinking_end",
"text_start",
"text_delta",
"text_end",
"toolcall_start",
"toolcall_delta",
"toolcall_end",
"done",
]);
expect(output.responseId).toBe("response-1");
expect(output.stopReason).toBe("toolUse");
expect(output.content).toEqual([
{ type: "thinking", thinking: "thinking", thinkingSignature: "dGhpbms=" },
{ type: "text", text: "hello" },
{
type: "toolCall",
id: "generated-lookup",
name: "lookup",
arguments: { query: "cats" },
},
]);
expect(output.usage).toMatchObject({
input: 8,
output: 7,
cacheRead: 2,
totalTokens: 17,
});
expect(output.usage.cost.total).toBeGreaterThan(0);
});
it("preserves MAX_TOKENS when the partial response contains a function call", async () => {
const output = createOutput();
const stream = new AssistantMessageEventStream();
const terminalReason = (async () => {
for await (const event of stream) {
if (event.type === "done") {
return event.reason;
}
}
return undefined;
})();
await consumeGoogleGenerateContentStream({
chunks: chunks([
{
candidates: [
{
content: {
parts: [{ functionCall: { name: "lookup", args: { query: "cats" } } }],
},
finishReason: FinishReason.MAX_TOKENS,
},
],
} as unknown as GenerateContentResponse,
]),
model,
output,
stream,
nextToolCallId: (name) => `generated-${name}`,
});
expect(await terminalReason).toBe("length");
expect(output.stopReason).toBe("length");
expect(output.content).toEqual([expect.objectContaining({ type: "toolCall", name: "lookup" })]);
});
it("generates a new id when Google repeats a streamed tool-call id", async () => {
const output = createOutput();
const stream = new AssistantMessageEventStream();
const events: string[] = [];
const collect = (async () => {
for await (const event of stream) {
events.push(event.type);
}
})();
await consumeGoogleGenerateContentStream({
chunks: chunks([
{
candidates: [
{
content: {
parts: [{ functionCall: { id: "call_1", name: "lookup", args: {} } }],
},
},
],
} as GenerateContentResponse,
{
candidates: [
{
content: {
parts: [{ functionCall: { id: "call_1", name: "lookup", args: {} } }],
},
finishReason: FinishReason.STOP,
},
],
} as GenerateContentResponse,
]),
model,
output,
stream,
nextToolCallId: (name) => `generated-${name}`,
});
await collect;
expect(events.at(-1)).toBe("done");
expect(output.content).toEqual([
{
type: "toolCall",
id: "call_1",
name: "lookup",
arguments: {},
},
{
type: "toolCall",
id: "generated-lookup",
name: "lookup",
arguments: {},
},
]);
});
});
describe("buildGoogleGenerateContentParams", () => {
it("forwards stop sequences to Google generation config", () => {
const params = buildGoogleGenerateContentParams(
model,
{ messages: [{ role: "user", content: "hello", timestamp: 0 }] },
{ stop: ["STOP"] },
);
expect(params.config?.stopSequences).toEqual(["STOP"]);
});
it("strips the internal cache boundary marker from systemInstruction", () => {
const params = buildGoogleGenerateContentParams(model, {
systemPrompt: `Stable${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic`,
messages: [{ role: "user", content: "hello", timestamp: 0 }],
});
expect(params.config?.systemInstruction).toBe("Stable\nDynamic");
expect(JSON.stringify(params)).not.toContain("OPENCLAW_CACHE_BOUNDARY");
});
});

View File

@@ -0,0 +1,924 @@
/**
* Shared utilities for Google Generative AI and Google Vertex providers.
*/
import {
type Content,
FinishReason,
FunctionCallingConfigMode,
type GenerateContentConfig,
type GenerateContentParameters,
type GenerateContentResponse,
type Part,
type ThinkingConfig,
} from "@google/genai";
import { calculateCost, clampThinkingLevel } from "../model-utils.js";
import type {
Api,
AssistantMessage,
Context,
ImageContent,
Model,
SimpleStreamOptions,
StopReason,
TextContent,
ThinkingBudgets,
ThinkingContent,
ThinkingLevel as AgentThinkingLevel,
Tool,
ToolCall,
StreamOptions,
} from "../types.js";
import type { AssistantMessageEventStream } from "../utils/event-stream.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import { stripSystemPromptCacheBoundary } from "../utils/system-prompt-cache-boundary.js";
import { describeToolResultMediaPlaceholder, extractToolResultText } from "./tool-result-text.js";
import { transformMessages } from "./transform-messages.js";
export type GoogleApiType = "google-generative-ai" | "google-vertex";
/**
* Thinking level for Gemini 3 models.
* Mirrors Google's ThinkingLevel enum values.
*/
export type GoogleThinkingLevel =
| "THINKING_LEVEL_UNSPECIFIED"
| "MINIMAL"
| "LOW"
| "MEDIUM"
| "HIGH";
export type GoogleToolChoice = "auto" | "none" | "any";
export type GoogleThinkingOptions = {
enabled: boolean;
budgetTokens?: number;
level?: GoogleThinkingLevel;
};
export type GoogleProviderOptions = StreamOptions & {
toolChoice?: GoogleToolChoice;
thinking?: GoogleThinkingOptions;
};
type GoogleGenerateContentClient = {
models: {
generateContentStream(
params: GenerateContentParameters,
): Promise<AsyncIterable<GenerateContentResponse>> | AsyncIterable<GenerateContentResponse>;
};
};
type ClampedGoogleThinkingLevel = Exclude<AgentThinkingLevel, "xhigh" | "max">;
/**
* Determines whether a streamed Gemini `Part` should be treated as "thinking".
*
* Protocol note (Gemini / Vertex AI thought signatures):
* - `thought: true` is the definitive marker for thinking content (thought summaries).
* - `thoughtSignature` is an encrypted representation of the model's internal thought process
* used to preserve reasoning context across multi-turn interactions.
* - `thoughtSignature` can appear on ANY part type (text, functionCall, etc.) - it does NOT
* indicate the part itself is thinking content.
* - For non-functionCall responses, the signature appears on the last part for context replay.
* - When persisting/replaying model outputs, signature-bearing parts must be preserved as-is;
* do not merge/move signatures across parts.
*
* See: https://ai.google.dev/gemini-api/docs/thought-signatures
*/
export function isThinkingPart(part: Pick<Part, "thought" | "thoughtSignature">): boolean {
return part.thought === true;
}
/**
* Retain thought signatures during streaming.
*
* Some backends only send `thoughtSignature` on the first delta for a given part/block; later deltas may omit it.
* This helper preserves the last non-empty signature for the current block.
*
* Note: this does NOT merge or move signatures across distinct response parts. It only prevents
* a signature from being overwritten with `undefined` within the same streamed block.
*/
export function retainThoughtSignature(
existing: string | undefined,
incoming: string | undefined,
): string | undefined {
if (typeof incoming === "string" && incoming.length > 0) {
return incoming;
}
return existing;
}
// Thought signatures must be base64 for Google APIs (TYPE_BYTES).
const base64SignaturePattern = /^[A-Za-z0-9+/]+={0,2}$/;
function isValidThoughtSignature(signature: string | undefined): boolean {
if (!signature) {
return false;
}
if (signature.length % 4 !== 0) {
return false;
}
return base64SignaturePattern.test(signature);
}
/**
* Only keep signatures from the same provider/model and with valid base64.
*/
function resolveThoughtSignature(
isSameProviderAndModel: boolean,
signature: string | undefined,
): string | undefined {
return isSameProviderAndModel && isValidThoughtSignature(signature) ? signature : undefined;
}
/**
* Models via Google APIs that require explicit tool call IDs in function calls/responses.
*/
export function requiresToolCallId(modelId: string): boolean {
return modelId.startsWith("claude-") || modelId.startsWith("gpt-oss-");
}
function getGeminiMajorVersion(modelId: string): number | undefined {
const match = modelId.toLowerCase().match(/^gemini(?:-live)?-(\d+)/);
if (!match) {
return undefined;
}
return Number.parseInt(match[1], 10);
}
function supportsMultimodalFunctionResponse(modelId: string): boolean {
const geminiMajorVersion = getGeminiMajorVersion(modelId);
if (geminiMajorVersion !== undefined) {
return geminiMajorVersion >= 3;
}
return true;
}
/**
* Convert internal messages to Gemini Content[] format.
*/
export function convertMessages<T extends GoogleApiType>(
model: Model<T>,
context: Context,
): Content[] {
const contents: Content[] = [];
const normalizeToolCallId = (id: string): string => {
if (!requiresToolCallId(model.id)) {
return id;
}
return id.replace(/[^a-zA-Z0-9_-]/g, "_").slice(0, 64);
};
const transformedMessages = transformMessages(context.messages, model, normalizeToolCallId);
// Parallel calls need one immediate function-response turn. Gemini < 3 images cannot
// live inside functionResponse, so hold them until the consecutive result run ends.
const pendingToolResultImageTurns: Content[] = [];
let activeToolResultParts: Part[] | undefined;
const flushToolResultRun = (): void => {
contents.push(...pendingToolResultImageTurns);
pendingToolResultImageTurns.length = 0;
activeToolResultParts = undefined;
};
for (const msg of transformedMessages) {
if (msg.role !== "toolResult") {
flushToolResultRun();
}
if (msg.role === "user") {
if (typeof msg.content === "string") {
contents.push({
role: "user",
parts: [{ text: sanitizeSurrogates(msg.content) }],
});
} else {
const parts: Part[] = msg.content.map((item) => {
if (item.type === "text") {
return { text: sanitizeSurrogates(item.text) };
}
return {
inlineData: {
mimeType: item.mimeType,
data: item.data,
},
};
});
if (parts.length === 0) {
continue;
}
contents.push({
role: "user",
parts,
});
}
} else if (msg.role === "assistant") {
const parts: Part[] = [];
// Check if message is from same provider and model - only then keep thinking blocks
const isSameProviderAndModel = msg.provider === model.provider && msg.model === model.id;
for (const block of msg.content) {
if (block.type === "text") {
// Skip empty text blocks
if (!block.text || block.text.trim() === "") {
continue;
}
const thoughtSignature = resolveThoughtSignature(
isSameProviderAndModel,
block.textSignature,
);
parts.push({
text: sanitizeSurrogates(block.text),
...(thoughtSignature && { thoughtSignature }),
});
} else if (block.type === "thinking") {
// Skip empty thinking blocks
if (!block.thinking || block.thinking.trim() === "") {
continue;
}
// Only keep as thinking block if same provider AND same model
// Otherwise convert to plain text (no tags to avoid model mimicking them)
if (isSameProviderAndModel) {
const thoughtSignature = resolveThoughtSignature(
isSameProviderAndModel,
block.thinkingSignature,
);
parts.push({
thought: true,
text: sanitizeSurrogates(block.thinking),
...(thoughtSignature && { thoughtSignature }),
});
} else {
parts.push({
text: sanitizeSurrogates(block.thinking),
});
}
} else if (block.type === "toolCall") {
const thoughtSignature = resolveThoughtSignature(
isSameProviderAndModel,
block.thoughtSignature,
);
const part: Part = {
functionCall: {
name: block.name,
args: block.arguments ?? {},
...(requiresToolCallId(model.id) ? { id: block.id } : {}),
},
...(thoughtSignature && { thoughtSignature }),
};
parts.push(part);
}
}
if (parts.length === 0) {
continue;
}
contents.push({
role: "model",
parts,
});
} else if (msg.role === "toolResult") {
// Extract text and image content
const textResult = extractToolResultText(msg.content);
const imageContent = model.input.includes("image")
? msg.content.filter((c): c is ImageContent => c.type === "image")
: [];
const hasText = textResult.length > 0;
const hasImages = imageContent.length > 0;
const mediaPlaceholder = describeToolResultMediaPlaceholder(msg.content);
// Gemini 3+ models support multimodal function responses with images nested inside
// functionResponse.parts. Claude and other non-Gemini models behind Cloud Code Assist /
// Gemini < 3 still needs a separate user image turn.
const modelSupportsMultimodalFunctionResponse = supportsMultimodalFunctionResponse(model.id);
// Use "output" key for success, "error" key for errors as per SDK documentation
const responseValue = hasText ? sanitizeSurrogates(textResult) : (mediaPlaceholder ?? "");
const imageParts: Part[] = imageContent.map((imageBlock) => ({
inlineData: {
mimeType: imageBlock.mimeType,
data: imageBlock.data,
},
}));
const includeId = requiresToolCallId(model.id);
const functionResponsePart: Part = {
functionResponse: {
name: msg.toolName,
response: msg.isError ? { error: responseValue } : { output: responseValue },
...(hasImages && modelSupportsMultimodalFunctionResponse && { parts: imageParts }),
...(includeId ? { id: msg.toolCallId } : {}),
},
};
// Cloud Code Assist API requires all function responses to be in a single user turn.
if (activeToolResultParts) {
activeToolResultParts.push(functionResponsePart);
} else {
activeToolResultParts = [functionResponsePart];
contents.push({
role: "user",
parts: activeToolResultParts,
});
}
// For Gemini < 3, add images in a separate user message
if (hasImages && !modelSupportsMultimodalFunctionResponse) {
pendingToolResultImageTurns.push({
role: "user",
parts: [{ text: "Tool result image:" }, ...imageParts],
});
}
}
}
flushToolResultRun();
return contents;
}
const JSON_SCHEMA_META_DECLARATIONS = new Set([
"$schema",
"$id",
"$anchor",
"$dynamicAnchor",
"$vocabulary",
"$comment",
"$defs",
"definitions", // pre-draft-2019-09 equivalent of $defs
]);
/**
* Strip meta-declarations from a schema obj
*/
function sanitizeForOpenApi(schema: unknown): unknown {
if (typeof schema !== "object" || schema === null || Array.isArray(schema)) {
return schema;
}
const result: Record<string, unknown> = {};
for (const [key, value] of Object.entries(schema)) {
if (JSON_SCHEMA_META_DECLARATIONS.has(key)) {
continue;
}
result[key] = sanitizeForOpenApi(value);
}
return result;
}
/**
* Convert tools to Gemini function declarations format.
*
* By default uses `parametersJsonSchema` which supports full JSON Schema (including
* anyOf, oneOf, const, etc.). Set `useParameters` to true to use the legacy `parameters`
* field instead (OpenAPI 3.03 Schema). This is needed for Cloud Code Assist with Claude
* models, where the API translates `parameters` into Anthropic's `input_schema`.
*/
export function convertTools(
tools: Tool[],
useParameters = false,
): { functionDeclarations: Record<string, unknown>[] }[] | undefined {
if (tools.length === 0) {
return undefined;
}
return [
{
functionDeclarations: tools.map((tool) => ({
name: tool.name,
description: tool.description,
...(useParameters
? { parameters: sanitizeForOpenApi(tool.parameters as unknown) }
: { parametersJsonSchema: tool.parameters }),
})),
},
];
}
/**
* Map tool choice string to Gemini FunctionCallingConfigMode.
*/
export function mapToolChoice(choice: string): FunctionCallingConfigMode {
switch (choice) {
case "auto":
return FunctionCallingConfigMode.AUTO;
case "none":
return FunctionCallingConfigMode.NONE;
case "any":
return FunctionCallingConfigMode.ANY;
default:
return FunctionCallingConfigMode.AUTO;
}
}
export function createGoogleAssistantOutput<T extends GoogleApiType>(
model: Model<T>,
api: Api = model.api,
): AssistantMessage {
return {
role: "assistant",
content: [],
api,
provider: model.provider,
model: model.id,
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "stop",
timestamp: Date.now(),
};
}
export async function runGoogleGenerateContentLifecycle<T extends GoogleApiType>(params: {
stream: AssistantMessageEventStream;
model: Model<T>;
output: AssistantMessage;
options?: Pick<StreamOptions, "signal" | "onPayload">;
createClient: () => GoogleGenerateContentClient;
buildParams: () => GenerateContentParameters;
nextToolCallId: (name: string | undefined) => string;
}): Promise<void> {
const { stream, model, output, options } = params;
try {
const client = params.createClient();
let requestParams = params.buildParams();
const nextParams = await options?.onPayload?.(requestParams, model);
if (nextParams !== undefined) {
requestParams = nextParams as GenerateContentParameters;
}
const googleStream = await client.models.generateContentStream(requestParams);
await consumeGoogleGenerateContentStream({
chunks: googleStream,
model,
output,
stream,
signal: options?.signal,
nextToolCallId: params.nextToolCallId,
});
} catch (error) {
for (const block of output.content) {
if ("index" in block) {
delete (block as { index?: number }).index;
}
}
output.stopReason = options?.signal?.aborted ? "aborted" : "error";
output.errorMessage = error instanceof Error ? error.message : JSON.stringify(error);
stream.push({ type: "error", reason: output.stopReason, error: output });
stream.end();
}
}
export function buildGoogleGenerateContentParams<T extends GoogleApiType>(
model: Model<T>,
context: Context,
options: GoogleProviderOptions = {},
configHooks?: {
mapThinkingLevel?: (level: GoogleThinkingLevel) => ThinkingConfig["thinkingLevel"];
getDisabledThinkingConfig?: (model: Model<T>) => ThinkingConfig;
},
): GenerateContentParameters {
const contents = convertMessages(model, context);
const generationConfig: GenerateContentConfig = {};
if (options.temperature !== undefined) {
generationConfig.temperature = options.temperature;
}
if (options.maxTokens !== undefined) {
generationConfig.maxOutputTokens = options.maxTokens;
}
if (options.stop !== undefined && options.stop.length > 0) {
generationConfig.stopSequences = options.stop;
}
const config: GenerateContentConfig = {
...(Object.keys(generationConfig).length > 0 && generationConfig),
...(context.systemPrompt && {
systemInstruction: sanitizeSurrogates(stripSystemPromptCacheBoundary(context.systemPrompt)),
}),
...(context.tools && context.tools.length > 0 && { tools: convertTools(context.tools) }),
};
if (context.tools && context.tools.length > 0 && options.toolChoice) {
config.toolConfig = {
functionCallingConfig: {
mode: mapToolChoice(options.toolChoice),
},
};
} else {
config.toolConfig = undefined;
}
if (options.thinking?.enabled && model.reasoning) {
const thinkingConfig: ThinkingConfig = { includeThoughts: true };
if (options.thinking.level !== undefined) {
thinkingConfig.thinkingLevel = configHooks?.mapThinkingLevel
? configHooks.mapThinkingLevel(options.thinking.level)
: (options.thinking.level as ThinkingConfig["thinkingLevel"]);
} else if (options.thinking.budgetTokens !== undefined) {
thinkingConfig.thinkingBudget = options.thinking.budgetTokens;
}
config.thinkingConfig = thinkingConfig;
} else if (model.reasoning && options.thinking && !options.thinking.enabled) {
config.thinkingConfig = configHooks?.getDisabledThinkingConfig
? configHooks.getDisabledThinkingConfig(model)
: getDisabledGoogleThinkingConfig(model);
}
if (options.signal) {
if (options.signal.aborted) {
throw new Error("Request aborted");
}
config.abortSignal = options.signal;
}
return {
model: model.id,
contents,
config,
};
}
export function buildGoogleSimpleThinking<T extends GoogleApiType>(
model: Model<T>,
options: SimpleStreamOptions | undefined,
config?: {
includeGemma4ThinkingLevel?: boolean;
useFlashLiteBudgets?: boolean;
},
): GoogleThinkingOptions {
if (!options?.reasoning) {
return { enabled: false };
}
const clampedReasoning = clampThinkingLevel(model, options.reasoning);
const effort = (
clampedReasoning === "off" || clampedReasoning === "max" ? "high" : clampedReasoning
) as ClampedGoogleThinkingLevel;
if (
isGemini3ProModel(model) ||
isGemini3FlashModel(model) ||
(config?.includeGemma4ThinkingLevel && isGemma4Model(model))
) {
return {
enabled: true,
level: getGoogleThinkingLevel(effort, model, {
includeGemma4: config?.includeGemma4ThinkingLevel,
}),
};
}
return {
enabled: true,
budgetTokens: getGoogleBudget(model, effort, options.thinkingBudgets, {
useFlashLiteBudgets: config?.useFlashLiteBudgets,
}),
};
}
export function getDisabledGoogleThinkingConfig<T extends GoogleApiType>(
model: Model<T>,
config?: {
includeGemma4?: boolean;
mapThinkingLevel?: (level: GoogleThinkingLevel) => ThinkingConfig["thinkingLevel"];
},
): ThinkingConfig {
const mapThinkingLevel = (level: GoogleThinkingLevel): ThinkingConfig["thinkingLevel"] =>
config?.mapThinkingLevel
? config.mapThinkingLevel(level)
: (level as ThinkingConfig["thinkingLevel"]);
// Google docs: Gemini 3.1 Pro cannot disable thinking, and Gemini 3 Flash / Flash-Lite
// do not support full thinking-off either. For Gemini 3 models, use the lowest supported
// thinkingLevel without includeThoughts so hidden thinking remains invisible to OpenClaw.
if (isGemini3ProModel(model)) {
return { thinkingLevel: mapThinkingLevel("LOW") };
}
if (isGemini3FlashModel(model)) {
return { thinkingLevel: mapThinkingLevel("MINIMAL") };
}
if (config?.includeGemma4 && isGemma4Model(model)) {
return { thinkingLevel: mapThinkingLevel("MINIMAL") };
}
// Gemini 2.x supports disabling via thinkingBudget = 0.
return { thinkingBudget: 0 };
}
export function isGemma4Model<T extends GoogleApiType>(model: Model<T>): boolean {
return /gemma-?4/.test(model.id.toLowerCase());
}
export function isGemini3ProModel<T extends GoogleApiType>(model: Model<T>): boolean {
return /gemini-3(?:\.\d+)?-pro/.test(model.id.toLowerCase());
}
export function isGemini3FlashModel<T extends GoogleApiType>(model: Model<T>): boolean {
return /gemini-3(?:\.\d+)?-flash/.test(model.id.toLowerCase());
}
function getGoogleThinkingLevel<T extends GoogleApiType>(
effort: ClampedGoogleThinkingLevel,
model: Model<T>,
config?: { includeGemma4?: boolean },
): GoogleThinkingLevel {
if (isGemini3ProModel(model)) {
switch (effort) {
case "minimal":
case "low":
return "LOW";
case "medium":
case "high":
return "HIGH";
}
}
if (config?.includeGemma4 && isGemma4Model(model)) {
switch (effort) {
case "minimal":
case "low":
return "MINIMAL";
case "medium":
case "high":
return "HIGH";
}
}
switch (effort) {
case "minimal":
return "MINIMAL";
case "low":
return "LOW";
case "medium":
return "MEDIUM";
case "high":
return "HIGH";
}
return "HIGH";
}
function getGoogleBudget<T extends GoogleApiType>(
model: Model<T>,
effort: ClampedGoogleThinkingLevel,
customBudgets?: ThinkingBudgets,
config?: { useFlashLiteBudgets?: boolean },
): number {
if (customBudgets?.[effort] !== undefined) {
return customBudgets[effort];
}
if (model.id.includes("2.5-pro")) {
const budgets: Record<ClampedGoogleThinkingLevel, number> = {
minimal: 128,
low: 2048,
medium: 8192,
high: 32768,
};
return budgets[effort];
}
if (config?.useFlashLiteBudgets && model.id.includes("2.5-flash-lite")) {
const budgets: Record<ClampedGoogleThinkingLevel, number> = {
minimal: 512,
low: 2048,
medium: 8192,
high: 24576,
};
return budgets[effort];
}
if (model.id.includes("2.5-flash")) {
const budgets: Record<ClampedGoogleThinkingLevel, number> = {
minimal: 128,
low: 2048,
medium: 8192,
high: 24576,
};
return budgets[effort];
}
return -1;
}
/**
* Map Gemini FinishReason to our StopReason.
*/
export function mapStopReason(reason: FinishReason): StopReason {
switch (reason) {
case FinishReason.STOP:
return "stop";
case FinishReason.MAX_TOKENS:
return "length";
case FinishReason.BLOCKLIST:
case FinishReason.PROHIBITED_CONTENT:
case FinishReason.SPII:
case FinishReason.SAFETY:
case FinishReason.IMAGE_SAFETY:
case FinishReason.IMAGE_PROHIBITED_CONTENT:
case FinishReason.IMAGE_RECITATION:
case FinishReason.IMAGE_OTHER:
case FinishReason.RECITATION:
case FinishReason.FINISH_REASON_UNSPECIFIED:
case FinishReason.OTHER:
case FinishReason.LANGUAGE:
case FinishReason.MALFORMED_FUNCTION_CALL:
case FinishReason.UNEXPECTED_TOOL_CALL:
case FinishReason.NO_IMAGE:
return "error";
default: {
const exhaustive: never = reason;
throw new Error(`Unhandled stop reason: ${String(exhaustive)}`);
}
}
}
export async function consumeGoogleGenerateContentStream<T extends GoogleApiType>(params: {
chunks: AsyncIterable<GenerateContentResponse>;
model: Model<T>;
output: AssistantMessage;
stream: AssistantMessageEventStream;
signal?: AbortSignal;
nextToolCallId: (name: string | undefined) => string;
}): Promise<void> {
params.stream.push({ type: "start", partial: params.output });
let currentBlock: TextContent | ThinkingContent | null = null;
const blocks = params.output.content;
const toolCallIds = new Set<string>();
for (const block of blocks) {
if (block.type === "toolCall") {
toolCallIds.add(block.id);
}
}
const blockIndex = () => blocks.length - 1;
const endCurrentBlock = () => {
if (!currentBlock) {
return;
}
if (currentBlock.type === "text") {
params.stream.push({
type: "text_end",
contentIndex: blockIndex(),
content: currentBlock.text,
partial: params.output,
});
} else {
params.stream.push({
type: "thinking_end",
contentIndex: blockIndex(),
content: currentBlock.thinking,
partial: params.output,
});
}
currentBlock = null;
};
for await (const chunk of params.chunks) {
params.output.responseId ||= chunk.responseId;
const candidate = chunk.candidates?.[0];
if (candidate?.content?.parts) {
for (const part of candidate.content.parts) {
if (part.text !== undefined) {
const isThinking = isThinkingPart(part);
if (
!currentBlock ||
(isThinking && currentBlock.type !== "thinking") ||
(!isThinking && currentBlock.type !== "text")
) {
endCurrentBlock();
if (isThinking) {
currentBlock = { type: "thinking", thinking: "", thinkingSignature: undefined };
params.output.content.push(currentBlock);
params.stream.push({
type: "thinking_start",
contentIndex: blockIndex(),
partial: params.output,
});
} else {
currentBlock = { type: "text", text: "" };
params.output.content.push(currentBlock);
params.stream.push({
type: "text_start",
contentIndex: blockIndex(),
partial: params.output,
});
}
}
if (currentBlock.type === "thinking") {
currentBlock.thinking += part.text;
currentBlock.thinkingSignature = retainThoughtSignature(
currentBlock.thinkingSignature,
part.thoughtSignature,
);
params.stream.push({
type: "thinking_delta",
contentIndex: blockIndex(),
delta: part.text,
partial: params.output,
});
} else {
currentBlock.text += part.text;
currentBlock.textSignature = retainThoughtSignature(
currentBlock.textSignature,
part.thoughtSignature,
);
params.stream.push({
type: "text_delta",
contentIndex: blockIndex(),
delta: part.text,
partial: params.output,
});
}
}
if (part.functionCall) {
endCurrentBlock();
const providedId = part.functionCall.id;
const needsNewId = !providedId || toolCallIds.has(providedId);
const toolCall: ToolCall = {
type: "toolCall",
id: needsNewId ? params.nextToolCallId(part.functionCall.name) : providedId,
name: part.functionCall.name || "",
arguments: (part.functionCall.args as Record<string, unknown>) ?? {},
...(part.thoughtSignature && { thoughtSignature: part.thoughtSignature }),
};
params.output.content.push(toolCall);
toolCallIds.add(toolCall.id);
params.stream.push({
type: "toolcall_start",
contentIndex: blockIndex(),
partial: params.output,
});
params.stream.push({
type: "toolcall_delta",
contentIndex: blockIndex(),
delta: JSON.stringify(toolCall.arguments),
partial: params.output,
});
params.stream.push({
type: "toolcall_end",
contentIndex: blockIndex(),
toolCall,
partial: params.output,
});
}
}
}
if (candidate?.finishReason) {
params.output.stopReason = mapStopReason(candidate.finishReason);
// MAX_TOKENS can leave a complete-looking partial call. Only a normal
// Google stop may promote parsed calls into an executable tool-use turn.
if (
params.output.stopReason === "stop" &&
params.output.content.some((block) => block.type === "toolCall")
) {
params.output.stopReason = "toolUse";
}
}
if (chunk.usageMetadata) {
params.output.usage = {
input:
(chunk.usageMetadata.promptTokenCount || 0) -
(chunk.usageMetadata.cachedContentTokenCount || 0),
output:
(chunk.usageMetadata.candidatesTokenCount || 0) +
(chunk.usageMetadata.thoughtsTokenCount || 0),
cacheRead: chunk.usageMetadata.cachedContentTokenCount || 0,
cacheWrite: 0,
totalTokens: chunk.usageMetadata.totalTokenCount || 0,
cost: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
total: 0,
},
};
calculateCost(params.model, params.output.usage);
}
}
endCurrentBlock();
if (params.signal?.aborted) {
throw new Error("Request was aborted");
}
if (params.output.stopReason === "aborted" || params.output.stopReason === "error") {
throw new Error("An unknown error occurred");
}
params.stream.push({
type: "done",
reason: params.output.stopReason,
message: params.output,
});
params.stream.end();
}

View File

@@ -0,0 +1,193 @@
// Google Vertex provider wires Google shared streaming through Vertex credentials.
import {
type GenerateContentParameters,
GoogleGenAI,
type HttpOptions,
ResourceScope,
ThinkingLevel as VertexThinkingLevel,
} from "@google/genai";
import type { Context, Model, SimpleStreamOptions, StreamFunction } from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import type { GoogleThinkingLevel } from "./google-shared.js";
import {
buildGoogleGenerateContentParams,
buildGoogleSimpleThinking,
createGoogleAssistantOutput,
getDisabledGoogleThinkingConfig,
type GoogleProviderOptions,
runGoogleGenerateContentLifecycle,
} from "./google-shared.js";
import { buildBaseOptions } from "./simple-options.js";
export interface GoogleVertexOptions extends GoogleProviderOptions {
project?: string;
location?: string;
}
const API_VERSION = "v1";
const GCP_VERTEX_CREDENTIALS_MARKER = "gcp-vertex-credentials";
const THINKING_LEVEL_MAP: Record<GoogleThinkingLevel, VertexThinkingLevel> = {
THINKING_LEVEL_UNSPECIFIED: VertexThinkingLevel.THINKING_LEVEL_UNSPECIFIED,
MINIMAL: VertexThinkingLevel.MINIMAL,
LOW: VertexThinkingLevel.LOW,
MEDIUM: VertexThinkingLevel.MEDIUM,
HIGH: VertexThinkingLevel.HIGH,
};
// Counter for generating unique tool call IDs
let toolCallCounter = 0;
export const streamGoogleVertex: StreamFunction<"google-vertex", GoogleVertexOptions> = (
model: Model<"google-vertex">,
context: Context,
options?: GoogleVertexOptions,
) => {
const stream = new AssistantMessageEventStream();
const output = createGoogleAssistantOutput(model, "google-vertex");
void runGoogleGenerateContentLifecycle({
stream,
model,
output,
options,
createClient: () => {
const apiKey = resolveApiKey(options);
// Create the client using either a Vertex API key, if provided, or ADC with project and location
return apiKey
? createClientWithApiKey(model, apiKey, options?.headers)
: createClient(model, resolveProject(options), resolveLocation(options), options?.headers);
},
buildParams: () => buildParams(model, context, options),
nextToolCallId: (name) => `${name}_${Date.now()}_${++toolCallCounter}`,
});
return stream;
};
export const streamSimpleGoogleVertex: StreamFunction<"google-vertex", SimpleStreamOptions> = (
model: Model<"google-vertex">,
context: Context,
options?: SimpleStreamOptions,
) => {
const base = buildBaseOptions(model, options, undefined);
return streamGoogleVertex(model, context, {
...base,
thinking: buildGoogleSimpleThinking(model, options),
} satisfies GoogleVertexOptions);
};
function createClient(
model: Model<"google-vertex">,
project: string,
location: string,
optionsHeaders?: Record<string, string>,
): GoogleGenAI {
return new GoogleGenAI({
vertexai: true,
project,
location,
apiVersion: API_VERSION,
httpOptions: buildHttpOptions(model, optionsHeaders),
});
}
function createClientWithApiKey(
model: Model<"google-vertex">,
apiKey: string,
optionsHeaders?: Record<string, string>,
): GoogleGenAI {
return new GoogleGenAI({
vertexai: true,
apiKey,
apiVersion: API_VERSION,
httpOptions: buildHttpOptions(model, optionsHeaders),
});
}
function buildHttpOptions(
model: Model<"google-vertex">,
optionsHeaders?: Record<string, string>,
): HttpOptions | undefined {
const httpOptions: HttpOptions = {};
const baseUrl = resolveCustomBaseUrl(model.baseUrl);
if (baseUrl) {
httpOptions.baseUrl = baseUrl;
httpOptions.baseUrlResourceScope = ResourceScope.COLLECTION;
if (baseUrlIncludesApiVersion(baseUrl)) {
httpOptions.apiVersion = "";
}
}
if (model.headers || optionsHeaders) {
httpOptions.headers = { ...model.headers, ...optionsHeaders };
}
return Object.keys(httpOptions).length > 0 ? httpOptions : undefined;
}
function resolveCustomBaseUrl(baseUrl: string): string | undefined {
const trimmed = baseUrl.trim();
if (!trimmed || trimmed.includes("{location}")) {
return undefined;
}
return trimmed;
}
function baseUrlIncludesApiVersion(baseUrl: string): boolean {
try {
const url = new URL(baseUrl);
return url.pathname.split("/").some((part) => /^v\d+(?:beta\d*)?$/.test(part));
} catch {
return /(?:^|\/)v\d+(?:beta\d*)?(?:\/|$)/.test(baseUrl);
}
}
function resolveApiKey(options?: GoogleVertexOptions): string | undefined {
const apiKey = options?.apiKey?.trim() || process.env.GOOGLE_CLOUD_API_KEY?.trim();
if (!apiKey || apiKey === GCP_VERTEX_CREDENTIALS_MARKER || isPlaceholderApiKey(apiKey)) {
return undefined;
}
return apiKey;
}
function isPlaceholderApiKey(apiKey: string): boolean {
return /^<[^>]+>$/.test(apiKey);
}
function resolveProject(options?: GoogleVertexOptions): string {
const project =
options?.project || process.env.GOOGLE_CLOUD_PROJECT || process.env.GCLOUD_PROJECT;
if (!project) {
throw new Error(
"Vertex AI requires a project ID. Set GOOGLE_CLOUD_PROJECT/GCLOUD_PROJECT or pass project in options.",
);
}
return project;
}
function resolveLocation(options?: GoogleVertexOptions): string {
const location = options?.location || process.env.GOOGLE_CLOUD_LOCATION;
if (!location) {
throw new Error(
"Vertex AI requires a location. Set GOOGLE_CLOUD_LOCATION or pass location in options.",
);
}
return location;
}
function buildParams(
model: Model<"google-vertex">,
context: Context,
options: GoogleVertexOptions = {},
): GenerateContentParameters {
return buildGoogleGenerateContentParams(model, context, options, {
mapThinkingLevel: mapVertexThinkingLevel,
getDisabledThinkingConfig: (modelLocal) =>
getDisabledGoogleThinkingConfig(modelLocal, { mapThinkingLevel: mapVertexThinkingLevel }),
});
}
function mapVertexThinkingLevel(level: GoogleThinkingLevel): VertexThinkingLevel {
return THINKING_LEVEL_MAP[level];
}

View File

@@ -0,0 +1,95 @@
// Google provider adapts Gemini streams and tools to the agent runtime.
import { type GenerateContentParameters, GoogleGenAI } from "@google/genai";
import { getEnvApiKey } from "../env-api-keys.js";
import type { Context, Model, SimpleStreamOptions, StreamFunction } from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import {
buildGoogleGenerateContentParams,
buildGoogleSimpleThinking,
createGoogleAssistantOutput,
getDisabledGoogleThinkingConfig,
type GoogleProviderOptions,
runGoogleGenerateContentLifecycle,
} from "./google-shared.js";
import { buildBaseOptions } from "./simple-options.js";
export type GoogleOptions = GoogleProviderOptions;
// Counter for generating unique tool call IDs
let toolCallCounter = 0;
export const streamGoogle: StreamFunction<"google-generative-ai", GoogleOptions> = (
model: Model<"google-generative-ai">,
context: Context,
options?: GoogleOptions,
) => {
const stream = new AssistantMessageEventStream();
const output = createGoogleAssistantOutput(model, "google-generative-ai");
void runGoogleGenerateContentLifecycle({
stream,
model,
output,
options,
createClient: () => {
const apiKey = options?.apiKey || getEnvApiKey(model.provider) || "";
return createClient(model, apiKey, options?.headers);
},
buildParams: () => buildParams(model, context, options),
nextToolCallId: (name) => `${name}_${Date.now()}_${++toolCallCounter}`,
});
return stream;
};
export const streamSimpleGoogle: StreamFunction<"google-generative-ai", SimpleStreamOptions> = (
model: Model<"google-generative-ai">,
context: Context,
options?: SimpleStreamOptions,
) => {
const apiKey = options?.apiKey || getEnvApiKey(model.provider);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const base = buildBaseOptions(model, options, apiKey);
return streamGoogle(model, context, {
...base,
thinking: buildGoogleSimpleThinking(model, options, {
includeGemma4ThinkingLevel: true,
useFlashLiteBudgets: true,
}),
} satisfies GoogleOptions);
};
function createClient(
model: Model<"google-generative-ai">,
apiKey?: string,
optionsHeaders?: Record<string, string>,
): GoogleGenAI {
const httpOptions: { baseUrl?: string; apiVersion?: string; headers?: Record<string, string> } =
{};
if (model.baseUrl) {
httpOptions.baseUrl = model.baseUrl;
httpOptions.apiVersion = ""; // baseUrl already includes version path, don't append
}
if (model.headers || optionsHeaders) {
httpOptions.headers = { ...model.headers, ...optionsHeaders };
}
return new GoogleGenAI({
apiKey,
httpOptions: Object.keys(httpOptions).length > 0 ? httpOptions : undefined,
});
}
function buildParams(
model: Model<"google-generative-ai">,
context: Context,
options: GoogleOptions = {},
): GenerateContentParameters {
return buildGoogleGenerateContentParams(model, context, options, {
getDisabledThinkingConfig: (modelLocal) =>
getDisabledGoogleThinkingConfig(modelLocal, { includeGemma4: true }),
});
}

View File

@@ -0,0 +1,184 @@
// Mistral provider tests cover bounded-stream-read helper (`createBoundedMistralFetcher`).
import http from "node:http";
import type { AddressInfo } from "node:net";
import { describe, expect, it } from "vitest";
import { createBoundedMistralFetcher } from "./mistral.js";
const MAX = 16 * 1024 * 1024;
const TOTAL = 18 * 1024 * 1024;
async function readAllChunks(body: ReadableStream<Uint8Array> | null): Promise<{ total: number }> {
if (!body) {
return { total: 0 };
}
const reader = body.getReader();
let total = 0;
while (true) {
const { done, value } = await reader.read();
if (done) {
break;
}
if (value) {
total += value.byteLength;
}
}
return { total };
}
describe("Mistral bounded-stream-read real wire proof (loopback http.createServer)", () => {
it("caps an oversized body streamed chunked over real wire", async () => {
const fetcher = createBoundedMistralFetcher(MAX);
const CHUNK = 1024 * 1024;
const server = http.createServer((req, res) => {
res.writeHead(200, { "content-type": "application/octet-stream" });
let sent = 0;
const tick = setInterval(() => {
if (sent < 18) {
res.write(Buffer.alloc(CHUNK));
sent++;
} else {
clearInterval(tick);
res.end();
}
}, 1);
});
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", () => {
resolve();
});
});
const port = (server.address() as AddressInfo).port;
let captured: Error | undefined;
let totalGot = 0;
try {
const response = await fetcher(`http://127.0.0.1:${port}/`);
// Wire framing merges TCP packets, so the reported size at throw time
// is between MAX (cap) and TOTAL (cap + last merged packet). Both
// bounds prove (a) cap fired (got > MAX) and (b) we did not buffer
// beyond the server's full 18 MiB (got < TOTAL).
try {
const result = await readAllChunks(response.body);
totalGot = result.total;
} catch (err) {
captured = err as Error;
}
expect(captured).toBeInstanceOf(Error);
const match = (captured as Error).message.match(
/mistral: stream body exceeds \d+ bytes \(got (\d+)\)/,
);
expect(match).not.toBeNull();
const got = Number(match![1]);
expect(got).toBeGreaterThan(MAX);
expect(got).toBeLessThan(TOTAL);
// Print to vitest stdout for PR-body real behavior proof capture.
console.log(
`[mistral bounded-stream proof] oversized path: cap=${MAX} reported=${got} server_total=${TOTAL}`,
);
} finally {
await new Promise<void>((resolve) => {
server.close(() => {
resolve();
});
});
if (totalGot > 0) {
// Use the value to satisfy strict unused rules without affecting asserts.
expect(totalGot).toBeGreaterThan(0);
}
}
});
it("returns a Response with exact bytes for normal-size responses on real wire", async () => {
const fetcher = createBoundedMistralFetcher(MAX);
const bodyText = 'data: {"choices":[{"delta":{"content":"hello"}}]}\n\ndata: [DONE]\n\n';
const server = http.createServer((req, res) => {
res.writeHead(200, { "content-type": "text/event-stream" });
res.end(bodyText);
});
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(0, "127.0.0.1", () => {
resolve();
});
});
const port = (server.address() as AddressInfo).port;
try {
const response = await fetcher(`http://127.0.0.1:${port}/`);
expect(response.status).toBe(200);
const { total } = await readAllChunks(response.body);
expect(total).toBe(Buffer.byteLength(bodyText, "utf8"));
console.log(
`[mistral bounded-stream proof] normal path: cap=${MAX} returned=${total} body=${JSON.stringify(bodyText)}`,
);
} finally {
await new Promise<void>((resolve) => {
server.close(() => {
resolve();
});
});
}
});
});
// Drive the bounded fetcher directly against a synthetic ReadableStream that
// exceeds the cap. Bypasses any HTTP layer; proves the cap fires against an
// unbounded chunked source, mirroring what the Mistral SDK's internal SSE
// parser (`EventStream`) would see when a streaming body exceeds 16 MiB.
describe("Mistral bounded-stream-read direct (synthetic ReadableStream)", () => {
it("caps an oversized synthetic ReadableStream at 16 MiB", async () => {
const fetcher = createBoundedMistralFetcher(MAX);
const CHUNK = 1024 * 1024;
let sent = 0;
const synthetic = new ReadableStream<Uint8Array>({
pull(controller) {
if (sent < 18) {
controller.enqueue(new Uint8Array(CHUNK));
sent++;
} else {
controller.close();
}
},
});
// Build the same shape `fetcher` expects from a real fetch(): a
// `Response` whose `body` is a ReadableStream.
const syntheticResponse = new Response(synthetic, {
status: 200,
headers: { "content-type": "application/octet-stream" },
});
let captured: Error | undefined;
try {
// Replace the fetcher's internal `fetch` call by exercising the
// post-fetchResponse code path directly: build a `Wrapped`
// that re-enters `fetcher` as if a real fetch returned our
// synthetic Response, by patching the global fetch.
const originalFetch = globalThis.fetch;
globalThis.fetch = (() => Promise.resolve(syntheticResponse)) as typeof globalThis.fetch;
try {
const wrapped = await fetcher("http://unused.invalid/");
try {
await readAllChunks(wrapped.body);
} catch (err) {
captured = err as Error;
}
} finally {
globalThis.fetch = originalFetch;
}
expect(captured).toBeInstanceOf(Error);
const match = (captured as Error).message.match(
/mistral: stream body exceeds \d+ bytes \(got (\d+)\)/,
);
expect(match).not.toBeNull();
const got = Number(match![1]);
// Synthetic stream chunks are exactly 1 MiB aligned, so cap+1 reads
// give exactly cap + 1 MiB = 16 MiB + 1 MiB = 17 825 792 bytes.
expect(got).toBe(16777216 + CHUNK);
} finally {
// Best-effort cleanup if the test threw mid-flight.
// No intervals to clear for this test; the synthetic stream closes
// automatically when `sent >= 18`.
}
});
});

View File

@@ -0,0 +1,362 @@
// Mistral provider tests cover request mapping and stream conversion.
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { configureAiTransportHost } from "../host.js";
import type { Context, Model } from "../types.js";
import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "../utils/system-prompt-cache-boundary.js";
const mistralMockState = vi.hoisted(() => ({
payloads: [] as unknown[],
}));
vi.mock("@mistralai/mistralai", async () => {
// Preserve real exports for everything except `Mistral`, so the new
// imports of `HTTPClient` and `Fetcher` introduced by the bounded-stream
// helper (`createBoundedMistralHttpClient`) resolve correctly. Only
// `Mistral` itself is overridden so the test can capture payloads without
// any actual HTTP traffic.
const actual =
await vi.importActual<typeof import("@mistralai/mistralai")>("@mistralai/mistralai");
return {
...actual,
Mistral: class MockMistral {
chat = {
stream: vi.fn(async (payload: unknown) => {
mistralMockState.payloads.push(payload);
throw new Error("stop before network");
}),
};
},
};
});
import { streamMistral, streamSimpleMistral } from "./mistral.js";
function makeMistralModel(): Model<"mistral-conversations"> {
return {
id: "mistral-large-latest",
name: "Mistral Large",
api: "mistral-conversations",
provider: "mistral",
baseUrl: "https://api.mistral.ai",
reasoning: false,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128_000,
maxTokens: 8192,
};
}
const context = {
messages: [{ role: "user", content: "hello", timestamp: 0 }],
} satisfies Context;
function makeUnreadableParameterTool() {
const tool = {
name: "broken_tool",
description: "broken tool",
parameters: { type: "object", properties: {} },
async execute() {
return { content: [{ type: "text", text: "broken" }] };
},
};
Object.defineProperty(tool, "parameters", {
enumerable: true,
get() {
throw new Error("fuzzplugin parameters getter exploded");
},
});
return tool;
}
describe("Mistral provider", () => {
beforeEach(() => {
mistralMockState.payloads = [];
});
afterEach(() => {
configureAiTransportHost({});
});
it("forwards simple stop sequences to Mistral stop", async () => {
const stream = streamSimpleMistral(makeMistralModel(), context, {
apiKey: "sk-mistral-provider",
stop: ["STOP"],
});
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect((mistralMockState.payloads[0] as { stop?: unknown }).stop).toEqual(["STOP"]);
});
it("skips unreadable tool schemas while preserving healthy Mistral tools", async () => {
const stream = streamMistral(
makeMistralModel(),
{
...context,
tools: [
makeUnreadableParameterTool(),
{
name: "healthy_tool",
description: "healthy tool",
parameters: {
type: "object",
properties: {
query: { type: "string" },
},
},
async execute() {
return { content: [{ type: "text", text: "ok" }] };
},
},
] as never,
},
{
apiKey: "sk-mistral-provider",
},
);
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect((mistralMockState.payloads[0] as { tools?: unknown[] }).tools).toEqual([
{
type: "function",
function: {
name: "healthy_tool",
description: "healthy tool",
parameters: {
type: "object",
properties: {
query: { type: "string" },
},
},
strict: false,
},
},
]);
});
it("omits tools and automatic tool choice when every schema is unreadable", async () => {
const stream = streamMistral(
makeMistralModel(),
{
...context,
tools: [makeUnreadableParameterTool()] as never,
},
{
apiKey: "sk-mistral-provider",
toolChoice: "auto",
},
);
const result = await stream.result();
const payload = mistralMockState.payloads[0] as Record<string, unknown>;
expect(result.stopReason).toBe("error");
expect(payload).not.toHaveProperty("tools");
expect(payload).not.toHaveProperty("toolChoice");
});
it("fails locally when a pinned Mistral tool choice is skipped", async () => {
const stream = streamMistral(
makeMistralModel(),
{
...context,
tools: [
makeUnreadableParameterTool(),
{
name: "healthy_tool",
description: "healthy tool",
parameters: { type: "object", properties: {} },
async execute() {
return { content: [{ type: "text", text: "ok" }] };
},
},
] as never,
},
{
apiKey: "sk-mistral-provider",
toolChoice: { type: "function", function: { name: "broken_tool" } },
},
);
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toContain(
'Mistral tool_choice requested unavailable tool "broken_tool"',
);
expect(mistralMockState.payloads).toHaveLength(0);
});
it("validates and emits one snapshot of a pinned Mistral tool name", async () => {
let nameReads = 0;
const stream = streamMistral(
makeMistralModel(),
{
...context,
tools: [
{
name: "healthy_tool",
description: "healthy tool",
parameters: { type: "object", properties: {} },
async execute() {
return { content: [{ type: "text", text: "ok" }] };
},
},
] as never,
},
{
apiKey: "sk-mistral-provider",
toolChoice: {
type: "function",
function: {
get name() {
nameReads += 1;
return nameReads === 1 ? "healthy_tool" : "broken_tool";
},
},
},
},
);
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(nameReads).toBe(1);
expect((mistralMockState.payloads[0] as { toolChoice?: unknown }).toolChoice).toEqual({
type: "function",
function: { name: "healthy_tool" },
});
});
it("strips the internal cache boundary marker from the system message", async () => {
const stream = streamSimpleMistral(
makeMistralModel(),
{
systemPrompt: `Stable${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic`,
messages: [{ role: "user", content: "hello", timestamp: 0 }],
},
{ apiKey: "sk-mistral-provider" },
);
await stream.result();
const payload = mistralMockState.payloads[0] as {
messages: Array<{ role: string; content: string }>;
};
const systemMessage = payload.messages.find((message) => message.role === "system");
expect(systemMessage?.content).toBe("Stable\nDynamic");
expect(JSON.stringify(payload)).not.toContain("OPENCLAW_CACHE_BOUNDARY");
});
it("serializes structured non-image blocks in tool results as JSON text", async () => {
// Prove the host redaction port is applied to structured tool-result text.
configureAiTransportHost({
redactToolPayloadText: (text) => text.replaceAll('"value"', '"***"'),
});
const testContext = {
messages: [
{
role: "user",
content: "hello",
timestamp: 1,
},
{
role: "assistant",
provider: "mistral",
api: "mistral-conversations",
model: "mistral-large-latest",
stopReason: "toolUse",
timestamp: 0,
content: [{ type: "toolCall", id: "tool_1", name: "fetch", arguments: {} }],
},
{
role: "toolResult",
toolCallId: "tool_1",
content: [
{
type: "resource",
resource: {
uri: "https://example.com/data.json",
mimeType: "application/json",
text: '{"key":"value"}',
},
},
],
isError: false,
timestamp: 0,
},
],
} as unknown as Context;
const stream = streamMistral(makeMistralModel(), testContext, {
apiKey: "sk-mistral-provider",
});
await stream.result();
const payload = mistralMockState.payloads[0] as {
messages: Array<{ role: string; content: string | Array<{ type: string; text?: string }> }>;
};
const toolMessage = payload.messages.find((message) => message.role === "tool");
expect(toolMessage).toBeDefined();
const toolContent = Array.isArray(toolMessage!.content) ? toolMessage!.content : [];
const textBlock = toolContent.find((block) => block.type === "text");
expect(textBlock?.text).toEqual(expect.stringContaining('{"type":"resource"'));
expect(textBlock?.text).toContain('{\\"key\\":\\"***\\"}');
expect(textBlock?.text).not.toContain('{\\"key\\":\\"value\\"}');
});
it("serializes structured-only tool results instead of empty fallback", async () => {
const testContext = {
messages: [
{
role: "user",
content: "hello",
timestamp: 1,
},
{
role: "assistant",
provider: "mistral",
api: "mistral-conversations",
model: "mistral-large-latest",
stopReason: "toolUse",
timestamp: 0,
content: [{ type: "toolCall", id: "tool_1", name: "get_file", arguments: {} }],
},
{
role: "toolResult",
toolCallId: "tool_1",
content: [
{
type: "resource_link",
uri: "https://example.com/file.txt",
name: "file.txt",
mimeType: "text/plain",
size: 100,
},
],
isError: false,
timestamp: 0,
},
],
} as unknown as Context;
const stream = streamMistral(makeMistralModel(), testContext, {
apiKey: "sk-mistral-provider",
});
await stream.result();
const payload = mistralMockState.payloads[0] as {
messages: Array<{ role: string; content: string | Array<{ type: string; text?: string }> }>;
};
const toolMessage = payload.messages.find((message) => message.role === "tool");
expect(toolMessage).toBeDefined();
const toolContent = Array.isArray(toolMessage!.content) ? toolMessage!.content : [];
const textBlock = toolContent.find((block) => block.type === "text");
// Structured blocks should provide the output, not an empty fallback
expect(textBlock?.text).toEqual(expect.stringContaining('{"type":"resource_link"'));
expect(textBlock?.text).not.toContain("(no tool output)");
});
});

View File

@@ -0,0 +1,829 @@
// Mistral provider adapts Mistral streams and tool calls to the runtime.
import { HTTPClient, Mistral, type Fetcher } from "@mistralai/mistralai";
import type {
ChatCompletionStreamRequest,
ChatCompletionStreamRequestMessage,
CompletionEvent,
ContentChunk,
FunctionTool,
} from "@mistralai/mistralai/models/components";
import { getEnvApiKey } from "../env-api-keys.js";
import { calculateCost, clampThinkingLevel } from "../model-utils.js";
import type {
AssistantMessage,
Context,
Message,
Model,
SimpleStreamOptions,
StopReason,
StreamFunction,
StreamOptions,
TextContent,
ThinkingContent,
Tool,
ToolCall,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { shortHash } from "../utils/hash.js";
import { parseStreamingJson } from "../utils/json-parse.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
import { createSseByteGuard } from "../utils/streaming-byte-guard.js";
import { stripSystemPromptCacheBoundary } from "../utils/system-prompt-cache-boundary.js";
import { buildBaseOptions } from "./simple-options.js";
import { describeToolResultMediaPlaceholder, extractToolResultText } from "./tool-result-text.js";
import { transformMessages } from "./transform-messages.js";
const MISTRAL_TOOL_CALL_ID_LENGTH = 9;
const MAX_MISTRAL_ERROR_BODY_CHARS = 4000;
// 16 MiB cap on Mistral streaming success bodies, matching the
// `PROVIDER_TEXT_RESPONSE_MAX_BYTES` / `PROVIDER_JSON_RESPONSE_MAX_BYTES`
// 16 MiB cap used elsewhere. A hostile or malfunctioning Mistral-compatible
// endpoint cannot exhaust memory by streaming an unbounded SSE body;
// `createSseByteGuard` cancels the upstream reader and throws once the
// accumulated byte count exceeds this cap.
const MISTRAL_STREAM_BODY_MAX_BYTES = 16 * 1024 * 1024;
/**
* Builds a `Fetcher` that wraps the default `fetch` with a 16 MiB byte cap
* on streamed response bodies. The wrapped `Response.body` exposes a
* `ReadableStream` whose chunks flow through `createSseByteGuard`, so the
* SDK's internal SSE parser (`EventStream` in
* `@mistralai/mistralai/lib/event-streams.ts`) reads exactly as it would on
* an unbounded body — but bounded.
*
* Bodyless responses (no `body` or no `getReader`) are returned unchanged so
* the SDK's error-path `res.arrayBuffer()` call still works.
*/
export function createBoundedMistralFetcher(
maxBytes: number = MISTRAL_STREAM_BODY_MAX_BYTES,
): Fetcher {
return async (input, init) => {
const response = init == null ? await fetch(input) : await fetch(input, init);
if (!response.body || typeof response.body.getReader !== "function") {
return response;
}
const reader = response.body.getReader();
const guard = createSseByteGuard(reader, {
maxBytes,
onOverflow: ({ size, maxBytes: cap }) =>
new Error(`mistral: stream body exceeds ${cap} bytes (got ${size})`),
});
// Re-shape the response body so the SDK's `responseBody.getReader()`
// call inside `EventStream` resolves to a stream whose `read()` is
// routed through `guard.read()`. Cancellation is also forwarded.
const guardedStream = new ReadableStream<Uint8Array>({
async pull(controller) {
const { done, value } = await guard.read();
if (done) {
controller.close();
return;
}
controller.enqueue(value);
},
async cancel(reason) {
await guard.cancel(reason);
},
});
return new Response(guardedStream, {
status: response.status,
statusText: response.statusText,
headers: response.headers,
});
};
}
/**
* Provider-specific options for the Mistral API.
*/
type MistralReasoningEffort = "none" | "high";
export interface MistralOptions extends StreamOptions {
toolChoice?:
| "auto"
| "none"
| "any"
| "required"
| { type: "function"; function: { name: string } };
promptMode?: "reasoning";
reasoningEffort?: MistralReasoningEffort;
}
/**
* Stream responses from Mistral using `chat.stream`.
*/
export const streamMistral: StreamFunction<"mistral-conversations", MistralOptions> = (
model: Model<"mistral-conversations">,
context: Context,
options?: MistralOptions,
) => {
const stream = new AssistantMessageEventStream();
void (async () => {
const output = createOutput(model);
try {
const apiKey = options?.apiKey || getEnvApiKey(model.provider);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
// Intentionally per-request: avoids shared SDK mutable state across concurrent consumers.
const mistral = new Mistral({
apiKey,
serverURL: model.baseUrl,
// Bound the streamed Mistral response body at 16 MiB so a hostile or
// malfunctioning endpoint cannot exhaust memory. The fetcher is
// injected via the SDK's `HTTPClient` (see
// `@mistralai/mistralai/lib/sdks.ts` `ClientSDK` constructor: when
// `httpClient` is passed, `ClientSDK.#httpClient` is set from it and
// every `chat.stream` / `complete` call routes through
// `HTTPClient.request` → `this.fetcher(req)`).
httpClient: new HTTPClient({ fetcher: createBoundedMistralFetcher() }),
});
const normalizeMistralToolCallId = createMistralToolCallIdNormalizer();
const transformedMessages = transformMessages(context.messages, model, (id) =>
normalizeMistralToolCallId(id),
);
let payload = buildChatPayload(model, context, transformedMessages, options);
const nextPayload = await options?.onPayload?.(payload, model);
if (nextPayload !== undefined) {
payload = nextPayload as ChatCompletionStreamRequest;
}
const mistralStream = await mistral.chat.stream(payload, buildRequestOptions(model, options));
stream.push({ type: "start", partial: output });
await consumeChatStream(model, output, stream, mistralStream);
if (options?.signal?.aborted) {
throw new Error("Request was aborted");
}
if (output.stopReason === "aborted" || output.stopReason === "error") {
throw new Error("An unknown error occurred");
}
stream.push({ type: "done", reason: output.stopReason, message: output });
stream.end();
} catch (error) {
for (const block of output.content) {
// partialArgs is only a streaming scratch buffer; never persist it.
delete (block as { partialArgs?: string }).partialArgs;
}
output.stopReason = options?.signal?.aborted ? "aborted" : "error";
output.errorMessage = formatMistralError(error);
stream.push({ type: "error", reason: output.stopReason, error: output });
stream.end();
}
})();
return stream;
};
/**
* Maps provider-agnostic `SimpleStreamOptions` to Mistral options.
*/
export const streamSimpleMistral: StreamFunction<"mistral-conversations", SimpleStreamOptions> = (
model: Model<"mistral-conversations">,
context: Context,
options?: SimpleStreamOptions,
) => {
const apiKey = options?.apiKey || getEnvApiKey(model.provider);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const base = buildBaseOptions(model, options, apiKey);
const clampedReasoning = options?.reasoning
? clampThinkingLevel(model, options.reasoning)
: undefined;
const reasoning = clampedReasoning === "off" ? undefined : clampedReasoning;
const shouldUseReasoning = model.reasoning && reasoning !== undefined;
return streamMistral(model, context, {
...base,
promptMode: shouldUseReasoning && usesPromptModeReasoning(model) ? "reasoning" : undefined,
reasoningEffort:
shouldUseReasoning && usesReasoningEffort(model)
? mapReasoningEffort(model, reasoning)
: undefined,
} satisfies MistralOptions);
};
function createOutput(model: Model<"mistral-conversations">): AssistantMessage {
return {
role: "assistant",
content: [],
api: model.api,
provider: model.provider,
model: model.id,
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "stop",
timestamp: Date.now(),
};
}
function createMistralToolCallIdNormalizer(): (id: string) => string {
const idMap = new Map<string, string>();
const reverseMap = new Map<string, string>();
return (id: string): string => {
const existing = idMap.get(id);
if (existing) {
return existing;
}
let attempt = 0;
while (true) {
const candidate = deriveMistralToolCallId(id, attempt);
const owner = reverseMap.get(candidate);
if (!owner || owner === id) {
idMap.set(id, candidate);
reverseMap.set(candidate, id);
return candidate;
}
attempt++;
}
};
}
function deriveMistralToolCallId(id: string, attempt: number): string {
const normalized = id.replace(/[^a-zA-Z0-9]/g, "");
if (attempt === 0 && normalized.length === MISTRAL_TOOL_CALL_ID_LENGTH) {
return normalized;
}
const seedBase = normalized || id;
const seed = attempt === 0 ? seedBase : `${seedBase}:${attempt}`;
return shortHash(seed)
.replace(/[^a-zA-Z0-9]/g, "")
.slice(0, MISTRAL_TOOL_CALL_ID_LENGTH);
}
function formatMistralError(error: unknown): string {
if (error instanceof Error) {
const sdkError = error as Error & { statusCode?: unknown; body?: unknown };
const statusCode = typeof sdkError.statusCode === "number" ? sdkError.statusCode : undefined;
const bodyText = typeof sdkError.body === "string" ? sdkError.body.trim() : undefined;
if (statusCode !== undefined && bodyText) {
return `Mistral API error (${statusCode}): ${truncateErrorText(bodyText, MAX_MISTRAL_ERROR_BODY_CHARS)}`;
}
if (statusCode !== undefined) {
return `Mistral API error (${statusCode}): ${error.message}`;
}
return error.message;
}
return safeJsonStringify(error);
}
function truncateErrorText(text: string, maxChars: number): string {
if (text.length <= maxChars) {
return text;
}
return `${text.slice(0, maxChars)}... [truncated ${text.length - maxChars} chars]`;
}
function safeJsonStringify(value: unknown): string {
try {
const serialized = JSON.stringify(value);
return serialized === undefined ? String(value) : serialized;
} catch {
return String(value);
}
}
function buildRequestOptions(model: Model<"mistral-conversations">, options?: MistralOptions) {
const requestOptions: {
signal?: AbortSignal;
retries: { strategy: "none" };
headers?: Record<string, string>;
} = {
retries: { strategy: "none" },
};
if (options?.signal) {
requestOptions.signal = options.signal;
}
const headers: Record<string, string> = {};
if (model.headers) {
Object.assign(headers, model.headers);
}
if (options?.headers) {
Object.assign(headers, options.headers);
}
// Mistral infrastructure uses `x-affinity` for KV-cache reuse (prefix caching).
// Respect explicit caller-provided header values.
if (options?.sessionId && !headers["x-affinity"]) {
headers["x-affinity"] = options.sessionId;
}
if (Object.keys(headers).length > 0) {
requestOptions.headers = headers;
}
return requestOptions;
}
function buildChatPayload(
model: Model<"mistral-conversations">,
context: Context,
messages: Message[],
options?: MistralOptions,
): ChatCompletionStreamRequest {
const payload: ChatCompletionStreamRequest = {
model: model.id,
stream: true,
messages: toChatMessages(messages, model.input.includes("image")),
};
let convertedToolNames: Set<string> | undefined;
if (context.tools?.length) {
const tools = toFunctionTools(context.tools);
convertedToolNames = new Set(tools.map((tool) => tool.function.name));
if (tools.length > 0) {
payload.tools = tools;
}
}
if (options?.temperature !== undefined) {
payload.temperature = options.temperature;
}
if (options?.maxTokens !== undefined) {
payload.maxTokens = options.maxTokens;
}
if (options?.stop !== undefined && options.stop.length > 0) {
payload.stop = options.stop;
}
if (options?.toolChoice) {
const toolChoice = mapToolChoice(options.toolChoice, convertedToolNames);
if (toolChoice) {
payload.toolChoice = toolChoice;
}
}
if (options?.promptMode) {
payload.promptMode = options.promptMode;
}
if (options?.reasoningEffort) {
payload.reasoningEffort = options.reasoningEffort;
}
if (context.systemPrompt) {
payload.messages.unshift({
role: "system",
content: sanitizeSurrogates(stripSystemPromptCacheBoundary(context.systemPrompt)),
});
}
return payload;
}
async function consumeChatStream(
model: Model<"mistral-conversations">,
output: AssistantMessage,
stream: AssistantMessageEventStream,
mistralStream: AsyncIterable<CompletionEvent>,
): Promise<void> {
let currentBlock: TextContent | ThinkingContent | null = null;
const blocks = output.content;
const blockIndex = () => blocks.length - 1;
const toolBlocksByKey = new Map<string, number>();
const finishCurrentBlock = (block?: typeof currentBlock) => {
if (!block) {
return;
}
if (block.type === "text") {
stream.push({
type: "text_end",
contentIndex: blockIndex(),
content: block.text,
partial: output,
});
return;
}
if (block.type === "thinking") {
stream.push({
type: "thinking_end",
contentIndex: blockIndex(),
content: block.thinking,
partial: output,
});
}
};
for await (const event of mistralStream) {
const chunk = event.data;
// Mistral's streamed CompletionChunk carries an id field. Keep the first non-empty one,
// mirroring how OpenAI-style streaming exposes a stable response identifier per stream.
output.responseId ||= chunk.id;
if (chunk.usage) {
output.usage.input = chunk.usage.promptTokens || 0;
output.usage.output = chunk.usage.completionTokens || 0;
output.usage.cacheRead = 0;
output.usage.cacheWrite = 0;
output.usage.totalTokens =
chunk.usage.totalTokens || output.usage.input + output.usage.output;
calculateCost(model, output.usage);
}
const choice = chunk.choices[0];
if (!choice) {
continue;
}
if (choice.finishReason) {
output.stopReason = mapChatStopReason(choice.finishReason);
}
const delta = choice.delta;
if (delta.content !== null && delta.content !== undefined) {
const contentItems = typeof delta.content === "string" ? [delta.content] : delta.content;
for (const item of contentItems) {
if (typeof item === "string") {
const textDelta = sanitizeSurrogates(item);
if (!currentBlock || currentBlock.type !== "text") {
finishCurrentBlock(currentBlock);
currentBlock = { type: "text", text: "" };
output.content.push(currentBlock);
stream.push({ type: "text_start", contentIndex: blockIndex(), partial: output });
}
currentBlock.text += textDelta;
stream.push({
type: "text_delta",
contentIndex: blockIndex(),
delta: textDelta,
partial: output,
});
continue;
}
if (item.type === "thinking") {
const deltaText = item.thinking
.map((part) => ("text" in part ? part.text : ""))
.filter((text) => text.length > 0)
.join("");
const thinkingDelta = sanitizeSurrogates(deltaText);
if (!thinkingDelta) {
continue;
}
if (!currentBlock || currentBlock.type !== "thinking") {
finishCurrentBlock(currentBlock);
currentBlock = { type: "thinking", thinking: "" };
output.content.push(currentBlock);
stream.push({ type: "thinking_start", contentIndex: blockIndex(), partial: output });
}
currentBlock.thinking += thinkingDelta;
stream.push({
type: "thinking_delta",
contentIndex: blockIndex(),
delta: thinkingDelta,
partial: output,
});
continue;
}
if (item.type === "text") {
const textDelta = sanitizeSurrogates(item.text);
if (!currentBlock || currentBlock.type !== "text") {
finishCurrentBlock(currentBlock);
currentBlock = { type: "text", text: "" };
output.content.push(currentBlock);
stream.push({ type: "text_start", contentIndex: blockIndex(), partial: output });
}
currentBlock.text += textDelta;
stream.push({
type: "text_delta",
contentIndex: blockIndex(),
delta: textDelta,
partial: output,
});
}
}
}
const toolCalls = delta.toolCalls || [];
for (const toolCall of toolCalls) {
if (currentBlock) {
finishCurrentBlock(currentBlock);
currentBlock = null;
}
const callId =
toolCall.id && toolCall.id !== "null"
? toolCall.id
: deriveMistralToolCallId(`toolcall:${toolCall.index ?? 0}`, 0);
const key = `${callId}:${toolCall.index || 0}`;
const existingIndex = toolBlocksByKey.get(key);
let block: (ToolCall & { partialArgs?: string }) | undefined;
if (existingIndex !== undefined) {
const existing = output.content[existingIndex];
if (existing?.type === "toolCall") {
block = existing as ToolCall & { partialArgs?: string };
}
}
if (!block) {
block = {
type: "toolCall",
id: callId,
name: toolCall.function.name,
arguments: {},
partialArgs: "",
};
output.content.push(block);
toolBlocksByKey.set(key, output.content.length - 1);
stream.push({
type: "toolcall_start",
contentIndex: output.content.length - 1,
partial: output,
});
}
const argsDelta =
typeof toolCall.function.arguments === "string"
? toolCall.function.arguments
: JSON.stringify(toolCall.function.arguments || {});
block.partialArgs = (block.partialArgs || "") + argsDelta;
block.arguments = parseStreamingJson(block.partialArgs);
stream.push({
type: "toolcall_delta",
contentIndex: toolBlocksByKey.get(key)!,
delta: argsDelta,
partial: output,
});
}
}
finishCurrentBlock(currentBlock);
for (const index of toolBlocksByKey.values()) {
const block = output.content[index];
if (block.type !== "toolCall") {
continue;
}
const toolBlock = block as ToolCall & { partialArgs?: string };
toolBlock.arguments = parseStreamingJson(toolBlock.partialArgs);
// Finalize in-place and strip the scratch buffer so replay only
// carries parsed arguments.
delete toolBlock.partialArgs;
stream.push({
type: "toolcall_end",
contentIndex: index,
toolCall: toolBlock,
partial: output,
});
}
}
function toFunctionTools(tools: Tool[]): Array<FunctionTool & { type: "function" }> {
return tools.flatMap((tool) => {
try {
return {
type: "function",
function: {
name: tool.name,
description: tool.description,
parameters: stripSymbolKeys(tool.parameters) as Record<string, unknown>,
strict: false,
},
};
} catch {
return [];
}
});
}
function stripSymbolKeys(value: unknown): unknown {
if (Array.isArray(value)) {
return value.map((item) => stripSymbolKeys(item));
}
if (value && typeof value === "object") {
const result: Record<string, unknown> = {};
for (const [key, entry] of Object.entries(value)) {
result[key] = stripSymbolKeys(entry);
}
return result;
}
return value;
}
function toChatMessages(
messages: Message[],
supportsImages: boolean,
): ChatCompletionStreamRequestMessage[] {
const result: ChatCompletionStreamRequestMessage[] = [];
for (const msg of messages) {
if (msg.role === "user") {
if (typeof msg.content === "string") {
result.push({ role: "user", content: sanitizeSurrogates(msg.content) });
continue;
}
const hadImages = msg.content.some((item) => item.type === "image");
const content: ContentChunk[] = msg.content
.filter((item) => item.type === "text" || supportsImages)
.map((item) => {
if (item.type === "text") {
return { type: "text", text: sanitizeSurrogates(item.text) };
}
return { type: "image_url", imageUrl: `data:${item.mimeType};base64,${item.data}` };
});
if (content.length > 0) {
result.push({ role: "user", content });
continue;
}
if (hadImages && !supportsImages) {
result.push({ role: "user", content: "(image omitted: model does not support images)" });
}
continue;
}
if (msg.role === "assistant") {
const contentParts: ContentChunk[] = [];
const toolCalls: Array<{
id: string;
type: "function";
function: { name: string; arguments: string };
}> = [];
for (const block of msg.content) {
if (block.type === "text") {
if (block.text.trim().length > 0) {
contentParts.push({ type: "text", text: sanitizeSurrogates(block.text) });
}
continue;
}
if (block.type === "thinking") {
if (block.thinking.trim().length > 0) {
contentParts.push({
type: "thinking",
thinking: [{ type: "text", text: sanitizeSurrogates(block.thinking) }],
});
}
continue;
}
toolCalls.push({
id: block.id,
type: "function",
function: { name: block.name, arguments: JSON.stringify(block.arguments || {}) },
});
}
const assistantMessage: ChatCompletionStreamRequestMessage = { role: "assistant" };
if (contentParts.length > 0) {
assistantMessage.content = contentParts;
}
if (toolCalls.length > 0) {
assistantMessage.toolCalls = toolCalls;
}
if (contentParts.length > 0 || toolCalls.length > 0) {
result.push(assistantMessage);
}
continue;
}
const toolContent: ContentChunk[] = [];
const textResult = extractToolResultText(msg.content);
const mediaPlaceholder = describeToolResultMediaPlaceholder(msg.content);
const hasImages = msg.content.some((part) => part.type === "image");
const toolText = buildToolResultText(
textResult,
mediaPlaceholder,
hasImages,
supportsImages,
msg.isError,
);
toolContent.push({ type: "text", text: toolText });
for (const part of msg.content) {
if (!supportsImages) {
continue;
}
if (part.type !== "image") {
continue;
}
toolContent.push({
type: "image_url",
imageUrl: `data:${part.mimeType};base64,${part.data}`,
});
}
result.push({
role: "tool",
toolCallId: msg.toolCallId,
name: msg.toolName,
content: toolContent,
});
}
return result;
}
function buildToolResultText(
text: string,
mediaPlaceholder: string | undefined,
hasImages: boolean,
supportsImages: boolean,
isError: boolean,
): string {
const trimmed = text.trim();
const errorPrefix = isError ? "[tool error] " : "";
if (trimmed.length > 0) {
const imageSuffix =
hasImages && !supportsImages ? "\n[tool image omitted: model does not support images]" : "";
return `${errorPrefix}${trimmed}${imageSuffix}`;
}
if (mediaPlaceholder) {
if (!hasImages || supportsImages) {
return `${errorPrefix}${mediaPlaceholder}`;
}
const omitted =
mediaPlaceholder === "(see attached media)"
? "(media omitted: model does not support images)"
: "(image omitted: model does not support images)";
return `${errorPrefix}${omitted}`;
}
return isError ? "[tool error] (no tool output)" : "(no tool output)";
}
function usesReasoningEffort(model: Model<"mistral-conversations">): boolean {
return (
model.id === "mistral-small-2603" ||
model.id === "mistral-small-latest" ||
model.id === "mistral-medium-3.5"
);
}
function usesPromptModeReasoning(model: Model<"mistral-conversations">): boolean {
return model.reasoning && !usesReasoningEffort(model);
}
function mapReasoningEffort(
model: Model<"mistral-conversations">,
level: Exclude<SimpleStreamOptions["reasoning"], undefined>,
): MistralReasoningEffort {
return (model.thinkingLevelMap?.[level] ?? "high") as MistralReasoningEffort;
}
function mapToolChoice(
choice: MistralOptions["toolChoice"],
convertedToolNames?: ReadonlySet<string>,
):
| "auto"
| "none"
| "any"
| "required"
| { type: "function"; function: { name: string } }
| undefined {
if (!choice) {
return undefined;
}
if (convertedToolNames && convertedToolNames.size === 0) {
if (choice === "none" || choice === "auto") {
return choice === "none" ? "none" : undefined;
}
throw new Error("Mistral tool_choice requires a tool, but no tools survived schema conversion");
}
if (choice === "auto" || choice === "none" || choice === "any" || choice === "required") {
return choice;
}
const toolName = choice.function.name;
if (convertedToolNames && !convertedToolNames.has(toolName)) {
throw new Error(
`Mistral tool_choice requested unavailable tool "${toolName}" after schema conversion`,
);
}
return {
type: "function",
function: { name: toolName },
};
}
function mapChatStopReason(reason: string | null): StopReason {
if (reason === null) {
return "stop";
}
switch (reason) {
case "stop":
return "stop";
case "length":
case "model_length":
return "length";
case "tool_calls":
return "toolUse";
case "error":
return "error";
default:
return "stop";
}
}

View File

@@ -0,0 +1,953 @@
import { arch, platform, release } from "node:os";
import { zstdDecompressSync } from "node:zlib";
// ChatGPT Responses provider tests cover stream handling and timeout behavior.
import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coercion";
import { afterEach, describe, expect, it, vi } from "vitest";
import type { Context, Model } from "../types.js";
import { SYSTEM_PROMPT_CACHE_BOUNDARY } from "../utils/system-prompt-cache-boundary.js";
import {
closeOpenAICodexWebSocketSessions,
extractOpenAICodexAccountId,
parseSSEForTest,
resetOpenAICodexWebSocketDebugStats,
streamSimpleOpenAICodexResponses,
streamOpenAICodexResponses,
} from "./openai-chatgpt-responses.js";
function createJwt(payload: Record<string, unknown>): string {
const header = Buffer.from(JSON.stringify({ alg: "none", typ: "JWT" })).toString("base64url");
const body = Buffer.from(JSON.stringify(payload)).toString("base64url");
return `${header}.${body}.signature`;
}
function stubTimeoutSignal(timeoutMs: number): void {
vi.spyOn(AbortSignal, "timeout").mockImplementation((actualTimeoutMs) => {
expect(actualTimeoutMs).toBe(timeoutMs);
const controller = new AbortController();
queueMicrotask(() => {
controller.abort(new DOMException("timed out", "TimeoutError"));
});
return controller.signal;
});
}
function stubHangingFetch(timeoutMs: number): void {
stubTimeoutSignal(timeoutMs);
vi.stubGlobal(
"fetch",
vi.fn(
(_input: Parameters<typeof fetch>[0], init?: Parameters<typeof fetch>[1]) =>
new Promise<Response>((_resolve, reject) => {
const signal = init?.signal;
if (!signal) {
reject(new Error("missing abort signal"));
return;
}
const abort = () => {
reject(
signal.reason instanceof Error
? signal.reason
: new DOMException("aborted", "AbortError"),
);
};
if (signal.aborted) {
abort();
return;
}
signal.addEventListener("abort", abort, { once: true });
}),
),
);
}
function completedSseResponse(responseId = "resp_test"): Response {
const event = {
type: "response.completed",
response: {
id: responseId,
status: "completed",
output: [],
usage: { input_tokens: 5, output_tokens: 3, total_tokens: 8 },
},
};
return new Response(`data: ${JSON.stringify(event)}\n\n`, {
status: 200,
headers: { "content-type": "text/event-stream" },
});
}
describe("extractOpenAICodexAccountId", () => {
it("decodes URL-safe base64 JWT payloads", () => {
const accessToken = createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "w_ébé_1fzcswWN6Pi5zL",
},
});
expect(accessToken.split(".")[1]).toContain("_");
expect(extractOpenAICodexAccountId(accessToken)).toBe("w_ébé_1fzcswWN6Pi5zL");
});
it("rejects tokens without a Codex account id", () => {
expect(() => extractOpenAICodexAccountId(createJwt({}))).toThrow(
"Failed to extract accountId from token",
);
});
});
describe("streamOpenAICodexResponses transport", () => {
afterEach(() => {
closeOpenAICodexWebSocketSessions();
vi.restoreAllMocks();
vi.unstubAllGlobals();
vi.useRealTimers();
resetOpenAICodexWebSocketDebugStats();
});
const model = {
id: "gpt-5.5",
name: "GPT-5.5",
api: "openai-chatgpt-responses",
provider: "openai",
baseUrl: "https://chatgpt.test/backend-api",
reasoning: true,
input: ["text"],
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
contextWindow: 128_000,
maxTokens: 16_000,
} satisfies Model<"openai-chatgpt-responses">;
const context = {
messages: [{ role: "user", content: "hi", timestamp: 1 }],
} satisfies Context;
it("builds the first Node request with an OS-specific user agent", async () => {
vi.resetModules();
const freshProvider = await import("./openai-chatgpt-responses.js");
let userAgent: string | null = null;
vi.stubGlobal(
"fetch",
vi.fn(async (_input, init) => {
userAgent = new Headers(init?.headers).get("user-agent");
return completedSseResponse();
}),
);
await freshProvider
.streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": { chatgpt_account_id: "acct-1" },
}),
transport: "sse",
})
.result();
expect(userAgent).toBe(`openclaw (${platform()} ${release()}; ${arch()})`);
});
it("zstd-compresses SSE bodies without overriding an existing encoding", async () => {
const captured: Array<{ body: BodyInit | null | undefined; encoding: string | null }> = [];
vi.stubGlobal(
"fetch",
vi.fn(async (_input, init) => {
captured.push({
body: init?.body,
encoding: new Headers(init?.headers).get("content-encoding"),
});
return completedSseResponse(`resp_${captured.length}`);
}),
);
const apiKey = createJwt({
"https://api.openai.com/auth": { chatgpt_account_id: "acct-1" },
});
await streamOpenAICodexResponses(model, context, { apiKey, transport: "sse" }).result();
await streamOpenAICodexResponses(model, context, {
apiKey,
transport: "sse",
headers: { "content-encoding": "identity" },
}).result();
expect(captured[0]?.encoding).toBe("zstd");
expect(captured[0]?.body).toBeInstanceOf(Uint8Array);
const decoded = JSON.parse(
Buffer.from(zstdDecompressSync(captured[0]?.body as Uint8Array)).toString("utf8"),
) as { model?: string };
expect(decoded.model).toBe(model.id);
expect(captured[1]).toMatchObject({ encoding: "identity", body: expect.any(String) });
});
it("keeps JSON request bodies for custom ChatGPT relays", async () => {
let capturedBody: BodyInit | null | undefined;
let capturedEncoding: string | null = null;
vi.stubGlobal(
"fetch",
vi.fn(async (_input, init) => {
capturedBody = init?.body;
capturedEncoding = new Headers(init?.headers).get("content-encoding");
return completedSseResponse();
}),
);
await streamOpenAICodexResponses(
{ ...model, provider: "custom-relay", baseUrl: "https://relay.test/backend-api" },
context,
{
apiKey: createJwt({
"https://api.openai.com/auth": { chatgpt_account_id: "acct-1" },
}),
transport: "sse",
},
).result();
expect(capturedEncoding).toBeNull();
expect(capturedBody).toEqual(expect.any(String));
expect(JSON.parse(capturedBody as string)).toMatchObject({ model: model.id });
});
it("reconnects once when the websocket connection limit is reached", async () => {
let connections = 0;
class ConnectionLimitWebSocket extends EventTarget {
private readonly limitReached = connections++ === 0;
constructor() {
super();
queueMicrotask(() => this.dispatchEvent(new Event("open")));
}
send(): void {
const event = this.limitReached
? { type: "error", error: { code: "websocket_connection_limit_reached" } }
: {
type: "response.completed",
response: {
id: "resp_ws",
status: "completed",
output: [],
usage: { input_tokens: 5, output_tokens: 3, total_tokens: 8 },
},
};
queueMicrotask(() => {
this.dispatchEvent(Object.assign(new Event("message"), { data: JSON.stringify(event) }));
});
}
close(): void {}
}
const fetchMock = vi.fn();
vi.stubGlobal("WebSocket", ConnectionLimitWebSocket);
vi.stubGlobal("fetch", fetchMock);
const result = await streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": { chatgpt_account_id: "acct-1" },
}),
transport: "websocket",
}).result();
expect(result.stopReason).toBe("stop");
expect(connections).toBe(2);
expect(fetchMock).not.toHaveBeenCalled();
});
it("rotates cached websockets before the backend connection age limit", async () => {
vi.useFakeTimers();
const startedAt = new Date("2026-07-03T00:00:00Z");
vi.setSystemTime(startedAt);
let connections = 0;
const sentConnectionIds: number[] = [];
class AgedWebSocket extends EventTarget {
readonly connectionId = ++connections;
readyState = 1;
constructor() {
super();
queueMicrotask(() => this.dispatchEvent(new Event("open")));
}
send(): void {
sentConnectionIds.push(this.connectionId);
queueMicrotask(() => {
this.dispatchEvent(
Object.assign(new Event("message"), {
data: JSON.stringify({
type: "response.completed",
response: {
id: `resp_${this.connectionId}`,
status: "completed",
output: [],
usage: { input_tokens: 5, output_tokens: 3, total_tokens: 8 },
},
}),
}),
);
});
}
close(): void {
this.readyState = 3;
}
}
vi.stubGlobal("WebSocket", AgedWebSocket);
const apiKey = createJwt({
"https://api.openai.com/auth": { chatgpt_account_id: "acct-1" },
});
const sessionId = "aged-session";
await streamOpenAICodexResponses(model, context, {
apiKey,
sessionId,
transport: "websocket-cached",
}).result();
vi.setSystemTime(new Date(startedAt.getTime() + 56 * 60 * 1000));
await streamOpenAICodexResponses(model, context, {
apiKey,
sessionId,
transport: "websocket-cached",
}).result();
expect(sentConnectionIds).toEqual([1, 2]);
expect(connections).toBe(2);
});
it("preserves max for GPT-5.6 simple Codex Responses requests", async () => {
let capturedPayload: Record<string, unknown> | undefined;
const stream = streamSimpleOpenAICodexResponses(
{
...model,
id: "gpt-5.6-sol",
name: "GPT-5.6 Sol",
contextWindow: 372_000,
thinkingLevelMap: { xhigh: "xhigh", max: "max" },
},
context,
{
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
reasoning: "max",
transport: "sse",
onPayload: (payload) => {
capturedPayload = payload as Record<string, unknown>;
throw new Error("stop after payload");
},
},
);
await stream.result();
expect(capturedPayload).toMatchObject({
reasoning: { effort: "max", summary: "auto" },
});
});
it("does not fall back to SSE when websocket transport is explicit", async () => {
const fetchMock = vi.fn(async () => {
throw new Error("fetch should not run");
});
vi.stubGlobal("fetch", fetchMock);
class FailingWebSocket {
constructor() {
throw new Error("websocket connect failed");
}
send(): void {}
close(): void {}
addEventListener(): void {}
removeEventListener(): void {}
}
vi.stubGlobal("WebSocket", FailingWebSocket);
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
sessionId: "session-explicit-websocket",
transport: "websocket",
});
const result = await stream.result();
expect(fetchMock).not.toHaveBeenCalled();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toContain("websocket connect failed");
});
it("honors timeoutMs for explicit SSE transport requests", async () => {
stubHangingFetch(5);
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
timeoutMs: 5,
transport: "sse",
});
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toContain("Request timed out after 5ms");
});
it("does not replay Responses item ids for store-disabled ChatGPT requests", async () => {
let capturedPayload:
| {
store?: unknown;
input?: Array<Record<string, unknown>>;
}
| undefined;
const stream = streamOpenAICodexResponses(
model,
{
messages: [
{
role: "assistant",
api: "openai-chatgpt-responses",
provider: model.provider,
model: model.id,
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "toolUse",
timestamp: 1,
content: [
{
type: "thinking",
thinking: "Need a tool.",
thinkingSignature: JSON.stringify({
type: "reasoning",
id: "rs_prior",
encrypted_content: "ciphertext",
}),
},
{
type: "text",
text: "Checking.",
textSignature: JSON.stringify({
v: 1,
id: "msg_prior",
phase: "commentary",
}),
},
{
type: "toolCall",
id: "call_abc|fc_prior",
name: "lookup",
arguments: {},
},
],
},
],
},
{
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
transport: "sse",
onPayload: (payload) => {
capturedPayload = payload as typeof capturedPayload;
throw new Error("stop after payload");
},
},
);
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toBe("stop after payload");
expect(capturedPayload?.store).toBe(false);
const reasoningItem = capturedPayload?.input?.find((item) => item.type === "reasoning");
expect(reasoningItem).toMatchObject({
type: "reasoning",
encrypted_content: "ciphertext",
summary: [],
});
expect(reasoningItem).not.toHaveProperty("id");
const messageItem = capturedPayload?.input?.find((item) => item.type === "message");
expect(messageItem).toMatchObject({
type: "message",
phase: "commentary",
});
expect(messageItem).not.toHaveProperty("id");
const functionCall = capturedPayload?.input?.find((item) => item.type === "function_call");
expect(functionCall).toMatchObject({
type: "function_call",
call_id: "call_abc",
});
expect(functionCall).not.toHaveProperty("id");
});
it("omits ChatGPT tool controls when every tool schema is unreadable", async () => {
let capturedPayload: Record<string, unknown> | undefined;
const stream = streamOpenAICodexResponses(
model,
{
...context,
tools: [
{
name: "broken",
description: "Broken tool.",
get parameters(): never {
throw new Error("parameters exploded");
},
},
],
},
{
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
transport: "sse",
onPayload: (payload) => {
capturedPayload = payload as Record<string, unknown>;
throw new Error("stop after payload");
},
},
);
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(capturedPayload).not.toHaveProperty("tools");
expect(capturedPayload).not.toHaveProperty("tool_choice");
expect(capturedPayload).not.toHaveProperty("parallel_tool_calls");
});
it("does not reread an unreadable ChatGPT tool inventory length", async () => {
let capturedPayload: Record<string, unknown> | undefined;
const tools = new Proxy([], {
get(target, property, receiver) {
if (property === "length") {
throw new Error("length exploded");
}
return Reflect.get(target, property, receiver);
},
});
const stream = streamOpenAICodexResponses(model, { ...context, tools } as never, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
transport: "sse",
onPayload: (payload) => {
capturedPayload = payload as Record<string, unknown>;
throw new Error("stop after payload");
},
});
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(capturedPayload).not.toHaveProperty("tools");
expect(capturedPayload).not.toHaveProperty("tool_choice");
expect(capturedPayload).not.toHaveProperty("parallel_tool_calls");
});
it("caps oversized timeoutMs before creating request abort signals", async () => {
stubHangingFetch(MAX_TIMER_TIMEOUT_MS);
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
timeoutMs: Number.MAX_SAFE_INTEGER,
transport: "sse",
});
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toContain(`Request timed out after ${MAX_TIMER_TIMEOUT_MS}ms`);
});
it("honors timeoutMs for default websocket transport requests", async () => {
stubTimeoutSignal(5);
const fetchMock = vi.fn(async () => {
throw new Error("fetch should not run before websocket timeout");
});
class HangingWebSocket {
send = vi.fn();
close = vi.fn();
addEventListener(): void {}
removeEventListener(): void {}
}
vi.stubGlobal("fetch", fetchMock);
vi.stubGlobal("WebSocket", HangingWebSocket);
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
timeoutMs: 5,
});
const result = await stream.result();
expect(fetchMock).not.toHaveBeenCalled();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toContain("Request timed out after 5ms");
});
it("times out default websocket streams when no first event arrives", async () => {
vi.useFakeTimers();
try {
const fetchMock = vi.fn(async () => {
throw new Error("fetch should not run after websocket first-event timeout");
});
const sendMock = vi.fn();
const closeMock = vi.fn();
class OpenNoMessageWebSocket {
send = sendMock;
close = closeMock;
addEventListener(type: string, listener: (event: unknown) => void): void {
if (type === "open") {
queueMicrotask(() => listener({}));
}
}
removeEventListener(): void {}
}
vi.stubGlobal("fetch", fetchMock);
vi.stubGlobal("WebSocket", OpenNoMessageWebSocket);
const onFirstEventTimeout = vi.fn();
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
firstEventTimeoutMs: 5,
onFirstEventTimeout,
} as Parameters<typeof streamOpenAICodexResponses>[2] & {
firstEventTimeoutMs: number;
onFirstEventTimeout: (reason: Error) => void;
});
const resultPromise = stream.result();
await Promise.resolve();
await vi.advanceTimersByTimeAsync(5);
const result = await resultPromise;
expect(fetchMock).not.toHaveBeenCalled();
expect(sendMock).toHaveBeenCalledTimes(1);
expect(closeMock).toHaveBeenCalled();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toMatch(
/responses HTTP stream opened but did not deliver a first SSE event within 5ms/,
);
expect(onFirstEventTimeout).toHaveBeenCalledWith(expect.any(Error));
} finally {
vi.useRealTimers();
}
});
it("does not send websocket payload after timeout fires during connect", async () => {
let timeoutController: AbortController | undefined;
vi.spyOn(AbortSignal, "timeout").mockImplementation((actualTimeoutMs) => {
expect(actualTimeoutMs).toBe(5);
timeoutController = new AbortController();
return timeoutController.signal;
});
const sendMock = vi.fn();
class OpeningThenTimedOutWebSocket {
send = sendMock;
close = vi.fn();
addEventListener(type: string, listener: (event: unknown) => void): void {
if (type === "open") {
queueMicrotask(() => {
listener({});
timeoutController?.abort(new DOMException("timed out", "TimeoutError"));
});
}
}
removeEventListener(): void {}
}
vi.stubGlobal("WebSocket", OpeningThenTimedOutWebSocket);
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
timeoutMs: 5,
});
const result = await stream.result();
expect(sendMock).not.toHaveBeenCalled();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toContain("Request timed out after 5ms");
});
it("strips the internal cache boundary marker from request instructions", async () => {
let capturedPayload: { instructions?: string } | undefined;
const stream = streamOpenAICodexResponses(
model,
{
systemPrompt: `Stable${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic`,
messages: [{ role: "user", content: "hi", timestamp: 1 }],
},
{
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
transport: "sse",
onPayload: (payload) => {
capturedPayload = payload as typeof capturedPayload;
throw new Error("stop after payload");
},
},
);
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(capturedPayload?.instructions).toBe("Stable\nDynamic");
expect(JSON.stringify(capturedPayload)).not.toContain("OPENCLAW_CACHE_BOUNDARY");
});
it("falls back to the default instructions when no system prompt is set", async () => {
let capturedPayload: { instructions?: string } | undefined;
const stream = streamOpenAICodexResponses(
model,
{ messages: [{ role: "user", content: "hi", timestamp: 1 }] },
{
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
transport: "sse",
onPayload: (payload) => {
capturedPayload = payload as typeof capturedPayload;
throw new Error("stop after payload");
},
},
);
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(capturedPayload?.instructions).toBe("You are a helpful assistant.");
});
it("prefers promptCacheKey over sessionId for request cache affinity", async () => {
let payload: unknown;
vi.stubGlobal(
"fetch",
vi.fn(async () => {
throw new Error("usage limit: stop after payload");
}),
);
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
sessionId: "run-session",
promptCacheKey: "stable-cache-key",
transport: "sse",
onPayload: (nextPayload) => {
payload = nextPayload;
},
});
await stream.result();
expect(payload).toMatchObject({ prompt_cache_key: "stable-cache-key" });
});
it.each(["1.5", "0x10"])(
"ignores invalid Retry-After header delay values: %s",
async (retryAfter) => {
const fetchMock = vi
.fn<typeof fetch>()
.mockResolvedValueOnce(
new Response("rate limited", {
status: 429,
headers: { "retry-after": retryAfter },
}),
)
.mockRejectedValueOnce(new Error("usage limit: stop after retry delay"));
vi.stubGlobal("fetch", fetchMock);
const setTimeoutSpy = vi
.spyOn(globalThis, "setTimeout")
.mockImplementation((callback: TimerHandler) => {
if (typeof callback === "function") {
callback();
}
return 0 as unknown as ReturnType<typeof setTimeout>;
});
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
transport: "sse",
});
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(fetchMock).toHaveBeenCalledTimes(2);
expect(setTimeoutSpy).toHaveBeenCalledWith(expect.any(Function), 1000);
},
);
it("caps oversized Retry-After delays before sleeping", async () => {
const fetchMock = vi
.fn<typeof fetch>()
.mockResolvedValueOnce(
new Response("rate limited", {
status: 429,
headers: { "retry-after": String(Number.MAX_SAFE_INTEGER) },
}),
)
.mockRejectedValueOnce(new Error("usage limit: stop after retry delay"));
vi.stubGlobal("fetch", fetchMock);
const setTimeoutSpy = vi
.spyOn(globalThis, "setTimeout")
.mockImplementation((callback: TimerHandler) => {
if (typeof callback === "function") {
callback();
}
return 0 as unknown as ReturnType<typeof setTimeout>;
});
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
transport: "sse",
});
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(fetchMock).toHaveBeenCalledTimes(2);
expect(setTimeoutSpy).toHaveBeenCalledWith(expect.any(Function), MAX_TIMER_TIMEOUT_MS);
});
it("bounds non-OK ChatGPT response bodies before formatting API errors", async () => {
const chunkSize = 1024 * 1024;
const totalChunks = 32;
const chunk = new TextEncoder()
.encode("usage limit ".repeat(Math.ceil(chunkSize / "usage limit ".length)))
.subarray(0, chunkSize);
let pullCount = 0;
let canceled = false;
const overflowing = new ReadableStream<Uint8Array>({
pull(controller) {
pullCount += 1;
if (pullCount > totalChunks) {
controller.close();
return;
}
controller.enqueue(chunk);
},
cancel() {
canceled = true;
},
});
const fetchMock = vi.fn<typeof fetch>().mockResolvedValueOnce(
new Response(overflowing, {
status: 400,
statusText: "Bad Request",
}),
);
vi.stubGlobal("fetch", fetchMock);
const stream = streamOpenAICodexResponses(model, context, {
apiKey: createJwt({
"https://api.openai.com/auth": {
chatgpt_account_id: "acct-1",
},
}),
transport: "sse",
});
const result = await stream.result();
expect(result.stopReason).toBe("error");
expect(result.errorMessage).toContain("usage limit");
expect(result.errorMessage?.length).toBeLessThanOrEqual(16 * 1024);
expect(canceled).toBe(true);
expect(pullCount).toBeGreaterThanOrEqual(1);
expect(pullCount).toBeLessThanOrEqual(3);
expect(fetchMock).toHaveBeenCalledTimes(1);
});
});
describe("parseSSEForTest", () => {
it("bounds streamed OpenAI ChatGPT Responses success bodies without content-length", async () => {
// 1 MiB chunks; cap is 16 MiB so the bounded reader cancels well before
// draining the full 32 MiB advertised body.
const CHUNK = 1024 * 1024;
const TOTAL = 32;
let pullCount = 0;
let cancelReason: unknown;
const overflowing = new ReadableStream<Uint8Array>({
pull(controller) {
pullCount += 1;
if (pullCount > TOTAL) {
controller.close();
return;
}
controller.enqueue(new Uint8Array(CHUNK));
},
cancel(reason) {
cancelReason = reason;
},
});
let caught: Error | null = null;
try {
// parseSSE expects a Response-like; pass the streaming body directly
// through a minimal Response shim that only exposes .body.
const response = { body: overflowing } as unknown as Response;
for await (const event of parseSSEForTest(response)) {
expect(event).toBeDefined();
}
} catch (err) {
caught = err as Error;
}
expect(caught?.message).toMatch(
/OpenAI ChatGPT Responses success body exceeded 16777216 bytes/,
);
expect(cancelReason).toBeInstanceOf(Error);
// 16 MiB + a couple of overshoot pulls, well under 32.
expect(pullCount).toBeGreaterThanOrEqual(17);
expect(pullCount).toBeLessThanOrEqual(20);
});
});

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,14 @@
/** Maximum prompt cache key length accepted by OpenAI-compatible request metadata. */
export const OPENAI_PROMPT_CACHE_KEY_MAX_LENGTH = 64;
/** Truncates a prompt cache key by Unicode code point count. */
export function clampOpenAIPromptCacheKey(key: string | undefined): string | undefined {
if (key === undefined) {
return undefined;
}
const chars = Array.from(key);
if (chars.length <= OPENAI_PROMPT_CACHE_KEY_MAX_LENGTH) {
return key;
}
return chars.slice(0, OPENAI_PROMPT_CACHE_KEY_MAX_LENGTH).join("");
}

View File

@@ -0,0 +1,114 @@
// Verifies model-specific OpenAI reasoning-effort normalization and disablement.
import { describe, expect, it } from "vitest";
import {
resolveOpenAIReasoningEffortForModel,
resolveOpenAISupportedReasoningEfforts,
} from "./openai-reasoning-effort.js";
describe("OpenAI reasoning effort support", () => {
it("preserves max for the GPT-5.6 series", () => {
const sol = { provider: "openai", id: "gpt-5.6-sol" };
const terra = { provider: "openai", id: "gpt-5.6-terra" };
const luna = { provider: "openai", id: "gpt-5.6-luna" };
expect(resolveOpenAIReasoningEffortForModel({ model: sol, effort: "max" })).toBe("max");
expect(resolveOpenAIReasoningEffortForModel({ model: terra, effort: "max" })).toBe("max");
expect(resolveOpenAIReasoningEffortForModel({ model: luna, effort: "max" })).toBe("max");
});
it.each([
{ provider: "openai", id: "gpt-5.5" },
{ provider: "openai", id: "gpt-5.5" },
])("preserves xhigh for $provider/$id", (model) => {
expect(resolveOpenAISupportedReasoningEfforts(model)).toContain("xhigh");
expect(resolveOpenAIReasoningEffortForModel({ model, effort: "xhigh" })).toBe("xhigh");
});
it("preserves reasoning_effort metadata for gpt-5.4-mini in Chat Completions", () => {
const model = { provider: "openai", id: "gpt-5.4-mini", api: "openai-completions" };
expect(resolveOpenAISupportedReasoningEfforts(model)).toContain("medium");
expect(resolveOpenAIReasoningEffortForModel({ model, effort: "medium" })).toBe("medium");
});
it("preserves reasoning_effort for gpt-5.4-mini in Responses", () => {
const model = { provider: "openai", id: "gpt-5.4-mini", api: "openai-responses" };
expect(resolveOpenAISupportedReasoningEfforts(model)).toContain("medium");
expect(resolveOpenAIReasoningEffortForModel({ model, effort: "medium" })).toBe("medium");
});
it("does not downgrade xhigh when model compat metadata declares it explicitly", () => {
const model = {
provider: "openai",
id: "gpt-5.5",
compat: {
supportedReasoningEfforts: ["low", "medium", "high", "xhigh"],
},
};
expect(resolveOpenAIReasoningEffortForModel({ model, effort: "xhigh" })).toBe("xhigh");
});
it("allows provider-native compat values when explicitly declared", () => {
// Some OpenAI-compatible providers expose their own reasoning effort labels.
const model = {
provider: "groq",
id: "qwen/qwen3-32b",
compat: {
supportedReasoningEfforts: ["none", "default"],
reasoningEffortMap: {
off: "none",
low: "default",
medium: "default",
high: "default",
},
},
};
expect(resolveOpenAISupportedReasoningEfforts(model)).toEqual(["none", "default"]);
expect(
resolveOpenAIReasoningEffortForModel({
model,
effort: "medium",
fallbackMap: model.compat.reasoningEffortMap,
}),
).toBe("default");
expect(
resolveOpenAIReasoningEffortForModel({
model,
effort: "off",
fallbackMap: model.compat.reasoningEffortMap,
}),
).toBe("none");
});
it("omits unsupported disabled reasoning instead of falling back to enabled effort", () => {
expect(
resolveOpenAIReasoningEffortForModel({
model: { provider: "groq", id: "openai/gpt-oss-120b" },
effort: "off",
}),
).toBeUndefined();
});
it("honors compat metadata that disables reasoning effort payloads", () => {
const model = {
provider: "xai",
id: "grok-4.20-beta-latest-reasoning",
compat: { supportsReasoningEffort: false },
};
expect(resolveOpenAISupportedReasoningEfforts(model)).toEqual([]);
expect(resolveOpenAIReasoningEffortForModel({ model, effort: "high" })).toBeUndefined();
});
it("does not turn disabled reasoning into a fallback effort when compat omits none", () => {
const model = {
provider: "xai",
id: "grok-4.3",
compat: { supportedReasoningEfforts: ["low", "medium", "high"] },
};
expect(resolveOpenAIReasoningEffortForModel({ model, effort: "none" })).toBeUndefined();
expect(resolveOpenAIReasoningEffortForModel({ model, effort: "high" })).toBe("high");
});
});

View File

@@ -0,0 +1,165 @@
/**
* OpenAI-compatible reasoning-effort normalization. Different GPT families
* expose different accepted effort enums, so callers map requested values here
* before constructing provider payloads.
*/
import { normalizeLowercaseStringOrEmpty } from "@openclaw/normalization-core/string-coerce";
import {
normalizeStringEntries,
uniqueStrings,
} from "@openclaw/normalization-core/string-normalization";
export type OpenAIReasoningEffort =
| "none"
| "minimal"
| "low"
| "medium"
| "high"
| "xhigh"
| "max";
export type OpenAIApiReasoningEffort = OpenAIReasoningEffort | (string & {});
type OpenAIReasoningModel = {
provider?: unknown;
id?: unknown;
name?: unknown;
api?: unknown;
baseUrl?: unknown;
compat?: unknown;
};
const GPT_5_REASONING_EFFORTS = ["minimal", "low", "medium", "high"] as const;
const GPT_51_REASONING_EFFORTS = ["none", "low", "medium", "high"] as const;
const GPT_52_REASONING_EFFORTS = ["none", "low", "medium", "high", "xhigh"] as const;
const GPT_56_REASONING_EFFORTS = ["low", "medium", "high", "xhigh", "max"] as const;
const GPT_CODEX_REASONING_EFFORTS = ["low", "medium", "high", "xhigh"] as const;
const GPT_PRO_REASONING_EFFORTS = ["medium", "high", "xhigh"] as const;
const GPT_5_PRO_REASONING_EFFORTS = ["high"] as const;
const GPT_51_CODEX_MAX_REASONING_EFFORTS = ["none", "medium", "high", "xhigh"] as const;
const GPT_51_CODEX_MINI_REASONING_EFFORTS = ["medium"] as const;
const GENERIC_REASONING_EFFORTS = ["low", "medium", "high"] as const;
function normalizeModelId(id: string | null | undefined): string {
return normalizeLowercaseStringOrEmpty(id ?? "").replace(/-\d{4}-\d{2}-\d{2}$/u, "");
}
/** Return whether a model is the GPT-5.4 mini family. */
export function isOpenAIGpt54MiniModel(model: OpenAIReasoningModel): boolean {
const id = normalizeModelId(typeof model.id === "string" ? model.id : undefined);
return /^gpt-5\.4-mini(?:-|$)/u.test(id);
}
/** Return whether a model is the GPT-5.5 family. */
export function isOpenAIGpt55Model(model: OpenAIReasoningModel): boolean {
const id = normalizeModelId(typeof model.id === "string" ? model.id : undefined);
const name = normalizeModelId(typeof model.name === "string" ? model.name : undefined);
return /^gpt-5\.5(?:-|$)/u.test(id) || /^gpt-5\.5(?:\s|\(|-|$)/u.test(name);
}
/** Normalize user-facing reasoning effort names to API effort names. */
export function normalizeOpenAIReasoningEffort(effort: string): string {
return effort === "minimal" ? "minimal" : effort;
}
function readCompatReasoningEfforts(compat: unknown): OpenAIApiReasoningEffort[] | undefined {
if (!compat || typeof compat !== "object") {
return undefined;
}
if ((compat as { supportsReasoningEffort?: unknown }).supportsReasoningEffort === false) {
return [];
}
const raw = (compat as { supportedReasoningEfforts?: unknown }).supportedReasoningEfforts;
if (!Array.isArray(raw)) {
return undefined;
}
const supported = uniqueStrings(
normalizeStringEntries(raw.filter((value) => typeof value === "string")),
);
return supported.length > 0 ? supported : undefined;
}
function isDisabledReasoningEffort(effort: string): boolean {
return effort === "none" || effort === "off";
}
/** Resolve the reasoning efforts accepted by a specific OpenAI-compatible model. */
export function resolveOpenAISupportedReasoningEfforts(
model: OpenAIReasoningModel,
): readonly OpenAIApiReasoningEffort[] {
const compatEfforts = readCompatReasoningEfforts(model.compat);
if (compatEfforts) {
return compatEfforts;
}
const id = normalizeModelId(typeof model.id === "string" ? model.id : undefined);
if (/^gpt-5\.6(?:-|$)/u.test(id)) {
return GPT_56_REASONING_EFFORTS;
}
if (id === "gpt-5.1-codex-mini") {
return GPT_51_CODEX_MINI_REASONING_EFFORTS;
}
if (id === "gpt-5.1-codex-max") {
return GPT_51_CODEX_MAX_REASONING_EFFORTS;
}
if (/^gpt-5(?:\.\d+)?-codex(?:-|$)/u.test(id)) {
return GPT_CODEX_REASONING_EFFORTS;
}
if (id === "gpt-5-pro") {
return GPT_5_PRO_REASONING_EFFORTS;
}
if (/^gpt-5\.[2-9](?:\.\d+)?-pro(?:-|$)/u.test(id)) {
return GPT_PRO_REASONING_EFFORTS;
}
if (/^gpt-5\.[2-9](?:\.\d+)?(?:-|$)/u.test(id)) {
return GPT_52_REASONING_EFFORTS;
}
if (/^gpt-5\.1(?:-|$)/u.test(id)) {
return GPT_51_REASONING_EFFORTS;
}
if (/^gpt-5(?:-|$)/u.test(id)) {
return GPT_5_REASONING_EFFORTS;
}
return GENERIC_REASONING_EFFORTS;
}
/** Return whether a model accepts a requested reasoning effort. */
export function supportsOpenAIReasoningEffort(
model: OpenAIReasoningModel,
effort: string,
): boolean {
return resolveOpenAISupportedReasoningEfforts(model).includes(
normalizeOpenAIReasoningEffort(effort) as OpenAIApiReasoningEffort,
);
}
/** Resolve a requested reasoning effort to the closest value supported by the model. */
export function resolveOpenAIReasoningEffortForModel(params: {
model: OpenAIReasoningModel;
effort: string;
fallbackMap?: Record<string, string>;
}): OpenAIApiReasoningEffort | undefined {
const requested = normalizeOpenAIReasoningEffort(params.effort);
const mapped = params.fallbackMap?.[requested] ?? requested;
const normalized = normalizeOpenAIReasoningEffort(mapped);
const supported = resolveOpenAISupportedReasoningEfforts(params.model);
if (supported.includes(normalized as OpenAIApiReasoningEffort)) {
return normalized as OpenAIApiReasoningEffort;
}
if (isDisabledReasoningEffort(requested) || isDisabledReasoningEffort(normalized)) {
return undefined;
}
if (requested === "minimal" && supported.includes("low")) {
return "low";
}
if ((requested === "minimal" || requested === "low") && supported.includes("medium")) {
return "medium";
}
if (requested === "xhigh" && supported.includes("high")) {
return "high";
}
if (requested === "max" && supported.includes("xhigh")) {
return "xhigh";
}
return supported.find((effort) => effort !== "none");
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,78 @@
export const OPENAI_RESPONSES_OUTPUT_TEXT_CONTENT_PART_TYPE = "output_text";
export const AZURE_RESPONSES_TEXT_CONTENT_PART_TYPE = "text";
export const OPENAI_RESPONSES_OUTPUT_TEXT_DELTA_EVENT_TYPE = "response.output_text.delta";
export const AZURE_RESPONSES_TEXT_DELTA_EVENT_TYPE = "response.text.delta";
export type ResponsesTextContentPartType =
| typeof OPENAI_RESPONSES_OUTPUT_TEXT_CONTENT_PART_TYPE
| typeof AZURE_RESPONSES_TEXT_CONTENT_PART_TYPE;
export type ResponsesTextDeltaEventType =
| typeof OPENAI_RESPONSES_OUTPUT_TEXT_DELTA_EVENT_TYPE
| typeof AZURE_RESPONSES_TEXT_DELTA_EVENT_TYPE;
export type AzureResponsesTextContentPart = {
type: typeof AZURE_RESPONSES_TEXT_CONTENT_PART_TYPE;
text: string;
};
export type AzureResponsesTextDeltaEvent = {
type: typeof AZURE_RESPONSES_TEXT_DELTA_EVENT_TYPE;
delta: string;
};
export function isResponsesTextContentPartType(
type: unknown,
): type is ResponsesTextContentPartType {
return (
type === OPENAI_RESPONSES_OUTPUT_TEXT_CONTENT_PART_TYPE ||
type === AZURE_RESPONSES_TEXT_CONTENT_PART_TYPE
);
}
export function isResponsesTextDeltaEventType(type: unknown): type is ResponsesTextDeltaEventType {
return (
type === OPENAI_RESPONSES_OUTPUT_TEXT_DELTA_EVENT_TYPE ||
type === AZURE_RESPONSES_TEXT_DELTA_EVENT_TYPE
);
}
export function isAzureResponsesTextDeltaEventType(
type: unknown,
): type is typeof AZURE_RESPONSES_TEXT_DELTA_EVENT_TYPE {
return type === AZURE_RESPONSES_TEXT_DELTA_EVENT_TYPE;
}
export function isAzureResponsesTextDeltaEvent(event: {
type?: unknown;
delta?: unknown;
}): event is AzureResponsesTextDeltaEvent {
return isAzureResponsesTextDeltaEventType(event.type) && typeof event.delta === "string";
}
export type ResponsesMessageSnapshotCollapse = { kind: "extend"; text: string } | { kind: "keep" };
// Some openai-responses providers re-emit the assistant message as cumulative
// snapshot items — each a strict prefix-superset of the previous one — instead
// of one final message item. A same-phase strict extension replaces the prior
// text block, or the visible reply repeats once per snapshot (#91959).
// Extension-only on purpose: equal or shrinking adjacent items stay distinct
// (the Responses protocol allows multiple message items per response), so a
// false positive can only merge rendering — it can never lose text.
// `prior` must be the immediately preceding output item: collapsing across
// reasoning/function_call boundaries would drop real post-tool messages and
// orphan reasoning items, which OpenAI replay rejects.
export function resolveResponsesMessageSnapshotCollapse(params: {
prior: { text: string; phase: string | undefined } | null;
nextText: string;
nextPhase: string | undefined;
}): ResponsesMessageSnapshotCollapse {
const { prior, nextText } = params;
if (!prior?.text || !nextText || prior.phase !== params.nextPhase) {
return { kind: "keep" };
}
if (nextText.length > prior.text.length && nextText.startsWith(prior.text)) {
return { kind: "extend", text: nextText };
}
return { kind: "keep" };
}

View File

@@ -0,0 +1,171 @@
// OpenAI Responses tool helpers convert runtime tools to Responses API schemas.
import { createHash } from "node:crypto";
import type { Tool as OpenAITool } from "openai/resources/responses/responses.js";
import { getAiTransportHost } from "../host.js";
import type { Model, Tool } from "../types.js";
import { projectOpenAITools, type OpenAIToolProjection } from "./openai-tool-projection.js";
import {
findOpenAIStrictToolProjectionDiagnostics,
normalizeOpenAIStrictToolParameters,
resolveOpenAIProjectedToolsStrictToolFlag,
} from "./openai-tool-schema.js";
/** Options for converting internal tool schemas to OpenAI Responses function tools. */
export interface ConvertResponsesToolsOptions {
strict?: boolean | null;
model?: Model;
supportsStrictMode?: boolean;
}
type OpenAIToolSchemaCompat = Parameters<typeof normalizeOpenAIStrictToolParameters>[2];
type ResponsesFunctionTool = {
type: "function";
name: string;
description?: string;
parameters: Record<string, unknown>;
strict?: boolean | null;
};
export type ConvertedResponsesTools = {
projection: OpenAIToolProjection;
tools: OpenAITool[];
};
// Converts OpenClaw tool schemas to OpenAI Responses tools, including strict-mode compatibility.
const LOG_SUBSYSTEM = "llm/openai-responses";
const MAX_STRICT_TOOL_DOWNGRADE_DIAGNOSTIC_KEYS = 64;
const loggedStrictToolDowngradeDiagnosticKeys = new Set<string>();
/** Converts tools to deterministic OpenAI Responses function tool definitions. */
export function convertResponsesTools(
tools: Tool[],
options?: ConvertResponsesToolsOptions,
): OpenAITool[] {
return convertResponsesToolPayload(tools, options).tools;
}
/** Converts and returns the projection used to reconcile tool choices. */
export function convertResponsesToolPayload(
tools: Tool[],
options?: ConvertResponsesToolsOptions,
): ConvertedResponsesTools {
const projection = projectOpenAITools(tools);
const strictSetting = resolveResponsesStrictToolSetting(options);
const strict = resolveResponsesStrictToolFlag(projection, strictSetting, options?.model);
// Sort tools before request construction so prompt-cache bytes stay deterministic.
const convertedTools = sortResponsesToolsByName(projection.tools).map((tool) => {
const result: ResponsesFunctionTool = {
type: "function",
name: tool.name,
description: tool.description,
parameters: normalizeOpenAIStrictToolParameters(
tool.parameters,
strict === true,
options?.model?.compat as OpenAIToolSchemaCompat,
),
};
if (strict !== undefined) {
result.strict = strict;
}
return result as OpenAITool;
});
return { projection, tools: convertedTools };
}
function resolveResponsesStrictToolSetting(
options: ConvertResponsesToolsOptions | undefined,
): boolean | null | undefined {
if (options?.strict !== undefined) {
return options.strict;
}
if (options?.model) {
return getAiTransportHost().resolveOpenAIStrictToolSetting(options.model, {
transport: "stream",
supportsStrictMode: options.supportsStrictMode,
});
}
return false;
}
function resolveResponsesStrictToolFlag(
projection: OpenAIToolProjection,
strictSetting: boolean | null | undefined,
model: Model | undefined,
): boolean | undefined {
const strict = resolveOpenAIProjectedToolsStrictToolFlag(projection, strictSetting);
if (strictSetting === true && strict === false && model) {
getAiTransportHost().logDebug(LOG_SUBSYSTEM, () => {
const diagnostics = findOpenAIStrictToolProjectionDiagnostics(projection);
if (!shouldLogStrictToolDowngradeDiagnostic(diagnostics, model)) {
return null;
}
const sample = diagnostics.slice(0, 5).map((entry) => ({
tool: entry.toolName ?? `tool[${entry.toolIndex}]`,
violations: entry.violations.slice(0, 8),
}));
return {
message:
`OpenAI responses tool schema strict mode downgraded to strict=false for ` +
`${model.provider ?? "unknown"}/${model.id ?? "unknown"} because ` +
`${diagnostics.length} tool schema(s) are not strict-compatible`,
data: {
provider: model.provider,
model: model.id,
incompatibleToolCount: diagnostics.length,
sample,
},
};
});
}
return strict;
}
function shouldLogStrictToolDowngradeDiagnostic(
diagnostics: ReturnType<typeof findOpenAIStrictToolProjectionDiagnostics>,
model: Model,
): boolean {
// Strict downgrade diagnostics can repeat per turn; hash details and cap memory.
const key = createHash("sha256")
.update(
JSON.stringify({
provider: model.provider,
model: model.id,
diagnostics: diagnostics.map((entry) => ({
toolIndex: entry.toolIndex,
toolName: entry.toolName ?? null,
violations: entry.violations,
})),
}),
)
.digest("hex");
if (loggedStrictToolDowngradeDiagnosticKeys.has(key)) {
return false;
}
if (loggedStrictToolDowngradeDiagnosticKeys.size >= MAX_STRICT_TOOL_DOWNGRADE_DIAGNOSTIC_KEYS) {
loggedStrictToolDowngradeDiagnosticKeys.clear();
}
loggedStrictToolDowngradeDiagnosticKeys.add(key);
return true;
}
function compareToolText(left: string | undefined, right: string | undefined): number {
const leftText = left ?? "";
const rightText = right ?? "";
if (leftText < rightText) {
return -1;
}
if (leftText > rightText) {
return 1;
}
return 0;
}
function sortResponsesToolsByName<T extends { name?: string; description?: string }>(
tools: readonly T[],
): T[] {
return tools.toSorted(
(left, right) =>
compareToolText(left.name, right.name) ||
compareToolText(left.description, right.description),
);
}

View File

@@ -0,0 +1,250 @@
// OpenAI Responses provider adapts OpenAI response streams to the agent runtime.
import OpenAI from "openai";
import type { ResponseCreateParamsStreaming } from "openai/resources/responses/responses.js";
import { getEnvApiKey } from "../env-api-keys.js";
import type {
CacheRetention,
Context,
Model,
OpenAIResponsesCompat,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
Usage,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
import { resolveCacheRetention } from "./cache-retention.js";
import { isCloudflareProvider, resolveCloudflareBaseUrl } from "./cloudflare.js";
import { buildCopilotDynamicHeaders, hasCopilotVisionInput } from "./github-copilot-headers.js";
import { clampOpenAIPromptCacheKey } from "./openai-prompt-cache.js";
import {
applyCommonResponsesParams,
convertResponsesMessages,
createResponsesAssistantOutput,
resolveResponsesReasoningEffort,
runResponsesStreamLifecycle,
} from "./openai-responses-shared.js";
import { buildBaseOptions } from "./simple-options.js";
const OPENAI_TOOL_CALL_PROVIDERS = new Set(["openai", "opencode"]);
function getCompat(model: Model<"openai-responses">): Required<OpenAIResponsesCompat> {
return {
sendSessionIdHeader: model.compat?.sendSessionIdHeader ?? true,
supportsLongCacheRetention: model.compat?.supportsLongCacheRetention ?? true,
};
}
function getPromptCacheRetention(
compat: Required<OpenAIResponsesCompat>,
cacheRetention: CacheRetention,
): "24h" | undefined {
return cacheRetention === "long" && compat.supportsLongCacheRetention ? "24h" : undefined;
}
function formatOpenAIResponsesError(error: unknown): string {
if (error instanceof Error) {
const status = (error as Error & { status?: unknown }).status;
const statusCode = typeof status === "number" ? status : undefined;
if (statusCode !== undefined) {
return `OpenAI API error (${statusCode}): ${error.message}`;
}
return error.message;
}
try {
return JSON.stringify(error);
} catch {
return String(error);
}
}
// OpenAI Responses-specific options
export interface OpenAIResponsesOptions extends StreamOptions {
reasoningEffort?: "minimal" | "low" | "medium" | "high" | "xhigh" | "max";
reasoningSummary?: "auto" | "detailed" | "concise" | null;
replayResponsesItemIds?: boolean;
serviceTier?: ResponseCreateParamsStreaming["service_tier"];
}
type OpenAIResponsesReplayOptions = SimpleStreamOptions & {
replayResponsesItemIds?: boolean;
};
/**
* Generate function for OpenAI Responses API
*/
export const streamOpenAIResponses: StreamFunction<"openai-responses", OpenAIResponsesOptions> = (
model: Model<"openai-responses">,
context: Context,
options?: OpenAIResponsesOptions,
) => {
const stream = new AssistantMessageEventStream();
const output = createResponsesAssistantOutput(model);
// Start async processing
void runResponsesStreamLifecycle({
stream,
model,
output,
options,
createClient: () => {
const apiKey = options?.apiKey || getEnvApiKey(model.provider) || "";
const cacheRetention = resolveCacheRetention(options?.cacheRetention);
const cacheSessionId = cacheRetention === "none" ? undefined : options?.sessionId;
return createClient(model, context, apiKey, options?.headers, cacheSessionId);
},
buildParams: () => buildParams(model, context, options),
processStreamOptions: {
serviceTier: options?.serviceTier,
applyServiceTierPricing: (usage, serviceTier) =>
applyServiceTierPricing(usage, serviceTier, model),
},
formatError: formatOpenAIResponsesError,
});
return stream;
};
export const streamSimpleOpenAIResponses: StreamFunction<
"openai-responses",
SimpleStreamOptions
> = (model: Model<"openai-responses">, context: Context, options?: SimpleStreamOptions) => {
const apiKey = options?.apiKey || getEnvApiKey(model.provider);
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const base = buildBaseOptions(model, options, apiKey);
return streamOpenAIResponses(model, context, {
...base,
reasoningEffort: resolveResponsesReasoningEffort(model, options?.reasoning),
replayResponsesItemIds: (options as OpenAIResponsesReplayOptions | undefined)
?.replayResponsesItemIds,
} satisfies OpenAIResponsesOptions);
};
function createClient(
model: Model<"openai-responses">,
context: Context,
apiKey?: string,
optionsHeaders?: Record<string, string>,
sessionId?: string,
) {
if (!apiKey) {
throw new Error(`No API key for provider: ${model.provider}`);
}
const compat = getCompat(model);
const headers = { ...model.headers };
if (model.provider === "github-copilot") {
const hasImages = hasCopilotVisionInput(context.messages);
const copilotHeaders = buildCopilotDynamicHeaders({
messages: context.messages,
hasImages,
});
Object.assign(headers, copilotHeaders);
}
if (sessionId) {
if (compat.sendSessionIdHeader) {
headers.session_id = sessionId;
}
headers["x-client-request-id"] = sessionId;
}
// Merge options headers last so they can override defaults
if (optionsHeaders) {
Object.assign(headers, optionsHeaders);
}
const defaultHeaders =
model.provider === "cloudflare-ai-gateway"
? {
...headers,
Authorization: headers.Authorization ?? null,
"cf-aig-authorization": `Bearer ${apiKey}`,
}
: headers;
return new OpenAI({
apiKey,
baseURL: isCloudflareProvider(model.provider) ? resolveCloudflareBaseUrl(model) : model.baseUrl,
dangerouslyAllowBrowser: true,
defaultHeaders,
});
}
function buildParams(
model: Model<"openai-responses">,
context: Context,
options?: OpenAIResponsesOptions,
) {
const messages = convertResponsesMessages(model, context, OPENAI_TOOL_CALL_PROVIDERS, {
replayResponsesItemIds: options?.replayResponsesItemIds ?? false,
});
const cacheRetention = resolveCacheRetention(options?.cacheRetention);
const compat = getCompat(model);
const params: ResponseCreateParamsStreaming = {
model: model.id,
input: messages,
stream: true,
prompt_cache_key:
cacheRetention === "none"
? undefined
: clampOpenAIPromptCacheKey(options?.promptCacheKey ?? options?.sessionId),
prompt_cache_retention: getPromptCacheRetention(compat, cacheRetention),
store: false,
};
if (options?.maxTokens) {
params.max_output_tokens = options?.maxTokens;
}
if (options?.temperature !== undefined) {
params.temperature = options?.temperature;
}
if (options?.serviceTier !== undefined) {
params.service_tier = options.serviceTier;
}
applyCommonResponsesParams(params, model, context, options, {
setDefaultReasoningOff: model.provider !== "github-copilot",
});
return params;
}
function getServiceTierCostMultiplier(
model: Pick<Model<"openai-responses">, "id">,
serviceTier: ResponseCreateParamsStreaming["service_tier"] | undefined,
): number {
switch (serviceTier) {
case "flex":
return 0.5;
case "priority":
return model.id === "gpt-5.5" ? 2.5 : 2;
default:
return 1;
}
}
function applyServiceTierPricing(
usage: Usage,
serviceTier: ResponseCreateParamsStreaming["service_tier"] | undefined,
model: Pick<Model<"openai-responses">, "id">,
) {
const multiplier = getServiceTierCostMultiplier(model, serviceTier);
if (multiplier === 1) {
return;
}
usage.cost.input *= multiplier;
usage.cost.output *= multiplier;
usage.cost.cacheRead *= multiplier;
usage.cost.cacheWrite *= multiplier;
usage.cost.total =
usage.cost.input + usage.cost.output + usage.cost.cacheRead + usage.cost.cacheWrite;
}

View File

@@ -0,0 +1,36 @@
import { describe, expect, it } from "vitest";
import { mapOpenAIStopReason } from "./openai-stop-reason.js";
describe("mapOpenAIStopReason", () => {
it.each([
["stop", { stopReason: "stop" }],
["end", { stopReason: "stop" }],
["length", { stopReason: "length" }],
["function_call", { stopReason: "toolUse" }],
["tool_calls", { stopReason: "toolUse" }],
[null, { stopReason: "stop" }],
] as const)("maps %s", (reason, expected) => {
expect(mapOpenAIStopReason(reason)).toEqual(expected);
});
it("keeps singular tool_call opt-in", () => {
expect(mapOpenAIStopReason("tool_call")).toEqual({
stopReason: "error",
errorMessage: "Provider finish_reason: tool_call",
});
expect(mapOpenAIStopReason("tool_call", { allowSingularToolCall: true })).toEqual({
stopReason: "toolUse",
});
});
it("surfaces provider errors and unknown reasons", () => {
expect(mapOpenAIStopReason("content_filter")).toEqual({
stopReason: "error",
errorMessage: "Provider finish_reason: content_filter",
});
expect(mapOpenAIStopReason("unexpected")).toEqual({
stopReason: "error",
errorMessage: "Provider finish_reason: unexpected",
});
});
});

View File

@@ -0,0 +1,40 @@
import type { StopReason } from "../types.js";
export type OpenAIStopReasonResult = {
stopReason: StopReason;
errorMessage?: string;
};
export function mapOpenAIStopReason(
reason: string | null,
options?: { allowSingularToolCall?: boolean },
): OpenAIStopReasonResult {
if (reason === null) {
return { stopReason: "stop" };
}
switch (reason) {
case "stop":
case "end":
return { stopReason: "stop" };
case "length":
return { stopReason: "length" };
case "function_call":
case "tool_calls":
return { stopReason: "toolUse" };
case "tool_call":
if (options?.allowSingularToolCall) {
return { stopReason: "toolUse" };
}
break;
case "content_filter":
return { stopReason: "error", errorMessage: "Provider finish_reason: content_filter" };
case "network_error":
return { stopReason: "error", errorMessage: "Provider finish_reason: network_error" };
}
return {
stopReason: "error",
errorMessage: `Provider finish_reason: ${reason}`,
};
}

View File

@@ -0,0 +1,283 @@
import { describe, expect, it } from "vitest";
import {
projectOpenAITools,
reconcileOpenAICompletionsToolChoice,
reconcileOpenAIResponsesToolChoice,
} from "./openai-tool-projection.js";
describe("OpenAI tool projection", () => {
it("keeps healthy tools when sibling descriptors or schemas are unreadable", () => {
const projection = projectOpenAITools([
{
get name(): never {
throw new Error("name exploded");
},
parameters: {},
},
{
name: "bad_schema",
parameters: {
type: "object",
get properties(): never {
throw new Error("properties exploded");
},
},
},
{
name: "lookup",
get description(): never {
throw new Error("description exploded");
},
parameters: { type: "object", properties: {} },
},
]);
expect(projection.tools).toEqual([
{
toolIndex: 2,
name: "lookup",
parameters: { type: "object", properties: {} },
},
]);
expect(projection.diagnostics).toHaveLength(2);
});
it("reads optional descriptions once before projecting them", () => {
let descriptionReads = 0;
const projection = projectOpenAITools([
{
name: "lookup",
get description() {
descriptionReads += 1;
return descriptionReads === 1 ? "Lookup" : Symbol("invalid");
},
parameters: {},
},
]);
expect(projection.tools[0]?.description).toBe("Lookup");
expect(descriptionReads).toBe(1);
});
it("keeps a healthy pinned Responses function choice", () => {
const projection = projectOpenAITools([{ name: "lookup", parameters: {} }]);
expect(
reconcileOpenAIResponsesToolChoice({ type: "function", name: "lookup" }, projection),
).toEqual({ type: "function", name: "lookup" });
});
it("materializes pinned function choices after one name read", () => {
const projection = projectOpenAITools([{ name: "lookup", parameters: {} }]);
let responsesNameReads = 0;
let completionsNameReads = 0;
const responsesChoice = {
type: "function",
get name() {
responsesNameReads += 1;
return responsesNameReads === 1 ? "lookup" : "broken";
},
};
const completionsChoice = {
type: "function",
function: {
get name() {
completionsNameReads += 1;
return completionsNameReads === 1 ? "lookup" : "broken";
},
},
};
expect(reconcileOpenAIResponsesToolChoice(responsesChoice as never, projection)).toEqual({
type: "function",
name: "lookup",
});
expect(reconcileOpenAICompletionsToolChoice(completionsChoice as never, projection)).toEqual({
type: "function",
function: { name: "lookup" },
});
expect(responsesNameReads).toBe(1);
expect(completionsNameReads).toBe(1);
});
it("normalizes omitted parameters to an empty schema", () => {
expect(projectOpenAITools([{ name: "lookup", parameters: undefined }]).tools).toEqual([
{
toolIndex: 0,
name: "lookup",
parameters: {},
},
]);
});
it("quarantines OpenAI tools with unsupported dynamic schema references", () => {
const projection = projectOpenAITools([
{
name: "dynamic",
parameters: {
type: "object",
properties: {
value: { $dynamicRef: "#value" },
},
},
},
]);
expect(projection.tools).toEqual([]);
expect(projection.diagnostics).toEqual([
{
toolIndex: 0,
toolName: "dynamic",
violations: ["dynamic.parameters.properties.value.$dynamicRef"],
},
]);
});
it("quarantines an inventory with an unreadable length", () => {
const tools = new Proxy([], {
get(target, property, receiver) {
if (property === "length") {
throw new Error("length exploded");
}
return Reflect.get(target, property, receiver);
},
});
expect(projectOpenAITools(tools)).toEqual({
inputToolCount: 0,
tools: [],
diagnostics: [{ toolIndex: 0, violations: ["tool[0] is unreadable"] }],
});
});
it("rejects pinned and required choices when their function tools are unavailable", () => {
const projection = projectOpenAITools([
{
name: "broken",
get parameters(): never {
throw new Error("parameters exploded");
},
},
]);
expect(() =>
reconcileOpenAIResponsesToolChoice({ type: "function", name: "broken" }, projection),
).toThrow('requested unavailable tool "broken"');
expect(() => reconcileOpenAIResponsesToolChoice("required", projection)).toThrow(
"no tools survived schema conversion",
);
expect(() =>
reconcileOpenAICompletionsToolChoice(
{ type: "function", function: { name: "broken" } },
projection,
),
).toThrow('requested unavailable tool "broken"');
expect(() => reconcileOpenAICompletionsToolChoice("required", projection)).toThrow(
"no tools survived schema conversion",
);
});
it("filters official Responses allowed_tools without broadening access", () => {
const projection = projectOpenAITools([{ name: "lookup", parameters: {} }]);
expect(
reconcileOpenAIResponsesToolChoice(
{
type: "allowed_tools",
mode: "required",
tools: [
{ type: "function", name: "broken" },
{ type: "function", name: "lookup" },
{ type: "web_search_preview" },
],
},
projection,
),
).toEqual({
type: "allowed_tools",
mode: "required",
tools: [{ type: "function", name: "lookup" }, { type: "web_search_preview" }],
});
});
it("disables an auto allowed_tools choice when no allowed tools survive", () => {
const projection = projectOpenAITools([{ name: "lookup", parameters: {} }]);
expect(
reconcileOpenAIResponsesToolChoice(
{
type: "allowed_tools",
mode: "auto",
tools: [{ type: "function", name: "broken" }],
},
projection,
),
).toBe("none");
});
it("filters official Chat Completions allowed_tools without broadening access", () => {
const projection = projectOpenAITools([{ name: "lookup", parameters: {} }]);
expect(
reconcileOpenAICompletionsToolChoice(
{
type: "allowed_tools",
allowed_tools: {
mode: "required",
tools: [
{ type: "function", function: { name: "broken" } },
{ type: "function", function: { name: "lookup" } },
{ type: "custom", custom: { name: "shell" } },
],
},
},
projection,
),
).toEqual({
type: "allowed_tools",
allowed_tools: {
mode: "required",
tools: [{ type: "function", function: { name: "lookup" } }],
},
});
});
it("rejects unsupported top-level Chat Completions custom choices", () => {
const projection = projectOpenAITools([{ name: "lookup", parameters: {} }]);
expect(() =>
reconcileOpenAICompletionsToolChoice(
{ type: "custom", custom: { name: "shell" } },
projection,
),
).toThrow("custom tool_choice is unsupported");
});
it("disables an auto Chat Completions allowed_tools choice when none survive", () => {
const projection = projectOpenAITools([{ name: "lookup", parameters: {} }]);
expect(
reconcileOpenAICompletionsToolChoice(
{
type: "allowed_tools",
allowed_tools: {
mode: "auto",
tools: [
{ type: "function", function: { name: "broken" } },
{ type: "custom", custom: { name: "shell" } },
],
},
},
projection,
),
).toBe("none");
});
it("preserves non-function Responses choices", () => {
const projection = projectOpenAITools([]);
expect(reconcileOpenAIResponsesToolChoice({ type: "web_search_preview" }, projection)).toEqual({
type: "web_search_preview",
});
});
});

View File

@@ -0,0 +1,300 @@
import { isRecord } from "@openclaw/normalization-core/record-coerce";
import type OpenAI from "openai";
import type { ResponseCreateParamsStreaming } from "openai/resources/responses/responses.js";
import { projectRuntimeToolInputSchema } from "./tool-schema-json-projection.js";
type OpenAIToolDescriptor = {
readonly name?: unknown;
readonly description?: unknown;
readonly parameters: unknown;
};
type OpenAIProjectedTool = {
readonly toolIndex: number;
readonly name: string;
readonly description?: string;
readonly parameters: Record<string, unknown>;
};
type OpenAIToolProjectionDiagnostic = {
readonly toolIndex: number;
readonly toolName?: string;
readonly violations: readonly string[];
};
export type OpenAIToolProjection = {
readonly inputToolCount: number;
readonly tools: readonly OpenAIProjectedTool[];
readonly diagnostics: readonly OpenAIToolProjectionDiagnostic[];
};
type OpenAIResponsesToolChoice = ResponseCreateParamsStreaming["tool_choice"];
type OpenAIResponsesAllowedToolChoice = Extract<
OpenAIResponsesToolChoice,
{ type: "allowed_tools" }
>;
type OpenAICompletionsSdkToolChoice =
OpenAI.Chat.Completions.ChatCompletionCreateParamsStreaming["tool_choice"];
type OpenAICompletionsAllowedToolChoice = Extract<
OpenAICompletionsSdkToolChoice,
{ type: "allowed_tools" }
>;
export type OpenAICompletionsToolChoice = Exclude<
OpenAICompletionsSdkToolChoice,
{ type: "custom" }
>;
function unreadableToolDiagnostic(toolIndex: number): OpenAIToolProjectionDiagnostic {
return {
toolIndex,
violations: [`tool[${toolIndex}] is unreadable`],
};
}
/** Snapshots direct/custom tool descriptors before OpenAI payload construction. */
export function projectOpenAITools(tools: readonly OpenAIToolDescriptor[]): OpenAIToolProjection {
let inputToolCount: number;
try {
inputToolCount = tools.length;
} catch {
return {
inputToolCount: 0,
tools: [],
diagnostics: [unreadableToolDiagnostic(0)],
};
}
const projectedTools: OpenAIProjectedTool[] = [];
const diagnostics: OpenAIToolProjectionDiagnostic[] = [];
for (let toolIndex = 0; toolIndex < inputToolCount; toolIndex += 1) {
let tool: OpenAIToolDescriptor;
try {
tool = tools[toolIndex];
} catch {
diagnostics.push(unreadableToolDiagnostic(toolIndex));
continue;
}
let name: unknown;
try {
name = tool.name;
} catch {
diagnostics.push({
toolIndex,
violations: [`tool[${toolIndex}].name is unreadable`],
});
continue;
}
if (typeof name !== "string" || !name) {
diagnostics.push({
toolIndex,
violations: [`tool[${toolIndex}].name is empty`],
});
continue;
}
let parameters: unknown;
try {
parameters = tool.parameters;
} catch {
diagnostics.push({
toolIndex,
toolName: name,
violations: [`${name}.parameters is unreadable`],
});
continue;
}
const schemaProjection = projectRuntimeToolInputSchema(parameters ?? {}, `${name}.parameters`);
if (!isRecord(schemaProjection.schema) || schemaProjection.violations.length > 0) {
diagnostics.push({
toolIndex,
toolName: name,
violations:
schemaProjection.violations.length > 0
? schemaProjection.violations
: [`${name}.parameters must be a JSON object schema`],
});
continue;
}
let descriptionValue: unknown;
try {
descriptionValue = tool.description;
} catch {
// Description is optional; preserve the usable function schema.
}
const description = typeof descriptionValue === "string" ? descriptionValue : undefined;
projectedTools.push({
toolIndex,
name,
...(description !== undefined ? { description } : {}),
parameters: schemaProjection.schema,
});
}
return {
inputToolCount,
tools: projectedTools,
diagnostics,
};
}
function requireProjectedFunction(
name: string,
projection: OpenAIToolProjection,
choiceLabel: string,
): void {
if (!projection.tools.some((tool) => tool.name === name)) {
throw new Error(`${choiceLabel} requested unavailable tool "${name}" after schema conversion`);
}
}
/** Keeps Responses tool choices aligned with surviving function schemas. */
export function reconcileOpenAIResponsesToolChoice(
choice: OpenAIResponsesToolChoice,
projection: OpenAIToolProjection,
): OpenAIResponsesToolChoice | undefined {
if (choice === "auto") {
return projection.tools.length > 0 ? choice : undefined;
}
if (choice === "required") {
if (projection.tools.length === 0) {
throw new Error(
"OpenAI Responses tool_choice requires a tool, but no tools survived schema conversion",
);
}
return choice;
}
if (choice === "none" || !isRecord(choice)) {
return choice;
}
const choiceType = choice.type;
if (choiceType === "function") {
const functionName = choice.name;
if (typeof functionName !== "string") {
return choice;
}
requireProjectedFunction(functionName, projection, "OpenAI Responses tool_choice");
return { type: "function", name: functionName };
}
if (choiceType !== "allowed_tools") {
return choice;
}
const mode = choice.mode;
const tools = choice.tools;
if ((mode !== "auto" && mode !== "required") || !Array.isArray(tools)) {
return choice;
}
const normalizedAllowedTools: OpenAIResponsesAllowedToolChoice["tools"] = [];
for (const tool of tools) {
if (!isRecord(tool) || tool.type !== "function") {
normalizedAllowedTools.push(tool);
continue;
}
const functionName = tool.name;
if (
typeof functionName === "string" &&
projection.tools.some((projectedTool) => projectedTool.name === functionName)
) {
normalizedAllowedTools.push({ type: "function", name: functionName });
}
}
if (normalizedAllowedTools.length === 0) {
if (mode === "auto") {
return "none";
}
throw new Error(
"OpenAI Responses tool_choice requires a tool, but no allowed tools survived schema conversion",
);
}
return {
type: "allowed_tools",
mode,
tools: normalizedAllowedTools,
};
}
/** Keeps Chat Completions tool choices aligned with surviving function schemas. */
export function reconcileOpenAICompletionsToolChoice(
choice: OpenAICompletionsSdkToolChoice,
projection: OpenAIToolProjection,
): OpenAICompletionsSdkToolChoice | undefined {
if (choice === "auto") {
return projection.tools.length > 0 ? choice : undefined;
}
if (choice === "required") {
if (projection.tools.length === 0) {
throw new Error(
"OpenAI Chat Completions tool_choice requires a tool, but no tools survived schema conversion",
);
}
return choice;
}
if (choice === "none" || !isRecord(choice)) {
return choice;
}
const choiceType = choice.type;
if (choiceType === "custom") {
throw new Error(
"OpenAI Chat Completions custom tool_choice is unsupported because this adapter emits function tools only",
);
}
if (choiceType === "function") {
const functionChoice = choice.function;
if (!isRecord(functionChoice)) {
return choice;
}
const functionName = functionChoice.name;
if (typeof functionName !== "string") {
return choice;
}
requireProjectedFunction(functionName, projection, "OpenAI Chat Completions tool_choice");
return { type: "function", function: { name: functionName } };
}
if (choiceType !== "allowed_tools") {
return choice;
}
const allowedConfig = choice.allowed_tools;
if (!isRecord(allowedConfig)) {
return choice;
}
const mode = allowedConfig.mode;
const tools = allowedConfig.tools;
if ((mode !== "auto" && mode !== "required") || !Array.isArray(tools)) {
return choice;
}
const normalizedAllowedTools: OpenAICompletionsAllowedToolChoice["allowed_tools"]["tools"] = [];
for (const tool of tools) {
if (!isRecord(tool) || tool.type !== "function") {
continue;
}
const functionChoice = tool.function;
const functionName = isRecord(functionChoice) ? functionChoice.name : undefined;
if (
typeof functionName === "string" &&
projection.tools.some((projectedTool) => projectedTool.name === functionName)
) {
normalizedAllowedTools.push({
type: "function",
function: { name: functionName },
});
}
}
if (normalizedAllowedTools.length === 0) {
if (mode === "auto") {
return "none";
}
throw new Error(
"OpenAI Chat Completions tool_choice requires a tool, but no allowed tools survived schema conversion",
);
}
return {
type: "allowed_tools",
allowed_tools: {
mode,
tools: normalizedAllowedTools,
},
};
}

View File

@@ -0,0 +1,174 @@
// Verifies OpenAI strict tool schema normalization and cache behavior.
import { beforeEach, describe, expect, it } from "vitest";
import { projectOpenAITools } from "./openai-tool-projection.js";
import {
clearOpenAIToolSchemaCacheForTest,
findOpenAIStrictToolProjectionDiagnostics,
isStrictOpenAIJsonSchemaCompatible,
normalizeOpenAIStrictToolParameters,
normalizeStrictOpenAIJsonSchema,
resolveOpenAIProjectedToolsStrictToolFlag,
} from "./openai-tool-schema.js";
describe("OpenAI strict tool schema normalization", () => {
beforeEach(() => {
clearOpenAIToolSchemaCacheForTest();
});
it("repairs top-level object schemas with missing or invalid properties", () => {
const schemas = [
{ type: "object" },
{ type: "object", properties: undefined },
{ type: "object", properties: null },
{ type: "object", properties: [] },
{ type: "object", properties: "invalid" },
];
for (const schema of schemas) {
expect(normalizeStrictOpenAIJsonSchema(schema)).toEqual({
type: "object",
properties: {},
required: [],
additionalProperties: false,
});
expect(isStrictOpenAIJsonSchemaCompatible(schema)).toBe(true);
}
});
it("does not close permissive nested object schemas implicitly", () => {
// Nested permissive objects stay incompatible unless callers make them strict.
const schema = {
type: "object",
properties: {
metadata: {
type: "object",
},
},
required: ["metadata"],
};
const normalized = normalizeStrictOpenAIJsonSchema(schema) as {
additionalProperties?: boolean;
properties?: { metadata?: { additionalProperties?: boolean } };
};
expect(normalized.additionalProperties).toBe(false);
expect(normalized.properties?.metadata).not.toHaveProperty("additionalProperties");
expect(isStrictOpenAIJsonSchemaCompatible(schema)).toBe(false);
expect(
resolveOpenAIProjectedToolsStrictToolFlag(
projectOpenAITools([{ name: "write", parameters: schema }]),
true,
),
).toBe(false);
});
it("normalizes truly empty MCP tool schema {} for strict mode", () => {
const schema = {};
const normalized = normalizeStrictOpenAIJsonSchema(schema) as Record<string, unknown>;
expect(normalized.type).toBe("object");
expect(normalized.properties).toStrictEqual({});
expect(normalized.required).toStrictEqual([]);
expect(normalized.additionalProperties).toBe(false);
expect(isStrictOpenAIJsonSchemaCompatible(schema)).toBe(true);
});
it("reuses normalized strict schemas for stable tool schema objects", () => {
// Cache keys include unsupported-keyword policy, not just object identity.
const schema = {
type: "object",
properties: {
path: { type: "string" },
},
required: ["path"],
};
const first = normalizeStrictOpenAIJsonSchema(schema);
const second = normalizeStrictOpenAIJsonSchema(schema);
const third = normalizeStrictOpenAIJsonSchema(schema, {
unsupportedToolSchemaKeywords: ["minimum"],
});
expect(second).toBe(first);
expect(third).not.toBe(first);
expect(
normalizeStrictOpenAIJsonSchema(schema, {
unsupportedToolSchemaKeywords: ["minimum"],
}),
).toBe(third);
});
it("reports unreadable nested tool schemas instead of throwing", () => {
const unreadable = {
name: "broken",
parameters: {
type: "object",
get properties(): never {
throw new Error("properties exploded");
},
},
};
const projection = projectOpenAITools([unreadable]);
expect(findOpenAIStrictToolProjectionDiagnostics(projection)).toEqual([
{
toolIndex: 0,
toolName: "broken",
violations: ["broken.parameters is not JSON-serializable"],
},
]);
});
it("keeps strict mode for emitted tools when unreadable tools are dropped", () => {
const projection = projectOpenAITools([
{
name: "broken",
parameters: {
type: "object",
get properties(): never {
throw new Error("properties exploded");
},
},
},
{
name: "lookup",
parameters: {
type: "object",
properties: {},
required: [],
additionalProperties: false,
},
},
]);
expect(resolveOpenAIProjectedToolsStrictToolFlag(projection, true)).toBe(true);
});
it("reuses projected schemas for strict checks and normalization", () => {
let serializationCount = 0;
const projection = projectOpenAITools([
{
name: "lookup",
parameters: {
toJSON() {
serializationCount += 1;
return {
type: "object",
properties: {},
required: [],
additionalProperties: false,
};
},
},
},
]);
const tool = projection.tools[0];
expect(tool).toBeDefined();
expect(resolveOpenAIProjectedToolsStrictToolFlag(projection, true)).toBe(true);
const normalized = normalizeOpenAIStrictToolParameters(tool?.parameters, true);
expect(normalizeOpenAIStrictToolParameters(tool?.parameters, true)).toBe(normalized);
expect(serializationCount).toBe(1);
});
});

View File

@@ -0,0 +1,324 @@
/**
* OpenAI strict JSON-schema normalization for tool inventories and request payloads.
*
* Caches normalized object inputs by provider compatibility so repeated inventory builds preserve identity.
*/
import {
normalizeToolParameterSchema,
shouldOmitEmptyArrayItems,
type ToolSchemaModelCompat,
} from "./agent-tools-parameter-schema.js";
import type { OpenAIToolProjection } from "./openai-tool-projection.js";
/**
* OpenAI strict-tool-schema normalization and diagnostics.
*
* Strict schemas need all object properties required and `additionalProperties: false`; model
* compatibility settings can also remove unsupported schema constructs before strict checks run.
*/
type ToolSchemaCompatInput = {
unsupportedToolSchemaKeywords?: unknown;
omitEmptyArrayItems?: unknown;
};
const MAX_STRICT_SCHEMA_CACHE_ENTRIES_PER_SCHEMA = 8;
let strictOpenAISchemaCache = new WeakMap<object, Array<{ key: string; value: unknown }>>();
function resolveToolSchemaModelCompat(
compat: ToolSchemaCompatInput | null | undefined,
): ToolSchemaModelCompat | undefined {
if (!compat) {
return undefined;
}
const unsupportedToolSchemaKeywords = Array.isArray(compat.unsupportedToolSchemaKeywords)
? compat.unsupportedToolSchemaKeywords.filter(
(keyword): keyword is string => typeof keyword === "string",
)
: [];
if (unsupportedToolSchemaKeywords.length === 0 && compat.omitEmptyArrayItems !== true) {
return undefined;
}
return {
...(unsupportedToolSchemaKeywords.length > 0 ? { unsupportedToolSchemaKeywords } : {}),
...(compat.omitEmptyArrayItems === true ? { omitEmptyArrayItems: true } : {}),
};
}
function resolveStrictOpenAISchemaCacheKey(
modelCompat: ToolSchemaCompatInput | null | undefined,
): string {
const compat = resolveToolSchemaModelCompat(modelCompat);
return JSON.stringify([
[...(compat?.unsupportedToolSchemaKeywords ?? [])].toSorted(),
shouldOmitEmptyArrayItems(compat),
]);
}
function readCachedStrictOpenAISchema(schema: object, key: string): unknown {
return strictOpenAISchemaCache.get(schema)?.find((entry) => entry.key === key)?.value;
}
function rememberStrictOpenAISchema(schema: object, key: string, value: unknown): unknown {
const entries = strictOpenAISchemaCache.get(schema) ?? [];
strictOpenAISchemaCache.set(
schema,
[{ key, value }, ...entries.filter((entry) => entry.key !== key)].slice(
0,
MAX_STRICT_SCHEMA_CACHE_ENTRIES_PER_SCHEMA,
),
);
return value;
}
export function clearOpenAIToolSchemaCacheForTest(): void {
strictOpenAISchemaCache = new WeakMap();
}
/** Normalizes a tool parameter schema into the OpenAI strict JSON-schema subset. */
export function normalizeStrictOpenAIJsonSchema(
schema: unknown,
modelCompat?: ToolSchemaCompatInput | null,
): unknown {
const schemaInput = schema ?? {};
if (!schemaInput || typeof schemaInput !== "object") {
return normalizeStrictOpenAIJsonSchemaRecursive(
normalizeToolParameterSchema(schemaInput, {
modelCompat: resolveToolSchemaModelCompat(modelCompat),
}),
0,
);
}
const cacheKey = resolveStrictOpenAISchemaCacheKey(modelCompat);
const cached = readCachedStrictOpenAISchema(schemaInput, cacheKey);
if (cached !== undefined) {
return cached;
}
return rememberStrictOpenAISchema(
schemaInput,
cacheKey,
// Cache by input object and compatibility key so repeated inventory generation preserves object
// identity without mixing schemas normalized for different provider limitations.
normalizeStrictOpenAIJsonSchemaRecursive(
normalizeToolParameterSchema(schemaInput, {
modelCompat: resolveToolSchemaModelCompat(modelCompat),
}),
0,
),
);
}
function normalizeStrictOpenAIJsonSchemaRecursive(schema: unknown, depth: number): unknown {
if (Array.isArray(schema)) {
let changed = false;
const normalized = schema.map((entry) => {
const next = normalizeStrictOpenAIJsonSchemaRecursive(entry, depth);
changed ||= next !== entry;
return next;
});
return changed ? normalized : schema;
}
if (!schema || typeof schema !== "object") {
return schema;
}
const record = schema as Record<string, unknown>;
let changed = false;
const normalized: Record<string, unknown> = {};
for (const [key, value] of Object.entries(record)) {
const next = normalizeStrictOpenAIJsonSchemaRecursive(
value,
key === "properties" ? depth : depth + 1,
);
normalized[key] = next;
changed ||= next !== value;
}
if (normalized.type === "object") {
const properties =
normalized.properties &&
typeof normalized.properties === "object" &&
!Array.isArray(normalized.properties)
? (normalized.properties as Record<string, unknown>)
: undefined;
if (properties && Object.keys(properties).length === 0 && !Array.isArray(normalized.required)) {
normalized.required = [];
changed = true;
}
if (depth === 0 && !("additionalProperties" in normalized)) {
normalized.additionalProperties = false;
changed = true;
}
}
return changed ? normalized : schema;
}
/** Normalizes tool parameters using strict OpenAI rules only when strict mode is active. */
export function normalizeOpenAIStrictToolParameters<T>(
schema: T,
strict: boolean,
modelCompat?: ToolSchemaCompatInput | null,
): T {
const toolSchemaCompat = resolveToolSchemaModelCompat(modelCompat);
if (!strict) {
return normalizeToolParameterSchema(schema ?? {}, { modelCompat: toolSchemaCompat }) as T;
}
return normalizeStrictOpenAIJsonSchema(schema, toolSchemaCompat) as T;
}
/** Returns whether a schema already satisfies OpenAI strict tool-schema constraints. */
export function isStrictOpenAIJsonSchemaCompatible(schema: unknown): boolean {
return isStrictOpenAIJsonSchemaCompatibleRecursive(normalizeStrictOpenAIJsonSchema(schema));
}
type OpenAIStrictToolSchemaDiagnostic = {
toolIndex: number;
toolName?: string;
violations: string[];
};
/** Returns strict-schema diagnostics for an already materialized OpenAI tool projection. */
export function findOpenAIStrictToolProjectionDiagnostics(
projection: OpenAIToolProjection,
): OpenAIStrictToolSchemaDiagnostic[] {
return [
...projection.diagnostics.map((diagnostic) => ({
toolIndex: diagnostic.toolIndex,
...(diagnostic.toolName ? { toolName: diagnostic.toolName } : {}),
violations: [...diagnostic.violations],
})),
...projection.tools.flatMap((tool) => {
const violations = findStrictOpenAIJsonSchemaViolations(
normalizeStrictOpenAIJsonSchema(tool.parameters),
`${tool.name}.parameters`,
);
return violations.length > 0
? [{ toolIndex: tool.toolIndex, toolName: tool.name, violations }]
: [];
}),
];
}
function isStrictOpenAIJsonSchemaCompatibleRecursive(schema: unknown): boolean {
if (Array.isArray(schema)) {
return schema.every((entry) => isStrictOpenAIJsonSchemaCompatibleRecursive(entry));
}
if (!schema || typeof schema !== "object") {
return true;
}
const record = schema as Record<string, unknown>;
if ("anyOf" in record || "oneOf" in record || "allOf" in record) {
return false;
}
if (Array.isArray(record.type)) {
return false;
}
if (record.type === "object" && record.additionalProperties !== false) {
return false;
}
if (record.type === "object") {
const properties =
record.properties &&
typeof record.properties === "object" &&
!Array.isArray(record.properties)
? (record.properties as Record<string, unknown>)
: {};
const required = Array.isArray(record.required)
? record.required.filter((entry): entry is string => typeof entry === "string")
: undefined;
if (!required) {
return false;
}
const requiredSet = new Set(required);
if (Object.keys(properties).some((key) => !requiredSet.has(key))) {
return false;
}
}
return Object.entries(record).every(([key, entry]) => {
if (key === "properties" && entry && typeof entry === "object" && !Array.isArray(entry)) {
return Object.values(entry as Record<string, unknown>).every((value) =>
isStrictOpenAIJsonSchemaCompatibleRecursive(value),
);
}
return isStrictOpenAIJsonSchemaCompatibleRecursive(entry);
});
}
function findStrictOpenAIJsonSchemaViolations(schema: unknown, path: string): string[] {
if (Array.isArray(schema)) {
return schema.flatMap((entry, index) =>
findStrictOpenAIJsonSchemaViolations(entry, `${path}[${index}]`),
);
}
if (!schema || typeof schema !== "object") {
return [];
}
const record = schema as Record<string, unknown>;
const violations: string[] = [];
for (const key of ["anyOf", "oneOf", "allOf"] as const) {
if (key in record) {
violations.push(`${path}.${key}`);
}
}
if (Array.isArray(record.type)) {
violations.push(`${path}.type`);
}
if (record.type === "object") {
if (record.additionalProperties !== false) {
violations.push(`${path}.additionalProperties`);
}
const properties =
record.properties &&
typeof record.properties === "object" &&
!Array.isArray(record.properties)
? (record.properties as Record<string, unknown>)
: {};
const required = Array.isArray(record.required)
? record.required.filter((entry): entry is string => typeof entry === "string")
: undefined;
if (!required) {
violations.push(`${path}.required`);
} else {
const requiredSet = new Set(required);
for (const key of Object.keys(properties)) {
if (!requiredSet.has(key)) {
violations.push(`${path}.required.${key}`);
}
}
}
}
if (
record.properties &&
typeof record.properties === "object" &&
!Array.isArray(record.properties)
) {
for (const [key, value] of Object.entries(record.properties)) {
violations.push(...findStrictOpenAIJsonSchemaViolations(value, `${path}.properties.${key}`));
}
}
for (const [key, value] of Object.entries(record)) {
if (key === "properties") {
continue;
}
if (value && typeof value === "object") {
violations.push(...findStrictOpenAIJsonSchemaViolations(value, `${path}.${key}`));
}
}
return violations;
}
/** Resolves strict mode for the projected tools that will be emitted in the request payload. */
export function resolveOpenAIProjectedToolsStrictToolFlag(
projection: OpenAIToolProjection,
strict: boolean | null | undefined,
): boolean | undefined {
if (strict !== true) {
return strict === false ? false : undefined;
}
return projection.tools.every((tool) => isStrictOpenAIJsonSchemaCompatible(tool.parameters));
}

View File

@@ -0,0 +1,164 @@
// Built-in provider registration installs lazy protocol adapters.
import type { ApiRegistry } from "../api-registry.js";
import type {
Api,
AssistantMessage,
AssistantMessageEvent,
Model,
SimpleStreamOptions,
StreamFunction,
StreamOptions,
} from "../types.js";
import { AssistantMessageEventStream } from "../utils/event-stream.js";
type ProviderStreams<TApi extends Api, TOptions extends StreamOptions> = {
stream: StreamFunction<TApi, TOptions>;
streamSimple: StreamFunction<TApi, SimpleStreamOptions>;
};
type RegisterBuiltIn = (registry: ApiRegistry) => void;
/** Source id used for built-in API provider registrations. */
export const BUILT_IN_API_PROVIDER_SOURCE_ID = "core:built-in";
function forwardStream(
target: AssistantMessageEventStream,
source: AsyncIterable<AssistantMessageEvent>,
): void {
void (async () => {
for await (const event of source) {
target.push(event);
}
target.end();
})();
}
function createLazyLoadErrorMessage<TApi extends Api>(
model: Model<TApi>,
error: unknown,
): AssistantMessage {
return {
role: "assistant",
content: [],
api: model.api,
provider: model.provider,
model: model.id,
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "error",
errorMessage: error instanceof Error ? error.message : String(error),
timestamp: Date.now(),
};
}
// Provider modules load on first use, while callers still receive a stream synchronously.
function createLazyStream<TApi extends Api, TOptions extends StreamOptions, TStreams>(
load: () => Promise<TStreams>,
select: (streams: TStreams) => StreamFunction<TApi, TOptions>,
): StreamFunction<TApi, TOptions> {
return (model, context, options) => {
const outer = new AssistantMessageEventStream();
load()
.then((streams) => forwardStream(outer, select(streams)(model, context, options)))
.catch((error: unknown) => {
const message = createLazyLoadErrorMessage(model, error);
outer.push({ type: "error", reason: "error", error: message });
outer.end(message);
});
return outer;
};
}
function createLazyRegistration<TApi extends Api, TOptions extends StreamOptions, TModule>(
api: TApi,
importModule: () => Promise<TModule>,
select: (module: TModule) => ProviderStreams<TApi, TOptions>,
): RegisterBuiltIn {
let streamsPromise: Promise<ProviderStreams<TApi, TOptions>> | undefined;
const load = () => (streamsPromise ??= importModule().then(select));
const stream = createLazyStream(load, (streams) => streams.stream);
const streamSimple = createLazyStream<TApi, SimpleStreamOptions, ProviderStreams<TApi, TOptions>>(
load,
(streams) => streams.streamSimple,
);
return (registry) => {
registry.registerApiProvider({ api, stream, streamSimple }, BUILT_IN_API_PROVIDER_SOURCE_ID);
};
}
const registerBuiltIns: RegisterBuiltIn[] = [
createLazyRegistration(
"anthropic-messages",
() => import("./anthropic.js"),
(module) => ({ stream: module.streamAnthropic, streamSimple: module.streamSimpleAnthropic }),
),
createLazyRegistration(
"openai-completions",
() => import("./openai-completions.js"),
(module) => ({
stream: module.streamOpenAICompletions,
streamSimple: module.streamSimpleOpenAICompletions,
}),
),
createLazyRegistration(
"mistral-conversations",
() => import("./mistral.js"),
(module) => ({ stream: module.streamMistral, streamSimple: module.streamSimpleMistral }),
),
createLazyRegistration(
"openai-responses",
() => import("./openai-responses.js"),
(module) => ({
stream: module.streamOpenAIResponses,
streamSimple: module.streamSimpleOpenAIResponses,
}),
),
createLazyRegistration(
"azure-openai-responses",
() => import("./azure-openai-responses.js"),
(module) => ({
stream: module.streamAzureOpenAIResponses,
streamSimple: module.streamSimpleAzureOpenAIResponses,
}),
),
createLazyRegistration(
"openai-chatgpt-responses",
() => import("./openai-chatgpt-responses.js"),
(module) => ({
stream: module.streamOpenAICodexResponses,
streamSimple: module.streamSimpleOpenAICodexResponses,
}),
),
createLazyRegistration(
"google-generative-ai",
() => import("./google.js"),
(module) => ({ stream: module.streamGoogle, streamSimple: module.streamSimpleGoogle }),
),
createLazyRegistration(
"google-vertex",
() => import("./google-vertex.js"),
(module) => ({
stream: module.streamGoogleVertex,
streamSimple: module.streamSimpleGoogleVertex,
}),
),
];
/** Registers every built-in API provider in one runtime registry. */
export function registerBuiltInApiProviders(registry: ApiRegistry): void {
for (const register of registerBuiltIns) {
register(registry);
}
}
/** Restores the built-in provider registry state for tests. */
export function resetApiProviders(registry: ApiRegistry): void {
registry.unregisterApiProviders(BUILT_IN_API_PROVIDER_SOURCE_ID);
registerBuiltInApiProviders(registry);
}

View File

@@ -0,0 +1,44 @@
/** Recursively remove schema keywords unsupported by a target provider/tool surface. */
export function stripUnsupportedSchemaKeywords(
schema: unknown,
unsupportedKeywords: ReadonlySet<string>,
): unknown {
if (!schema || typeof schema !== "object") {
return schema;
}
if (Array.isArray(schema)) {
return schema.map((entry) => stripUnsupportedSchemaKeywords(entry, unsupportedKeywords));
}
const obj = schema as Record<string, unknown>;
const cleaned: Record<string, unknown> = {};
for (const [key, value] of Object.entries(obj)) {
if (unsupportedKeywords.has(key)) {
continue;
}
// Schema containers hold nested schemas under different shapes. Recurse
// through each known container while preserving unrelated metadata fields.
if (key === "properties" && value && typeof value === "object" && !Array.isArray(value)) {
cleaned[key] = Object.fromEntries(
Object.entries(value as Record<string, unknown>).map(([childKey, childValue]) => [
childKey,
stripUnsupportedSchemaKeywords(childValue, unsupportedKeywords),
]),
);
continue;
}
if (key === "items" && value && typeof value === "object") {
cleaned[key] = Array.isArray(value)
? value.map((entry) => stripUnsupportedSchemaKeywords(entry, unsupportedKeywords))
: stripUnsupportedSchemaKeywords(value, unsupportedKeywords);
continue;
}
if ((key === "anyOf" || key === "oneOf" || key === "allOf") && Array.isArray(value)) {
cleaned[key] = value.map((entry) =>
stripUnsupportedSchemaKeywords(entry, unsupportedKeywords),
);
continue;
}
cleaned[key] = value;
}
return cleaned;
}

View File

@@ -0,0 +1,79 @@
// Simple provider option helpers normalize lightweight provider configuration.
import type {
Model,
SimpleStreamOptions,
StreamOptions,
ThinkingBudgets,
ThinkingLevel,
} from "../types.js";
type FirstEventStreamOptions = {
firstEventTimeoutMs?: number;
onFirstEventTimeout?: (reason: Error) => void;
};
export function buildBaseOptions(
model: Model,
options?: SimpleStreamOptions,
apiKey?: string,
): StreamOptions & FirstEventStreamOptions {
void model;
const firstEventOptions = options as FirstEventStreamOptions | undefined;
return {
temperature: options?.temperature,
maxTokens: options?.maxTokens,
stop: options?.stop,
signal: options?.signal,
apiKey: apiKey || options?.apiKey,
transport: options?.transport,
cacheRetention: options?.cacheRetention,
sessionId: options?.sessionId,
promptCacheKey: options?.promptCacheKey,
headers: options?.headers,
onPayload: options?.onPayload,
onResponse: options?.onResponse,
timeoutMs: options?.timeoutMs,
firstEventTimeoutMs: firstEventOptions?.firstEventTimeoutMs,
onFirstEventTimeout: firstEventOptions?.onFirstEventTimeout,
maxRetries: options?.maxRetries,
maxRetryDelayMs: options?.maxRetryDelayMs,
metadata: options?.metadata,
};
}
export function clampReasoning(
effort: ThinkingLevel | undefined,
): Exclude<ThinkingLevel, "xhigh"> | undefined {
return effort === "xhigh" ? "high" : effort;
}
export function adjustMaxTokensForThinking(
// Undefined means no explicit caller cap. Use the model cap and fit thinking inside it.
baseMaxTokens: number | undefined,
modelMaxTokens: number,
reasoningLevel: ThinkingLevel,
customBudgets?: ThinkingBudgets,
): { maxTokens: number; thinkingBudget: number } {
const defaultBudgets: ThinkingBudgets = {
minimal: 1024,
low: 2048,
medium: 8192,
high: 16384,
max: 32768,
};
const budgets = { ...defaultBudgets, ...customBudgets };
const minOutputTokens = 1024;
const level = clampReasoning(reasoningLevel)!;
let thinkingBudget = budgets[level]!;
const maxTokens =
baseMaxTokens === undefined
? modelMaxTokens
: Math.min(baseMaxTokens + thinkingBudget, modelMaxTokens);
if (maxTokens <= thinkingBudget) {
thinkingBudget = Math.max(0, maxTokens - minOutputTokens);
}
return { maxTokens, thinkingBudget };
}

View File

@@ -0,0 +1,118 @@
import { describe, expect, it } from "vitest";
import { describeToolResultMediaPlaceholder, extractToolResultText } from "./tool-result-text.js";
describe("extractToolResultText", () => {
it("keeps media-only blocks out of provider replay text", () => {
const text = extractToolResultText([
{ type: "text", text: "summary" },
{ type: "image", data: "image-binary", mimeType: "image/png" },
{ type: "image_url", image_url: { url: "data:image/png;base64,abc123" } },
{ type: "input_image", image_url: "data:image/png;base64,def456" },
{ type: "audio", data: "audio-binary", mimeType: "audio/mpeg" },
]);
expect(text).toBe("summary");
expect(text).not.toContain("image-binary");
expect(text).not.toContain("abc123");
expect(text).not.toContain("def456");
expect(text).not.toContain("audio-binary");
});
it("omits MIME-tagged binary data while preserving textual resource data", () => {
const text = extractToolResultText([
{ type: "resource", mime_type: "application/octet-stream", data: "AAECAwQFBgc=" },
{ type: "resource", mediaType: "application/json", data: '{"ok":true}' },
]);
expect(text).toContain('"data":"[binary data omitted: 12 chars]"');
expect(text).toContain('{\\"ok\\":true}');
expect(text).not.toContain("AAECAwQFBgc=");
});
it("redacts inline data URIs without touching ordinary data-colon prose", () => {
const text = extractToolResultText([
{
type: "json",
value: {
note: "metadata:ready",
prose: "data: is ordinary prose",
preview: "thumbnail=data:image/png;base64,abcdef done",
},
},
]);
expect(text).toContain("metadata:ready");
expect(text).toContain("data: is ordinary prose");
expect(text).toContain("[inline data URI:");
expect(text).not.toContain("abcdef");
});
it("omits opaque or binary structured fields", () => {
const text = extractToolResultText([
{
type: "json",
encrypted_content: "ciphertext",
bytes: [1, 2, 3],
visible: "safe-value",
},
]);
expect(text).toContain('"encrypted_content":"[omitted encrypted_content]"');
expect(text).toContain('"bytes":"[omitted bytes]"');
expect(text).toContain('"visible":"safe-value"');
expect(text).not.toContain("ciphertext");
});
it("uses structured replay only as a no-text fallback without capping explicit text", () => {
const textTail = "explicit-tail-marker";
const text = extractToolResultText([
{ type: "text", text: `${"x".repeat(8_200)}${textTail}` },
{ type: "json", internal: "extra structured detail" },
]);
expect(text).toContain(textTail);
expect(text).not.toContain("…(truncated)…");
expect(text).not.toContain("extra structured detail");
});
it("truncates structured fallback text before provider replay", () => {
const tail = "tail-marker";
const text = extractToolResultText([
{
type: "json",
data: {
payload: `${"x".repeat(8_200)}${tail}`,
},
},
]);
expect(text.length).toBeLessThan(8_100);
expect(text).toContain("…(truncated)…");
expect(text).not.toContain(tail);
});
});
describe("describeToolResultMediaPlaceholder", () => {
it("describes image-only tool result media", () => {
expect(
describeToolResultMediaPlaceholder([{ type: "image", mimeType: "image/png", data: "img" }]),
).toBe("(see attached image)");
});
it("describes audio-only tool result media", () => {
expect(
describeToolResultMediaPlaceholder([
{ type: "audio", mimeType: "audio/mpeg", data: "audio" },
]),
).toBe("(see attached audio)");
});
it("describes mixed image and audio tool result media", () => {
expect(
describeToolResultMediaPlaceholder([
{ type: "image", mimeType: "image/png", data: "img" },
{ type: "audio", mimeType: "audio/mpeg", data: "audio" },
]),
).toBe("(see attached media)");
});
});

View File

@@ -0,0 +1,197 @@
import { isRecord } from "@openclaw/normalization-core/record-coerce";
import { truncateUtf16Safe } from "@openclaw/normalization-core/utf16-slice";
import { getAiTransportHost } from "../host.js";
import { sanitizeSurrogates } from "../utils/sanitize-unicode.js";
const PROVIDER_TOOL_RESULT_MAX_CHARS = 8000;
const IMAGE_TOOL_RESULT_TYPES = new Set(["image", "image_url", "input_image"]);
const AUDIO_TOOL_RESULT_TYPES = new Set(["audio", "input_audio", "output_audio"]);
const MEDIA_ONLY_TOOL_RESULT_TYPES = new Set([
...IMAGE_TOOL_RESULT_TYPES,
...AUDIO_TOOL_RESULT_TYPES,
]);
const INLINE_DATA_URI_PATTERN =
/(^|[^A-Za-z0-9_])data:([a-z][a-z0-9.+-]*\/[a-z0-9.+-]+(?:;[a-z0-9.+-]+=[^,;"'\s]+|;base64)*,[^\s"'<>)]+)/gi;
const MIME_KEY_CANDIDATES = [
"mimeType",
"mime_type",
"mediaType",
"media_type",
"contentType",
"content_type",
];
const TEXTUAL_MIME_PATTERN =
/^(?:text\/|application\/(?:json|ld\+json|x-ndjson|xml|javascript|x-www-form-urlencoded)|[^/]+\/[^+]+\+(?:json|xml)$)/i;
const OPAQUE_OR_BINARY_FIELD_RE = /^(?:blob|buffer|bytes|encrypted_content|encrypted_stdout)$/i;
function readMimeType(value: unknown): string | undefined {
if (!isRecord(value)) {
return undefined;
}
for (const key of MIME_KEY_CANDIDATES) {
const mimeType = value[key];
if (typeof mimeType === "string" && mimeType.trim().length > 0) {
return mimeType;
}
}
return undefined;
}
function isBinaryMimeType(mimeType: string): boolean {
const normalized = mimeType.split(";", 1)[0]?.trim().toLowerCase();
return normalized ? !TEXTUAL_MIME_PATTERN.test(normalized) : false;
}
function describeOmittedValue(value: unknown, label: string): string {
const length = typeof value === "string" ? value.length : JSON.stringify(value)?.length;
return length ? `[${label} omitted: ${length} chars]` : `[${label} omitted]`;
}
function redactInlineDataUris(value: string): string {
return value.replace(
INLINE_DATA_URI_PATTERN,
(_match, prefix: string, uri: string) => `${prefix}[inline data URI: ${uri.length} chars]`,
);
}
function redactStructuredTextValue(value: string): string {
const host = getAiTransportHost();
const redacted = host.redactToolPayloadText(value);
const trimmed = redacted.trim();
if (!trimmed.startsWith("{") && !trimmed.startsWith("[")) {
return redacted;
}
try {
const redactedWrapper = host.redactSecrets({ structuredTextValue: JSON.parse(redacted) });
return JSON.stringify(redactedWrapper.structuredTextValue);
} catch {
return redacted;
}
}
function stringifyStructuredBlock(block: Record<string, unknown>): string | undefined {
const seen = new WeakSet<object>();
try {
const redactedWrapper = getAiTransportHost().redactSecrets({ structuredToolResult: block });
const redactedBlock = redactedWrapper.structuredToolResult;
const serialized = JSON.stringify(
redactedBlock,
function structuredToolResultReplacer(this: unknown, key, value) {
if (OPAQUE_OR_BINARY_FIELD_RE.test(key)) {
return `[omitted ${key}]`;
}
if (key === "data") {
const mimeType = readMimeType(this);
if (mimeType && isBinaryMimeType(mimeType)) {
return describeOmittedValue(value, "binary data");
}
}
if (typeof value === "bigint") {
return value.toString();
}
if (typeof value === "string") {
return redactInlineDataUris(redactStructuredTextValue(value));
}
if (typeof value === "function" || typeof value === "symbol" || value === undefined) {
return undefined;
}
if (!value || typeof value !== "object") {
return value;
}
if (seen.has(value)) {
return "[Circular]";
}
seen.add(value);
return value;
},
);
if (!serialized || serialized === "{}") {
return undefined;
}
return serialized;
} catch {
return undefined;
}
}
function truncateProviderToolText(text: string): string {
if (text.length <= PROVIDER_TOOL_RESULT_MAX_CHARS) {
return text;
}
return `${truncateUtf16Safe(text, PROVIDER_TOOL_RESULT_MAX_CHARS)}\n…(truncated)…`;
}
export function describeToolResultMediaPlaceholder(blocks: readonly unknown[]): string | undefined {
let hasImage = false;
let hasAudio = false;
for (const block of blocks) {
if (!block || typeof block !== "object") {
continue;
}
const record = block as Record<string, unknown>;
const type = typeof record.type === "string" ? record.type : undefined;
const mimeType = readMimeType(record);
if (
(type && IMAGE_TOOL_RESULT_TYPES.has(type)) ||
mimeType?.toLowerCase().startsWith("image/")
) {
hasImage = true;
}
if (
(type && AUDIO_TOOL_RESULT_TYPES.has(type)) ||
mimeType?.toLowerCase().startsWith("audio/")
) {
hasAudio = true;
}
}
if (hasImage && hasAudio) {
return "(see attached media)";
}
if (hasAudio) {
return "(see attached audio)";
}
if (hasImage) {
return "(see attached image)";
}
return undefined;
}
export function extractToolResultBlockText(block: unknown): string | undefined {
if (!block || typeof block !== "object") {
return undefined;
}
const record = block as Record<string, unknown>;
if (typeof record.type === "string" && MEDIA_ONLY_TOOL_RESULT_TYPES.has(record.type)) {
return undefined;
}
if (record.type === "text") {
const text = typeof record.text === "string" ? record.text : "";
return text ? sanitizeSurrogates(text) : undefined;
}
const structured = stringifyStructuredBlock(record);
return structured ? sanitizeSurrogates(truncateProviderToolText(structured)) : undefined;
}
export function extractToolResultText(blocks: readonly unknown[]): string {
const explicitTexts: string[] = [];
const structuredTexts: string[] = [];
for (const block of blocks) {
const text = extractToolResultBlockText(block);
if (!text) {
continue;
}
const record = block as Record<string, unknown>;
if (record.type === "text") {
explicitTexts.push(text);
} else {
structuredTexts.push(text);
}
}
if (explicitTexts.length > 0) {
return sanitizeSurrogates(explicitTexts.join("\n"));
}
return sanitizeSurrogates(truncateProviderToolText(structuredTexts.join("\n")));
}

View File

@@ -0,0 +1,131 @@
/** JSON-safe schema value used when projecting runtime tool parameters. */
export type RuntimeToolInputSchemaJson =
| null
| boolean
| number
| string
| RuntimeToolInputSchemaJson[]
| { [key: string]: RuntimeToolInputSchemaJson };
/** Projected runtime tool schema plus validation violations. */
export type RuntimeToolInputSchemaProjection = {
readonly schema: RuntimeToolInputSchemaJson;
readonly violations: readonly string[];
};
function isJsonValue(value: unknown): value is RuntimeToolInputSchemaJson {
if (value === null) {
return true;
}
switch (typeof value) {
case "boolean":
case "number":
case "string":
return true;
case "object":
if (Array.isArray(value)) {
return value.every(isJsonValue);
}
return Object.values(value as Record<string, unknown>).every(isJsonValue);
default:
return false;
}
}
function isJsonObject(value: RuntimeToolInputSchemaJson): value is {
[key: string]: RuntimeToolInputSchemaJson;
} {
return value !== null && typeof value === "object" && !Array.isArray(value);
}
function serializeToolInputSchema(value: unknown, path: string): RuntimeToolInputSchemaProjection {
let text: string | undefined;
try {
text = JSON.stringify(value);
} catch {
return {
schema: {},
violations: [`${path} is not JSON-serializable`],
};
}
if (!text) {
return {
schema: {},
violations: [`${path} is not JSON-serializable`],
};
}
const parsed = JSON.parse(text) as unknown;
if (!isJsonValue(parsed)) {
return {
schema: {},
violations: [`${path} is not a JSON value`],
};
}
return {
schema: parsed,
violations: [],
};
}
const schemaMapKeywords = new Set([
"$defs",
"definitions",
"dependencies",
"dependentSchemas",
"patternProperties",
"properties",
]);
function findDynamicSchemaKeywordViolations(
schema: RuntimeToolInputSchemaJson,
path: string,
): string[] {
if (Array.isArray(schema)) {
return schema.flatMap((entry, index) =>
findDynamicSchemaKeywordViolations(entry, `${path}[${index}]`),
);
}
if (!isJsonObject(schema)) {
return [];
}
const violations: string[] = [];
for (const key of ["$dynamicRef", "$dynamicAnchor"] as const) {
if (key in schema) {
violations.push(`${path}.${key}`);
}
}
for (const [key, value] of Object.entries(schema)) {
if (!value || typeof value !== "object") {
continue;
}
if (schemaMapKeywords.has(key) && isJsonObject(value)) {
for (const [schemaName, childSchema] of Object.entries(value)) {
violations.push(
...findDynamicSchemaKeywordViolations(childSchema, `${path}.${key}.${schemaName}`),
);
}
} else {
violations.push(...findDynamicSchemaKeywordViolations(value, `${path}.${key}`));
}
}
return violations;
}
/** Projects one runtime tool input schema to JSON and reports runtime incompatibilities. */
export function projectRuntimeToolInputSchema(
schema: unknown,
path = "parameters",
): RuntimeToolInputSchemaProjection {
const projection = serializeToolInputSchema(schema, path);
const violations = [...projection.violations];
if (!isJsonObject(projection.schema)) {
violations.push(`${path} must be a JSON object schema`);
} else if (projection.schema.type !== undefined && projection.schema.type !== "object") {
violations.push(`${path}.type must be "object"`);
}
violations.push(...findDynamicSchemaKeywordViolations(projection.schema, path));
return {
schema: projection.schema,
violations,
};
}

View File

@@ -0,0 +1,257 @@
// Provider message transform helpers convert runtime messages to provider payloads.
import type {
Api,
AssistantMessage,
ImageContent,
Message,
Model,
TextContent,
ToolCall,
ToolResultMessage,
} from "../types.js";
import { resolveModelBoundThinkingReplayMode } from "./anthropic-model-contract.js";
const NON_VISION_USER_IMAGE_PLACEHOLDER = "(image omitted: model does not support images)";
const NON_VISION_TOOL_IMAGE_PLACEHOLDER = "(tool image omitted: model does not support images)";
function replaceImagesWithPlaceholder(
content: (TextContent | ImageContent)[],
placeholder: string,
): TextContent[] {
const result: TextContent[] = [];
let previousWasPlaceholder = false;
for (const block of content) {
if (block.type === "image") {
if (!previousWasPlaceholder) {
result.push({ type: "text", text: placeholder });
}
previousWasPlaceholder = true;
continue;
}
result.push(block);
previousWasPlaceholder = block.text === placeholder;
}
return result;
}
function downgradeUnsupportedImages<TApi extends Api>(
messages: Message[],
model: Model<TApi>,
): Message[] {
if (model.input.includes("image")) {
return messages;
}
return messages.map((msg) => {
if (msg.role === "user" && Array.isArray(msg.content)) {
return {
...msg,
content: replaceImagesWithPlaceholder(msg.content, NON_VISION_USER_IMAGE_PLACEHOLDER),
};
}
if (msg.role === "toolResult") {
return {
...msg,
content: replaceImagesWithPlaceholder(msg.content, NON_VISION_TOOL_IMAGE_PLACEHOLDER),
};
}
return msg;
});
}
/**
* Normalize tool call ID for cross-provider compatibility.
* OpenAI Responses API generates IDs that are 450+ chars with special characters like `|`.
* Anthropic APIs require IDs matching ^[a-zA-Z0-9_-]+$ (max 64 chars).
*/
export function transformMessages<TApi extends Api>(
messages: Message[],
model: Model<TApi>,
normalizeToolCallId?: (id: string, model: Model<TApi>, source: AssistantMessage) => string,
): Message[] {
// Build a map of original tool call IDs to normalized IDs
const toolCallIdMap = new Map<string, string>();
const imageAwareMessages = downgradeUnsupportedImages(messages, model);
// First pass: transform messages (unsupported image downgrade, thinking blocks, tool call ID normalization)
const transformed = imageAwareMessages.map((msg) => {
// User messages pass through unchanged
if (msg.role === "user") {
return msg;
}
// Handle toolResult messages - normalize toolCallId if we have a mapping
if (msg.role === "toolResult") {
const normalizedId = toolCallIdMap.get(msg.toolCallId);
if (normalizedId && normalizedId !== msg.toolCallId) {
return Object.assign({}, msg, { toolCallId: normalizedId });
}
return msg;
}
// Assistant messages need transformation check
if (msg.role === "assistant") {
const assistantMsg = msg;
const modelBoundThinkingReplayMode = resolveModelBoundThinkingReplayMode({
source: {
provider: assistantMsg.provider,
api: assistantMsg.api,
modelId: assistantMsg.model,
responseModelId: assistantMsg.responseModel,
},
target: {
provider: model.provider,
api: model.api,
modelId: model.id,
modelParams: model.params,
},
});
const isSameModel =
modelBoundThinkingReplayMode === "preserve" ||
(assistantMsg.provider === model.provider &&
assistantMsg.api === model.api &&
assistantMsg.model === model.id);
// Public plugin-sdk/llm exports transformMessages; keep accepting legacy
// assistant strings from external provider adapters even though session
// JSONL replay normalizes them at ingest.
const contentBlocks =
typeof assistantMsg.content === "string"
? [{ type: "text" as const, text: assistantMsg.content }]
: assistantMsg.content;
const transformedContent = contentBlocks.flatMap((block) => {
if (block.type === "thinking") {
if (modelBoundThinkingReplayMode === "drop") {
return [];
}
// Redacted thinking is opaque encrypted content, only valid for the same model.
// Drop it for cross-model to avoid API errors.
if (block.redacted) {
return isSameModel ? block : [];
}
// For same model: keep thinking blocks with signatures (needed for replay)
// even if the thinking text is empty (OpenAI encrypted reasoning)
if (isSameModel && block.thinkingSignature) {
return block;
}
// Skip empty thinking blocks, convert others to plain text
if (!block.thinking || block.thinking.trim() === "") {
return [];
}
if (isSameModel) {
return block;
}
return {
type: "text" as const,
text: block.thinking,
};
}
if (block.type === "text") {
if (isSameModel) {
return block;
}
return {
type: "text" as const,
text: block.text,
};
}
if (block.type === "toolCall") {
const toolCall = block;
let normalizedToolCall: ToolCall = toolCall;
if (!isSameModel && toolCall.thoughtSignature) {
normalizedToolCall = Object.assign({}, toolCall);
delete (normalizedToolCall as { thoughtSignature?: string }).thoughtSignature;
}
if (!isSameModel && normalizeToolCallId) {
const normalizedId = normalizeToolCallId(toolCall.id, model, assistantMsg);
if (normalizedId !== toolCall.id) {
toolCallIdMap.set(toolCall.id, normalizedId);
normalizedToolCall = Object.assign({}, normalizedToolCall, { id: normalizedId });
}
}
return normalizedToolCall;
}
return block;
});
return Object.assign({}, assistantMsg, { content: transformedContent });
}
return msg;
});
// Second pass: insert synthetic empty tool results for orphaned tool calls
// This preserves thinking signatures and satisfies API requirements
const result: Message[] = [];
let pendingToolCalls: ToolCall[] = [];
let existingToolResultIds = new Set<string>();
const insertSyntheticToolResults = () => {
if (pendingToolCalls.length > 0) {
for (const tc of pendingToolCalls) {
if (!existingToolResultIds.has(tc.id)) {
result.push({
role: "toolResult",
toolCallId: tc.id,
toolName: tc.name,
content: [{ type: "text", text: "No result provided" }],
isError: true,
timestamp: Date.now(),
} as ToolResultMessage);
}
}
pendingToolCalls = [];
existingToolResultIds = new Set();
}
};
for (const msg of transformed) {
if (msg.role === "assistant") {
// If we have pending orphaned tool calls from a previous assistant, insert synthetic results now
insertSyntheticToolResults();
// Skip errored/aborted assistant messages entirely.
// These are incomplete turns that shouldn't be replayed:
// - May have partial content (reasoning without message, incomplete tool calls)
// - Replaying them can cause API errors (e.g., OpenAI "reasoning without following item")
// - The model should retry from the last valid state
const assistantMsg = msg as AssistantMessage;
if (assistantMsg.stopReason === "error" || assistantMsg.stopReason === "aborted") {
continue;
}
// Track tool calls from this assistant message
const toolCalls = assistantMsg.content.filter((b) => b.type === "toolCall");
if (toolCalls.length > 0) {
pendingToolCalls = toolCalls;
existingToolResultIds = new Set();
}
result.push(msg);
} else if (msg.role === "toolResult") {
existingToolResultIds.add(msg.toolCallId);
result.push(msg);
} else if (msg.role === "user") {
// User message interrupts tool flow - insert synthetic results for orphaned calls
insertSyntheticToolResults();
result.push(msg);
} else {
result.push(msg);
}
}
// If the conversation ends with unresolved tool calls, synthesize results now.
insertSyntheticToolResults();
return result;
}

View File

@@ -0,0 +1,28 @@
/** Cleanup callback for resources tied to an LLM session or all sessions. */
export type SessionResourceCleanup = (sessionId?: string) => void;
// Process-local registry of cleanup hooks owned by LLM providers/transports.
const sessionResourceCleanups = new Set<SessionResourceCleanup>();
/** Registers a session-resource cleanup hook and returns an unregister function. */
export function registerSessionResourceCleanup(cleanup: SessionResourceCleanup): () => void {
sessionResourceCleanups.add(cleanup);
return () => {
sessionResourceCleanups.delete(cleanup);
};
}
/** Runs all registered cleanup hooks, aggregating failures after every hook has run. */
export function cleanupSessionResources(sessionId?: string): void {
const errors: unknown[] = [];
for (const cleanup of sessionResourceCleanups) {
try {
cleanup(sessionId);
} catch (error) {
errors.push(error);
}
}
if (errors.length > 0) {
throw new AggregateError(errors, "Failed to cleanup session resources");
}
}

59
packages/ai/src/stream.ts Normal file
View File

@@ -0,0 +1,59 @@
// LLM Runtime module implements stream behavior.
import type {
Api,
AssistantMessage,
AssistantMessageEventStreamContract,
Context,
Model,
ProviderStreamOptions,
SimpleStreamOptions,
StreamOptions,
} from "@openclaw/llm-core";
import { createApiRegistry, type ApiRegistry } from "./api-registry.js";
/** Creates an isolated LLM runtime backed by the supplied provider registry. */
export function createLlmRuntime(registry: ApiRegistry = createApiRegistry()) {
function resolveApiProvider(api: Api) {
const provider = registry.getApiProvider(api);
if (!provider) {
throw new Error(`No API provider registered for api: ${api}`);
}
return provider;
}
function stream<TApi extends Api>(
model: Model<TApi>,
context: Context,
options?: ProviderStreamOptions,
): AssistantMessageEventStreamContract {
return resolveApiProvider(model.api).stream(model, context, options as StreamOptions);
}
async function complete<TApi extends Api>(
model: Model<TApi>,
context: Context,
options?: ProviderStreamOptions,
): Promise<AssistantMessage> {
return stream(model, context, options).result();
}
function streamSimple<TApi extends Api>(
model: Model<TApi>,
context: Context,
options?: SimpleStreamOptions,
): AssistantMessageEventStreamContract {
return resolveApiProvider(model.api).streamSimple(model, context, options);
}
async function completeSimple<TApi extends Api>(
model: Model<TApi>,
context: Context,
options?: SimpleStreamOptions,
): Promise<AssistantMessage> {
return streamSimple(model, context, options).result();
}
return { registry, stream, complete, streamSimple, completeSimple };
}
export type LlmRuntime = ReturnType<typeof createLlmRuntime>;

2
packages/ai/src/types.ts Normal file
View File

@@ -0,0 +1,2 @@
/** Shared model, message, tool, and streaming contracts. */
export * from "@openclaw/llm-core";

View File

@@ -0,0 +1,22 @@
type EventSink<T> = {
push(event: T): void;
};
export function createDeferredEventBuffer<T>(sink: EventSink<T>, onBufferedEvent?: () => void) {
let events: T[] = [];
return {
push(event: T): void {
events.push(event);
onBufferedEvent?.();
},
flush(): void {
for (const event of events) {
sink.push(event);
}
events = [];
},
discard(): void {
events = [];
},
};
}

View File

@@ -0,0 +1,2 @@
/** Shared provider diagnostics. */
export * from "@openclaw/llm-core/diagnostics";

View File

@@ -0,0 +1,2 @@
/** Assistant message event stream implementation. */
export * from "@openclaw/llm-core/event-stream";

View File

@@ -0,0 +1,13 @@
/** Fast deterministic hash to shorten long strings */
export function shortHash(str: string): string {
let h1 = 0xdeadbeef;
let h2 = 0x41c6ce57;
for (let i = 0; i < str.length; i++) {
const ch = str.charCodeAt(i);
h1 = Math.imul(h1 ^ ch, 2654435761);
h2 = Math.imul(h2 ^ ch, 1597334677);
}
h1 = Math.imul(h1 ^ (h1 >>> 16), 2246822507) ^ Math.imul(h2 ^ (h2 >>> 13), 3266489909);
h2 = Math.imul(h2 ^ (h2 >>> 16), 2246822507) ^ Math.imul(h1 ^ (h1 >>> 13), 3266489909);
return (h2 >>> 0).toString(36) + (h1 >>> 0).toString(36);
}

View File

@@ -0,0 +1,8 @@
/** Converts a Headers object to a plain record for provider request handling. */
export function headersToRecord(headers: Headers): Record<string, string> {
const result: Record<string, string> = {};
for (const [key, value] of headers.entries()) {
result[key] = value;
}
return result;
}

View File

@@ -0,0 +1,153 @@
// JSON parse helpers recover structured values from partial model output.
import { parse as partialParse } from "partial-json";
const VALID_JSON_ESCAPES = new Set(['"', "\\", "/", "b", "f", "n", "r", "t", "u"]);
const JSON_CONTROL_ESCAPES = new Set(["b", "f", "n", "r", "t"]);
function isControlCharacter(char: string): boolean {
const codePoint = char.codePointAt(0);
return codePoint !== undefined && codePoint >= 0x00 && codePoint <= 0x1f;
}
function escapeControlCharacter(char: string): string {
switch (char) {
case "\b":
return "\\b";
case "\f":
return "\\f";
case "\n":
return "\\n";
case "\r":
return "\\r";
case "\t":
return "\\t";
default:
return `\\u${char.codePointAt(0)?.toString(16).padStart(4, "0") ?? "0000"}`;
}
}
/**
* Repairs malformed JSON string literals by:
* - escaping raw control characters inside strings
* - doubling backslashes before invalid escape characters
*/
export function repairJson(json: string): string {
let repaired = "";
let inString = false;
let stringValuePrefix = "";
for (let index = 0; index < json.length; index++) {
const char = json[index];
if (!inString) {
repaired += char;
if (char === '"') {
inString = true;
stringValuePrefix = "";
}
continue;
}
if (char === '"') {
repaired += char;
inString = false;
stringValuePrefix = "";
continue;
}
if (char === "\\") {
const nextChar = json[index + 1];
if (nextChar === undefined) {
repaired += "\\\\";
continue;
}
if (nextChar === "u") {
const unicodeDigits = json.slice(index + 2, index + 6);
if (/^[0-9a-fA-F]{4}$/.test(unicodeDigits)) {
repaired += `\\u${unicodeDigits}`;
stringValuePrefix += `\\u${unicodeDigits}`;
index += 5;
continue;
}
// A \u not followed by four hex digits is an invalid escape: double the
// backslash like the other invalid escapes below. Falling through would
// hit the valid-escape branch (VALID_JSON_ESCAPES contains "u") and
// re-emit the broken \u, leaving the JSON unparseable.
repaired += "\\\\";
stringValuePrefix += "\\";
continue;
}
if (JSON_CONTROL_ESCAPES.has(nextChar) && looksLikeWindowsPathPrefix(stringValuePrefix)) {
repaired += "\\\\";
stringValuePrefix += "\\";
continue;
}
if (VALID_JSON_ESCAPES.has(nextChar)) {
repaired += `\\${nextChar}`;
stringValuePrefix += nextChar === "\\" ? "\\" : `\\${nextChar}`;
index += 1;
continue;
}
repaired += "\\\\";
stringValuePrefix += "\\";
continue;
}
repaired += isControlCharacter(char) ? escapeControlCharacter(char) : char;
stringValuePrefix += char;
}
return repaired;
}
export function parseJsonWithRepair(json: string): unknown {
const repairedJson = repairJson(json);
if (repairedJson !== json) {
return JSON.parse(repairedJson) as unknown;
}
return JSON.parse(json) as unknown;
}
function looksLikeWindowsPathPrefix(prefix: string): boolean {
const tail = prefix.slice(-160);
return /(?:^|[^A-Za-z0-9])[A-Za-z]:(?:[\\/][^"\\/:*?<>|\r\n]*)*$/.test(tail);
}
function asStreamingJsonRecord(value: unknown): Record<string, unknown> {
return value !== null && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: {};
}
/**
* Attempts to parse potentially incomplete JSON during streaming.
* Always returns a valid object, even if the JSON is incomplete.
*
* @param partialJson The partial JSON string from streaming
* @returns Parsed object or empty object if parsing fails
*/
export function parseStreamingJson(partialJson: string | undefined): Record<string, unknown> {
if (!partialJson || partialJson.trim() === "") {
return {};
}
try {
return asStreamingJsonRecord(parseJsonWithRepair(partialJson));
} catch {
try {
const result = partialParse(partialJson);
return asStreamingJsonRecord(result);
} catch {
try {
const result = partialParse(repairJson(partialJson));
return asStreamingJsonRecord(result);
} catch {
return {};
}
}
}
}

View File

@@ -0,0 +1,23 @@
const requestActivityListeners = new WeakMap<AbortSignal, Set<() => void>>();
export function notifyLlmRequestActivity(signal: AbortSignal | undefined): void {
if (!signal) {
return;
}
for (const listener of requestActivityListeners.get(signal) ?? []) {
listener();
}
}
export function onLlmRequestActivity(signal: AbortSignal, listener: () => void): () => void {
const listeners = requestActivityListeners.get(signal) ?? new Set<() => void>();
listeners.add(listener);
requestActivityListeners.set(signal, listeners);
return () => {
listeners.delete(listener);
if (listeners.size === 0) {
requestActivityListeners.delete(signal);
}
};
}

View File

@@ -0,0 +1,32 @@
// OpenAI ChatGPT JWT helpers inspect auth claims for ChatGPT OAuth sessions.
const OPENAI_CODEX_AUTH_CLAIM = "https://api.openai.com/auth";
export type OpenAICodexJwtPayload = {
[OPENAI_CODEX_AUTH_CLAIM]?: {
chatgpt_account_id?: unknown;
};
[key: string]: unknown;
};
export function decodeOpenAICodexJwtPayload(token: string): OpenAICodexJwtPayload | null {
const parts = token.split(".");
if (parts.length !== 3) {
return null;
}
try {
const decoded = Buffer.from(parts[1] ?? "", "base64url").toString("utf8");
const parsed = JSON.parse(decoded);
return parsed && typeof parsed === "object" && !Array.isArray(parsed)
? (parsed as OpenAICodexJwtPayload)
: null;
} catch {
return null;
}
}
export function resolveOpenAICodexAccountId(token: string): string | null {
const accountId =
decodeOpenAICodexJwtPayload(token)?.[OPENAI_CODEX_AUTH_CLAIM]?.chatgpt_account_id;
return typeof accountId === "string" && accountId.length > 0 ? accountId : null;
}

View File

@@ -0,0 +1,72 @@
import { describe, expect, it } from "vitest";
import type { AssistantMessage } from "../types.js";
import { isConfiguredContextSizeOverflowError, isContextOverflow } from "./overflow.js";
function errorMessage(message: string): AssistantMessage {
return {
role: "assistant",
content: [],
api: "test-api",
provider: "test-provider",
model: "test-model",
usage: {
input: 0,
output: 0,
cacheRead: 0,
cacheWrite: 0,
totalTokens: 0,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "error",
errorMessage: message,
timestamp: 1,
};
}
function successfulMessage(
contextUsage?: AssistantMessage["usage"]["contextUsage"],
): AssistantMessage {
return {
...errorMessage(""),
usage: {
input: 12,
output: 15_104,
cacheRead: 1_100_000,
cacheWrite: 93_130,
...(contextUsage ? { contextUsage } : {}),
totalTokens: 1_208_246,
cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 },
},
stopReason: "stop",
errorMessage: undefined,
};
}
describe("configured context size overflow", () => {
it.each([
"400 Prompt has 256468 tokens, but the configured context size is 256000 tokens",
"Prompt has 5,958,968 tokens, but the configured context size is 256,000 tokens",
])("detects %s", (text) => {
expect(isConfiguredContextSizeOverflowError(text)).toBe(true);
expect(isContextOverflow(errorMessage(text), 256_000)).toBe(true);
});
});
describe("usage-based overflow", () => {
it("prefers an available context snapshot over aggregate billing usage", () => {
expect(
isContextOverflow(
successfulMessage({
state: "available",
promptTokens: 148_874,
totalTokens: 163_978,
}),
1_000_000,
),
).toBe(false);
});
it("does not infer overflow from aggregate billing when context is unavailable", () => {
expect(isContextOverflow(successfulMessage({ state: "unavailable" }), 1_000_000)).toBe(false);
});
});

View File

@@ -0,0 +1,171 @@
// Overflow helpers classify provider overflow errors and retryable responses.
import type { AssistantMessage } from "../types.js";
const CONFIGURED_CONTEXT_SIZE_OVERFLOW_RE =
/prompt has [\d,]+ tokens?, but the configured context size is [\d,]+ tokens?/i;
/** Detects DS4-style raw token-count context overflow errors. */
export function isConfiguredContextSizeOverflowError(errorMessage: string): boolean {
return CONFIGURED_CONTEXT_SIZE_OVERFLOW_RE.test(errorMessage);
}
/**
* Regex patterns to detect context overflow errors from different providers.
*
* These patterns match error messages returned when the input exceeds
* the model's context window.
*
* Provider-specific patterns (with example error messages):
*
* - Anthropic: "prompt is too long: 213462 tokens > 200000 maximum"
* - Anthropic: "413 {\"error\":{\"type\":\"request_too_large\",\"message\":\"Request exceeds the maximum size\"}}"
* - OpenAI: "Your input exceeds the context window of this model"
* - OpenAI/LiteLLM: "Requested token count exceeds the model's maximum context length of 131072 tokens"
* - Google: "The input token count (1196265) exceeds the maximum number of tokens allowed (1048575)"
* - xAI: "This model's maximum prompt length is 131072 but the request contains 537812 tokens"
* - Groq: "Please reduce the length of the messages or completion"
* - OpenRouter: "This endpoint's maximum context length is X tokens. However, you requested about Y tokens"
* - Together AI: "The input (X tokens) is longer than the model's context length (Y tokens)."
* - llama.cpp: "the request exceeds the available context size, try increasing it"
* - LM Studio: "tokens to keep from the initial prompt is greater than the context length"
* - GitHub Copilot: "prompt token count of X exceeds the limit of Y"
* - MiniMax: "invalid params, context window exceeds limit"
* - Kimi For Coding: "Your request exceeded model token limit: X (requested: Y)"
* - Cerebras: "400/413 status code (no body)"
* - Mistral: "Prompt contains X tokens ... too large for model with Y maximum context length"
* - z.ai: Does NOT error, accepts overflow silently - handled via usage.input > contextWindow
* - Xiaomi MiMo: Truncates input to fill contextWindow exactly, then returns finish_reason "length"
* with output=0 (no room left to generate). Detected via stopReason "length" + zero output +
* input filling the context window.
* - Ollama: Some deployments truncate silently, others return errors like "prompt too long; exceeded max context length by X tokens"
*/
const OVERFLOW_PATTERNS = [
/prompt is too long/i, // Anthropic token overflow
/request_too_large/i, // Anthropic request byte-size overflow (HTTP 413)
/input is too long for requested model/i, // Amazon Bedrock
/exceeds the context window/i, // OpenAI (Completions & Responses API)
/exceeds (?:the )?(?:model'?s )?maximum context length of [\d,]+ tokens?/i, // OpenAI-compatible proxies (LiteLLM)
/input token count.*exceeds the maximum/i, // Google (Gemini)
/maximum prompt length is \d+/i, // xAI (Grok)
/reduce the length of the messages/i, // Groq
/maximum context length is \d+ tokens/i, // OpenRouter (all backends)
/input \(\d+ tokens\) is longer than the model'?s context length \(\d+ tokens\)/i, // Together AI
/exceeds the limit of \d+/i, // GitHub Copilot
/exceeds the available context size/i, // llama.cpp server
/greater than the context length/i, // LM Studio
/context window exceeds limit/i, // MiniMax
/exceeded model token limit/i, // Kimi For Coding
/too large for model with \d+ maximum context length/i, // Mistral
CONFIGURED_CONTEXT_SIZE_OVERFLOW_RE, // DS4 server
/model_context_window_exceeded/i, // z.ai non-standard finish_reason surfaced as error text
/prompt too long; exceeded (?:max )?context length/i, // Ollama explicit overflow error
/context[_ ]length[_ ]exceeded/i, // Generic fallback
/too many tokens/i, // Generic fallback
/token limit exceeded/i, // Generic fallback
/^4(?:00|13)\s*(?:status code)?\s*\(no body\)/i, // Cerebras: 400/413 with no body
];
/**
* Patterns that indicate non-overflow errors (e.g. rate limiting, server errors).
* Error messages matching unknown of these are excluded from overflow detection
* even if they also match an OVERFLOW_PATTERN.
*
* Example: Bedrock formats throttling errors as "ThrottlingException: Too many tokens,
* please wait before trying again." which would match the /too many tokens/i overflow
* pattern without this exclusion.
*/
const NON_OVERFLOW_PATTERNS = [
/^(Throttling error|Service unavailable):/i, // AWS Bedrock non-overflow errors (human-readable prefixes from formatBedrockError)
/rate limit/i, // Generic rate limiting
/too many requests/i, // Generic HTTP 429 style
];
function resolveContextInputTokens(message: AssistantMessage): number | undefined {
if (message.usage.contextUsage?.state === "available") {
return message.usage.contextUsage.promptTokens;
}
if (message.usage.contextUsage?.state === "unavailable") {
return undefined;
}
return message.usage.input + message.usage.cacheRead;
}
/**
* Check if an assistant message represents a context overflow error.
*
* This handles two cases:
* 1. Error-based overflow: Most providers return stopReason "error" with a
* specific error message pattern.
* 2. Silent overflow: Some providers accept overflow requests and return
* successfully. For these, we check if usage.input exceeds the context window.
*
* ## Reliability by Provider
*
* **Reliable detection (returns error with detectable message):**
* - Anthropic: "prompt is too long: X tokens > Y maximum" or "request_too_large"
* - OpenAI (Completions & Responses): "exceeds the context window" or "exceeds the model's maximum context length of X tokens"
* - Google Gemini: "input token count exceeds the maximum"
* - xAI (Grok): "maximum prompt length is X but request contains Y"
* - Groq: "reduce the length of the messages"
* - Cerebras: 400/413 status code (no body)
* - Mistral: "Prompt contains X tokens ... too large for model with Y maximum context length"
* - OpenRouter (all backends): "maximum context length is X tokens"
* - Together AI: "The input (X tokens) is longer than the model's context length (Y tokens)."
* - llama.cpp: "exceeds the available context size"
* - LM Studio: "greater than the context length"
* - Kimi For Coding: "exceeded model token limit: X (requested: Y)"
*
* **Unreliable detection:**
* - z.ai: Sometimes accepts overflow silently (detectable via usage.input > contextWindow),
* sometimes returns rate limit errors. Pass contextWindow param to detect silent overflow.
* - Xiaomi MiMo: Truncates input to fit contextWindow then returns stopReason "length" with
* output=0. Pass contextWindow param to detect via the "filled context + zero output" signal.
* - Ollama: May truncate input silently for some setups, but may also return explicit
* overflow errors that match the patterns above. Silent truncation still cannot be
* detected here because we do not know the expected token count.
*
* ## Custom Providers
*
* If you've added custom models via settings.json, this function may not detect
* overflow errors from those providers. To add support:
*
* 1. Send a request that exceeds the model's context window
* 2. Check the errorMessage in the response
* 3. Create a regex pattern that matches the error
* 4. The pattern should be added to OVERFLOW_PATTERNS in this file, or
* check the errorMessage yourself before calling this function
*
* @param message - The assistant message to check
* @param contextWindow - Optional context window size for detecting silent overflow (z.ai)
* @returns true if the message indicates a context overflow
*/
export function isContextOverflow(message: AssistantMessage, contextWindow?: number): boolean {
// Case 1: Check error message patterns
if (message.stopReason === "error" && message.errorMessage) {
// Skip messages matching known non-overflow patterns (e.g. throttling / rate-limit)
const isNonOverflow = NON_OVERFLOW_PATTERNS.some((p) => p.test(message.errorMessage!));
if (!isNonOverflow && OVERFLOW_PATTERNS.some((p) => p.test(message.errorMessage!))) {
return true;
}
}
// Case 2: Silent overflow (z.ai style) - successful but usage exceeds context
if (contextWindow && message.stopReason === "stop") {
const inputTokens = resolveContextInputTokens(message);
if (inputTokens !== undefined && inputTokens > contextWindow) {
return true;
}
}
// Case 3: Length-stop overflow (Xiaomi MiMo style) - server truncates oversized input
// to fit the context window, leaving no room for output. Returns stopReason "length"
// with output=0 and input+cacheRead filling the context window.
if (contextWindow && message.stopReason === "length" && message.usage.output === 0) {
const inputTokens = resolveContextInputTokens(message);
if (inputTokens !== undefined && inputTokens >= contextWindow * 0.99) {
return true;
}
}
return false;
}

View File

@@ -0,0 +1,29 @@
/**
* Prompt-cache normalization helpers. They keep generated prompt sections
* deterministic across platform newlines, trailing whitespace, and input
* ordering.
*/
import { normalizeLowercaseStringOrEmpty } from "@openclaw/normalization-core/string-coerce";
/** Normalize structured prompt text before hashing or snapshot comparison. */
export function normalizeStructuredPromptSection(text: string): string {
return text
.replace(/\r\n?/g, "\n")
.replace(/[ \t]+$/gm, "")
.trim();
}
/** Normalize, de-dupe, and sort capability ids for stable prompt payloads. */
export function normalizePromptCapabilityIds(capabilities: ReadonlyArray<string>): string[] {
const seen = new Set<string>();
const normalized: string[] = [];
for (const capability of capabilities) {
const value = normalizeLowercaseStringOrEmpty(normalizeStructuredPromptSection(capability));
if (!value || seen.has(value)) {
continue;
}
seen.add(value);
normalized.push(value);
}
return normalized.toSorted((left, right) => left.localeCompare(right));
}

View File

@@ -0,0 +1,297 @@
// Reasoning tag partitioner tests cover splitting reasoning and visible text segments.
import { describe, expect, it } from "vitest";
import { createReasoningTagTextPartitioner } from "./reasoning-tag-text-partitioner.js";
describe("createReasoningTagTextPartitioner", () => {
it("routes split inline reasoning tags away from visible text", () => {
const partitioner = createReasoningTagTextPartitioner();
const deltas = [
...partitioner.push("before <thi"),
...partitioner.push("nk>hidden"),
...partitioner.push("</think> after"),
...partitioner.flush(),
];
expect(deltas).toEqual([
{ kind: "text", text: "before " },
{ kind: "thinking", text: "hidden" },
{ kind: "text", text: " after" },
]);
});
it("keeps unterminated reasoning as thinking on flush", () => {
const partitioner = createReasoningTagTextPartitioner();
expect([...partitioner.push("visible <reasoning>hidden tail"), ...partitioner.flush()]).toEqual(
[
{ kind: "text", text: "visible " },
{ kind: "thinking", text: "hidden tail" },
],
);
});
it("emits ordinary angle-bracket text immediately", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.push("<div>visible")).toEqual([{ kind: "text", text: "<div>visible" }]);
expect(partitioner.flush()).toEqual([]);
});
it("reports pending partial tags and active reasoning", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.hasPending()).toBe(false);
expect(partitioner.push("before <thi")).toEqual([{ kind: "text", text: "before " }]);
expect(partitioner.hasPending()).toBe(true);
expect(partitioner.push("nk>hidden")).toEqual([{ kind: "thinking", text: "hidden" }]);
expect(partitioner.hasPending()).toBe(true);
expect(partitioner.isInsideReasoning()).toBe(true);
expect(partitioner.push("</think> after")).toEqual([{ kind: "text", text: " after" }]);
expect(partitioner.hasPending()).toBe(false);
expect(partitioner.isInsideReasoning()).toBe(false);
});
it("holds possible reasoning opens in visible mode until they are resolved", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("before <thi")).toEqual([{ kind: "text", text: "before " }]);
expect(partitioner.push("nk>hidden")).toEqual([{ kind: "thinking", text: "hidden" }]);
});
it("strips complete reasoning tags in visible mode", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Use <think>literal</think> here")).toEqual([
{ kind: "text", text: "Use " },
{ kind: "thinking", text: "literal" },
{ kind: "text", text: " here" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("keeps split reasoning tags with attributes out of visible text", () => {
const partitioner = createReasoningTagTextPartitioner();
const deltas = [
...partitioner.pushVisible("Before <think "),
...partitioner.pushVisible("id='x'>secret</think> after"),
...partitioner.flush(),
];
expect(deltas).toEqual([
{ kind: "text", text: "Before " },
{ kind: "thinking", text: "secret" },
{ kind: "text", text: " after" },
]);
});
it("keeps split antml reasoning tags out of visible text", () => {
const partitioner = createReasoningTagTextPartitioner();
const deltas = [
...partitioner.pushVisible("Before <antml:reas"),
...partitioner.pushVisible("oning>secret</antml:reasoning> after"),
...partitioner.flush(),
];
expect(deltas).toEqual([
{ kind: "text", text: "Before " },
{ kind: "thinking", text: "secret" },
{ kind: "text", text: " after" },
]);
});
it("keeps split mm reasoning tags out of visible text", () => {
const partitioner = createReasoningTagTextPartitioner();
const deltas = [
...partitioner.push("Before <mm:thi"),
...partitioner.push("nk>secret</mm:think> after"),
...partitioner.flush(),
];
expect(deltas).toEqual([
{ kind: "text", text: "Before " },
{ kind: "thinking", text: "secret" },
{ kind: "text", text: " after" },
]);
});
it("keeps nested reasoning hidden until the outer tag closes", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(
partitioner.pushVisible("<think>outer <think>inner</think> still outer</think>visible"),
).toEqual([
{ kind: "thinking", text: "outer inner still outer" },
{ kind: "text", text: "visible" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("drops malformed reasoning before orphan close tags in strict mode", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.push("private chain of thought </think> Visible answer")).toEqual([
{ kind: "text", text: " Visible answer" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("keeps unmatched close-tag prose visible in visible mode", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Use </think> to close the tag")).toEqual([
{ kind: "text", text: "Use </think> to close the tag" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("buffers split orphan close tags until the visible suffix arrives", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.push("private chain of thought </think>")).toEqual([]);
expect(partitioner.push(" Visible answer")).toEqual([
{ kind: "text", text: " Visible answer" },
]);
});
it("buffers split orphan close tag prefixes with their hidden prefix", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.push("private chain of thought </thi")).toEqual([]);
expect(partitioner.push("nk> Visible answer")).toEqual([
{ kind: "text", text: " Visible answer" },
]);
});
it("keeps close tags inside hidden code fences private", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("<think>\n```ts\nliteral ")).toEqual([]);
expect(partitioner.pushVisible("</think> still private")).toEqual([]);
expect(partitioner.flush()).toEqual([
{ kind: "thinking", text: "\n```ts\nliteral </think> still private" },
]);
});
it("recovers fully wrapped unclosed visible-mode text on flush", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("<think>Visible answer from a malformed local model")).toEqual(
[],
);
expect(partitioner.flush()).toEqual([
{ kind: "text", text: "Visible answer from a malformed local model" },
]);
});
it("keeps unclosed trailing tags as visible prose in visible mode", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Use <think> only in this mode")).toEqual([
{ kind: "text", text: "Use " },
]);
expect(partitioner.flush()).toEqual([{ kind: "text", text: "<think> only in this mode" }]);
});
it("does not treat code-span reasoning tag examples as hidden reasoning", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.push("Use `<think>literal</think>` here")).toEqual([
{ kind: "text", text: "Use `<think>literal</think>` here" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("preserves split code-span reasoning tag examples when active routing begins", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Use `<thi")).toEqual([{ kind: "text", text: "Use `<thi" }]);
expect(partitioner.push("nk>literal</think>` here")).toEqual([
{ kind: "text", text: "nk>literal</think>` here" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("preserves code-span reasoning tag examples when the closing backtick arrives later", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Use `<think>literal</think>")).toEqual([
{ kind: "text", text: "Use " },
]);
expect(partitioner.pushVisible("` here")).toEqual([
{ kind: "text", text: "`<think>literal</think>` here" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("preserves code-span reasoning tag examples when the stream splits after the opener", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Use `")).toEqual([{ kind: "text", text: "Use `" }]);
expect(partitioner.pushVisible("<think>literal</think>` here")).toEqual([
{ kind: "text", text: "<think>literal</think>` here" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("preserves multi-backtick code-span reasoning tag examples across chunks", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Use ``<think>")).toEqual([{ kind: "text", text: "Use " }]);
expect(partitioner.pushVisible("literal</think>`` here")).toEqual([
{ kind: "text", text: "``<think>literal</think>`` here" },
]);
expect(partitioner.flush()).toEqual([]);
});
it("reclassifies reasoning tags inside unclosed inline code on final flush", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Start `unclosed <think>secret</think> end")).toEqual([
{ kind: "text", text: "Start " },
]);
expect(partitioner.flush()).toEqual([
{ kind: "text", text: "`unclosed " },
{ kind: "thinking", text: "secret" },
{ kind: "text", text: " end" },
]);
});
it("keeps buffered unclosed reasoning hidden after strict mode is marked", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("<think>secret")).toEqual([]);
partitioner.markStrict();
expect(partitioner.flush()).toEqual([{ kind: "thinking", text: "secret" }]);
});
it("preserves fenced reasoning tag examples when the stream splits after the fence", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("Example:\n```")).toEqual([
{ kind: "text", text: "Example:\n```" },
]);
expect(partitioner.pushVisible("\n<think>literal</think>\n```\nDone.")).toEqual([
{ kind: "text", text: "\n<think>literal</think>\n```\nDone." },
]);
expect(partitioner.flush()).toEqual([]);
});
it("preserves fenced reasoning tag examples when the fence marker is split", () => {
const partitioner = createReasoningTagTextPartitioner();
expect(partitioner.pushVisible("``")).toEqual([]);
expect(
partitioner.pushVisible(
"`xml\n<thinking>literal</thinking>\n```\n<think>secret</think>answer",
),
).toEqual([
{
kind: "text",
text: "```xml\n<thinking>literal</thinking>\n```\n",
},
{ kind: "thinking", text: "secret" },
{ kind: "text", text: "answer" },
]);
expect(partitioner.flush()).toEqual([]);
});
});

View File

@@ -0,0 +1,364 @@
// Reasoning tag partitioner helpers split text around reasoning tag regions.
import {
buildCodeSpanIndex,
createInlineCodeState,
type InlineCodeState,
} from "@openclaw/markdown-core/code-spans";
import type { FenceScanState } from "@openclaw/markdown-core/fences";
export type ReasoningTagTextDelta =
| { kind: "text"; text: string }
| { kind: "thinking"; text: string };
const REASONING_TAG_RE =
/<\s*(\/?)\s*(?:(?:antml:|mm:)?(?:think(?:ing)?|thought|reasoning)|antthinking)\b[^<>]*>/gi;
const REASONING_TAG_NAMES = [
"think",
"thinking",
"thought",
"reasoning",
"antthinking",
"antml:think",
"antml:thinking",
"antml:thought",
"antml:reasoning",
"mm:think",
"mm:thinking",
"mm:thought",
"mm:reasoning",
] as const;
export interface ReasoningTagTextPartitioner {
markStrict(): void;
push(chunk: string): ReasoningTagTextDelta[];
pushVisible(chunk: string): ReasoningTagTextDelta[];
flush(): ReasoningTagTextDelta[];
hasPending(): boolean;
isInsideReasoning(): boolean;
}
export function createReasoningTagTextPartitioner(): ReasoningTagTextPartitioner {
let buffer = "";
let reasoningDepth = 0;
let strictMode = false;
let emittedVisibleText = false;
let inlineCodeState: InlineCodeState = createInlineCodeState();
let fenceState: FenceScanState | undefined;
let hiddenInlineCodeState: InlineCodeState = createInlineCodeState();
let hiddenFenceState: FenceScanState | undefined;
let recoverableOpenTagText: string | undefined;
const consume = (final: boolean, recoverFullUnclosed: boolean): ReasoningTagTextDelta[] => {
const output: ReasoningTagTextDelta[] = [];
const emit = (kind: ReasoningTagTextDelta["kind"], text: string) => {
if (!text) {
return;
}
if (kind === "text" && text.trim().length > 0) {
emittedVisibleText = true;
}
if (kind === "text") {
const nextCode = buildCodeSpanIndex(text, inlineCodeState, fenceState);
inlineCodeState = nextCode.inlineState;
fenceState = nextCode.fenceState;
} else {
const nextCode = buildCodeSpanIndex(text, hiddenInlineCodeState, hiddenFenceState);
hiddenInlineCodeState = nextCode.inlineState;
hiddenFenceState = nextCode.fenceState;
}
const previous = output[output.length - 1];
if (previous?.kind === kind) {
previous.text += text;
return;
}
output.push({ kind, text });
};
while (buffer) {
const activeInlineCodeState = reasoningDepth === 0 ? inlineCodeState : hiddenInlineCodeState;
const activeFenceState = reasoningDepth === 0 ? fenceState : hiddenFenceState;
const codeSpans = buildCodeSpanIndex(buffer, activeInlineCodeState, activeFenceState);
const hasUnclosedCode =
reasoningDepth === 0 && Boolean(codeSpans.inlineState.open || codeSpans.fenceState.open);
const hasRawReasoning = hasRawReasoningTag(buffer);
const tag = findNextReasoningTag(buffer, (index) =>
final && hasUnclosedCode && hasRawReasoning ? false : codeSpans.isInside(index),
);
if (!tag) {
if (final) {
const recoverAsText =
reasoningDepth > 0 && recoverFullUnclosed && !hasRawReasoningCloseTag(buffer);
const recoveredText =
recoverAsText && recoverableOpenTagText ? recoverableOpenTagText + buffer : buffer;
emit(reasoningDepth > 0 && !recoverAsText ? "thinking" : "text", recoveredText);
buffer = "";
reasoningDepth = 0;
recoverableOpenTagText = undefined;
return output;
}
if (
reasoningDepth > 0 &&
recoverFullUnclosed &&
(!emittedVisibleText || recoverableOpenTagText)
) {
return output;
}
if (hasUnclosedCode && hasRawReasoning) {
const openCodeIndex =
inlineCodeState.open || fenceState?.open ? 0 : findOpenCodeContextStart(buffer);
if (openCodeIndex !== -1) {
emit("text", buffer.slice(0, openCodeIndex));
buffer = buffer.slice(openCodeIndex);
return output;
}
}
const trailingFenceStart = findTrailingFenceFragmentStart(
buffer,
activeInlineCodeState,
activeFenceState,
);
if (trailingFenceStart !== -1) {
emit(reasoningDepth > 0 ? "thinking" : "text", buffer.slice(0, trailingFenceStart));
buffer = buffer.slice(trailingFenceStart);
return output;
}
const keepFrom = reasoningTagPrefixSuffixIndex(buffer, (index) =>
codeSpans.isInside(index),
);
if (keepFrom === -1) {
emit(reasoningDepth > 0 ? "thinking" : "text", buffer);
buffer = "";
return output;
}
if (
reasoningDepth === 0 &&
keepFrom > 0 &&
buffer.slice(0, keepFrom).trim().length > 0 &&
isReasoningCloseTagPrefix(buffer.slice(keepFrom))
) {
return output;
}
if (keepFrom > 0) {
emit(reasoningDepth > 0 ? "thinking" : "text", buffer.slice(0, keepFrom));
buffer = buffer.slice(keepFrom);
}
return output;
}
const beforeTag = buffer.slice(0, tag.index);
const afterTag = buffer.slice(tag.index + tag.text.length);
if (tag.isClose && reasoningDepth === 0) {
if (recoverFullUnclosed && beforeTag.trim().length > 0 && afterTag.trim().length > 0) {
emit("text", beforeTag + tag.text);
buffer = afterTag;
continue;
}
if (beforeTag.trim().length > 0 && afterTag.trim().length === 0 && !final) {
return output;
}
if (beforeTag.trim().length === 0 || afterTag.trim().length === 0) {
emit("text", beforeTag);
}
buffer = afterTag;
continue;
}
emit(reasoningDepth > 0 ? "thinking" : "text", buffer.slice(0, tag.index));
buffer = afterTag;
if (tag.isClose) {
reasoningDepth = Math.max(0, reasoningDepth - 1);
if (reasoningDepth === 0) {
recoverableOpenTagText = undefined;
hiddenInlineCodeState = createInlineCodeState();
hiddenFenceState = undefined;
}
} else {
if (reasoningDepth === 0) {
recoverableOpenTagText = recoverFullUnclosed && emittedVisibleText ? tag.text : undefined;
hiddenInlineCodeState = createInlineCodeState();
hiddenFenceState = undefined;
}
reasoningDepth += 1;
}
}
return output;
};
return {
markStrict() {
strictMode = true;
},
push(chunk: string) {
strictMode = true;
buffer += chunk;
return consume(false, false);
},
pushVisible(chunk: string) {
buffer += chunk;
return consume(false, true);
},
flush() {
return consume(true, !strictMode);
},
hasPending() {
return buffer.length > 0 || reasoningDepth > 0;
},
isInsideReasoning() {
return reasoningDepth > 0;
},
};
}
function hasRawReasoningTag(text: string): boolean {
REASONING_TAG_RE.lastIndex = 0;
return REASONING_TAG_RE.test(text);
}
function hasRawReasoningCloseTag(text: string): boolean {
REASONING_TAG_RE.lastIndex = 0;
for (;;) {
const match = REASONING_TAG_RE.exec(text);
if (!match) {
return false;
}
if (match[1] === "/") {
return true;
}
}
}
function findNextReasoningTag(
text: string,
isIndexInsideCode: (index: number) => boolean,
): { index: number; text: string; isClose: boolean } | null {
REASONING_TAG_RE.lastIndex = 0;
for (;;) {
const match = REASONING_TAG_RE.exec(text);
if (!match) {
return null;
}
if (!isIndexInsideCode(match.index)) {
return {
index: match.index,
text: match[0],
isClose: match[1] === "/",
};
}
}
}
function reasoningTagPrefixSuffixIndex(
text: string,
isIndexInsideCode: (index: number) => boolean,
): number {
for (let index = text.lastIndexOf("<"); index >= 0; ) {
if (!isIndexInsideCode(index) && isReasoningTagPrefix(text.slice(index))) {
return index;
}
if (index === 0) {
break;
}
index = text.lastIndexOf("<", index - 1);
}
return -1;
}
function isReasoningTagPrefix(text: string): boolean {
const name = normalizeReasoningTagPrefixName(text);
return REASONING_TAG_NAMES.some((tagName) => {
if (tagName.startsWith(name)) {
return true;
}
if (!name.startsWith(tagName)) {
return false;
}
const rest = name.slice(tagName.length);
return rest.length === 0 || /^[\s/>]/.test(rest);
});
}
function isReasoningCloseTagPrefix(text: string): boolean {
const normalized = text
.replace(/^<\s*/, "<")
.replace(/^<\s*\//, "</")
.replace(/^<\/\s*/, "</")
.toLowerCase();
return normalized.startsWith("</") && isReasoningTagPrefix(text);
}
function normalizeReasoningTagPrefixName(text: string): string {
const normalized = text
.replace(/^<\s*/, "<")
.replace(/^<\s*\//, "</")
.replace(/^<\/\s*/, "</")
.toLowerCase();
const rawName = normalized.startsWith("</") ? normalized.slice(2) : normalized.slice(1);
return rawName.trimStart();
}
function findOpenCodeContextStart(text: string): number {
const fence = findOpenFenceStart(text);
const inline = findOpenInlineCodeStart(text);
if (fence === -1) {
return inline;
}
if (inline === -1) {
return fence;
}
return Math.min(fence, inline);
}
function findOpenInlineCodeStart(text: string): number {
let openStart = -1;
let openTicks = 0;
let index = 0;
while (index < text.length) {
if (text[index] !== "`") {
index += 1;
continue;
}
const runStart = index;
let runLength = 0;
while (index < text.length && text[index] === "`") {
runLength += 1;
index += 1;
}
if (openStart === -1) {
openStart = runStart;
openTicks = runLength;
} else if (runLength === openTicks) {
openStart = -1;
openTicks = 0;
}
}
return openStart;
}
function findOpenFenceStart(text: string): number {
const fenceRe = /(^|\n)(```|~~~)[^\n]*(?:\n|$)/g;
let open: { marker: string; index: number } | null = null;
for (const match of text.matchAll(fenceRe)) {
const index = (match.index ?? 0) + match[1].length;
const marker = match[2] ?? "";
if (open !== null && open.marker === marker) {
open = null;
} else if (!open) {
open = { marker, index };
}
}
return open?.index ?? -1;
}
function findTrailingFenceFragmentStart(
text: string,
inlineState: InlineCodeState,
fenceState: FenceScanState | undefined,
): number {
if (inlineState.open || fenceState?.open) {
return -1;
}
const lineStart = Math.max(text.lastIndexOf("\n") + 1, 0);
const line = text.slice(lineStart);
const match = line.match(/^( {0,3})(`{1,2}|~{1,2})$/);
return match ? lineStart : -1;
}

View File

@@ -0,0 +1,28 @@
/**
* Removes unpaired Unicode surrogate characters from a string.
*
* Unpaired surrogates (high surrogates 0xD800-0xDBFF without matching low surrogates 0xDC00-0xDFFF,
* or vice versa) cause JSON serialization errors in many API providers.
*
* Valid emoji and other characters outside the Basic Multilingual Plane use properly paired
* surrogates and will NOT be affected by this function.
*
* @param text - The text to sanitize
* @returns The sanitized text with unpaired surrogates removed
*
* @example
* // Valid emoji (properly paired surrogates) are preserved
* sanitizeSurrogates("Hello 🙈 World") // => "Hello 🙈 World"
*
* // Unpaired high surrogate is removed
* const unpaired = String.fromCharCode(0xD83D); // high surrogate without low
* sanitizeSurrogates(`Text ${unpaired} here`) // => "Text here"
*/
export function sanitizeSurrogates(text: string): string {
// Replace unpaired high surrogates (0xD800-0xDBFF not followed by low surrogate)
// Replace unpaired low surrogates (0xDC00-0xDFFF not preceded by high surrogate)
return text.replace(
/[\uD800-\uDBFF](?![\uDC00-\uDFFF])|(?<![\uD800-\uDBFF])[\uDC00-\uDFFF]/g,
"",
);
}

View File

@@ -0,0 +1,171 @@
import { MAX_TIMER_TIMEOUT_MS } from "@openclaw/normalization-core/number-coercion";
import { describe, expect, it, vi } from "vitest";
import {
createFirstStreamEventAbortController,
withFirstStreamEventTimeout,
} from "./stream-first-event-timeout.js";
function createNeverYieldingStream(onReturn?: () => void): AsyncIterable<unknown> {
return {
[Symbol.asyncIterator]() {
return {
async next() {
return new Promise<IteratorResult<unknown>>(() => {});
},
async return() {
onReturn?.();
return { done: true, value: undefined };
},
};
},
};
}
describe("withFirstStreamEventTimeout", () => {
it("fails when the first event never arrives", async () => {
vi.useFakeTimers();
try {
const stream = withFirstStreamEventTimeout(createNeverYieldingStream(), {
provider: "local",
api: "openai-completions",
model: "test-model",
timeoutMs: 5,
stage: "completions",
});
const iterator = stream[Symbol.asyncIterator]();
const next = expect(iterator.next()).rejects.toThrow(
/completions HTTP stream opened but did not deliver a first SSE event within 5ms after streaming headers \(first-event timeout\)/,
);
await vi.advanceTimersByTimeAsync(5);
await next;
} finally {
vi.useRealTimers();
}
});
it("calls iterator return on first-event timeout", async () => {
vi.useFakeTimers();
try {
const onReturn = vi.fn();
const stream = withFirstStreamEventTimeout(createNeverYieldingStream(onReturn), {
timeoutMs: 5,
});
const iterator = stream[Symbol.asyncIterator]();
const next = iterator.next().catch((error: unknown) => error);
await vi.advanceTimersByTimeAsync(5);
await next;
expect(onReturn).toHaveBeenCalledTimes(1);
} finally {
vi.useRealTimers();
}
});
it("calls iterator return when the consumer closes after the first event", async () => {
const onReturn = vi.fn();
const source: AsyncIterable<unknown> = {
[Symbol.asyncIterator]() {
return {
async next() {
return { done: false, value: "first" };
},
async return() {
onReturn();
return { done: true, value: undefined };
},
};
},
};
const stream = withFirstStreamEventTimeout(source, { timeoutMs: 5 });
const iterator = stream[Symbol.asyncIterator]();
await expect(iterator.next()).resolves.toEqual({ done: false, value: "first" });
await iterator.return?.();
expect(onReturn).toHaveBeenCalledTimes(1);
});
it("aborts the underlying request on first-event timeout", async () => {
vi.useFakeTimers();
try {
const abort = vi.fn();
const onTimeout = vi.fn();
const stream = withFirstStreamEventTimeout(createNeverYieldingStream(), {
timeoutMs: 5,
abort,
onTimeout,
});
const iterator = stream[Symbol.asyncIterator]();
const next = iterator.next().catch((error: unknown) => error);
await vi.advanceTimersByTimeAsync(5);
const error = await next;
expect(error).toBeInstanceOf(Error);
expect(onTimeout).toHaveBeenCalledWith(error);
expect(abort).toHaveBeenCalledWith(error);
} finally {
vi.useRealTimers();
}
});
it("clamps oversized first-event timeouts before scheduling", async () => {
vi.useFakeTimers();
try {
const stream = withFirstStreamEventTimeout(createNeverYieldingStream(), {
timeoutMs: Number.MAX_SAFE_INTEGER,
});
const iterator = stream[Symbol.asyncIterator]();
const next = expect(iterator.next()).rejects.toThrow(
new RegExp(`within ${MAX_TIMER_TIMEOUT_MS}ms`),
);
await vi.advanceTimersByTimeAsync(MAX_TIMER_TIMEOUT_MS);
await next;
} finally {
vi.useRealTimers();
}
});
it("propagates parent aborts through derived first-event signals", () => {
const parent = new AbortController();
const firstEventAbort = createFirstStreamEventAbortController(parent.signal);
parent.abort("run-timeout");
expect(firstEventAbort.signal.aborted).toBe(true);
expect(firstEventAbort.signal.reason).toBe("run-timeout");
firstEventAbort.dispose();
});
it("passes through events after the first event without adding inter-event timing", async () => {
async function* delayedSecondEvent() {
yield "first";
await new Promise((resolve) => {
setTimeout(resolve, 50);
});
yield "second";
}
vi.useFakeTimers();
try {
const stream = withFirstStreamEventTimeout(delayedSecondEvent(), { timeoutMs: 5 });
const iterator = stream[Symbol.asyncIterator]();
await expect(iterator.next()).resolves.toEqual({ done: false, value: "first" });
const second = iterator.next();
await vi.advanceTimersByTimeAsync(50);
await expect(second).resolves.toEqual({ done: false, value: "second" });
} finally {
vi.useRealTimers();
}
});
it("returns the original stream when disabled", () => {
const stream = createNeverYieldingStream();
expect(withFirstStreamEventTimeout(stream, { timeoutMs: 0 })).toBe(stream);
expect(withFirstStreamEventTimeout(stream, { timeoutMs: Number.NaN })).toBe(stream);
});
});

View File

@@ -0,0 +1,134 @@
import { clampTimerTimeoutMs } from "@openclaw/normalization-core/number-coercion";
type StreamStage = "responses" | "completions";
export type FirstStreamEventTimeoutContext = {
provider?: string;
api?: string;
model?: string;
timeoutMs: number;
stage?: StreamStage;
hint?: string;
abort?: (reason: Error) => void;
onTimeout?: (reason: Error) => void;
};
export type FirstStreamEventInternalOptions = {
firstEventTimeoutMs?: number;
abortFirstEventStream?: (reason: Error) => void;
onFirstEventTimeout?: (reason: Error) => void;
};
export type FirstStreamEventAbortController = {
signal: AbortSignal;
abort: (reason: Error) => void;
dispose: () => void;
};
export function getFirstStreamEventTimeoutMs(options: unknown): number | undefined {
return (options as FirstStreamEventInternalOptions | undefined)?.firstEventTimeoutMs;
}
export function getFirstStreamEventTimeoutHandler(
options: unknown,
): ((reason: Error) => void) | undefined {
return (options as FirstStreamEventInternalOptions | undefined)?.onFirstEventTimeout;
}
function formatOptionalField(name: string, value: string | undefined): string {
return value ? ` ${name}=${value}` : "";
}
export function createFirstStreamEventTimeoutError(context: FirstStreamEventTimeoutContext): Error {
const stage = context.stage ? `${context.stage} ` : "";
const details = [
formatOptionalField("provider", context.provider),
formatOptionalField("api", context.api),
formatOptionalField("model", context.model),
].join("");
return new Error(
`${stage}HTTP stream opened but did not deliver a first SSE event within ${context.timeoutMs}ms after streaming headers (first-event timeout).${details}` +
(context.hint ? ` ${context.hint}` : ""),
);
}
export function createFirstStreamEventAbortController(
parentSignal?: AbortSignal,
): FirstStreamEventAbortController {
const controller = new AbortController();
const abortFromParent = () => {
if (!controller.signal.aborted) {
controller.abort(parentSignal?.reason);
}
};
if (parentSignal?.aborted) {
abortFromParent();
} else {
parentSignal?.addEventListener("abort", abortFromParent, { once: true });
}
return {
signal: controller.signal,
abort(reason: Error) {
if (!controller.signal.aborted) {
controller.abort(reason);
}
},
dispose() {
parentSignal?.removeEventListener("abort", abortFromParent);
},
};
}
export function withFirstStreamEventTimeout<T>(
stream: AsyncIterable<T>,
context: FirstStreamEventTimeoutContext,
): AsyncIterable<T> {
const timeoutMs = clampTimerTimeoutMs(context.timeoutMs);
if (timeoutMs === undefined || context.timeoutMs <= 0) {
return stream;
}
const timeoutContext = { ...context, timeoutMs };
return {
async *[Symbol.asyncIterator]() {
const iterator = stream[Symbol.asyncIterator]();
let timer: ReturnType<typeof setTimeout> | undefined;
let completed = false;
const clear = () => {
if (timer) {
clearTimeout(timer);
timer = undefined;
}
};
try {
const first = await new Promise<IteratorResult<T>>((resolve, reject) => {
timer = setTimeout(() => {
const timeoutError = createFirstStreamEventTimeoutError(timeoutContext);
timeoutContext.onTimeout?.(timeoutError);
timeoutContext.abort?.(timeoutError);
reject(timeoutError);
}, timeoutMs);
timer.unref?.();
iterator.next().then(resolve, reject);
}).finally(clear);
if (first.done) {
completed = true;
return;
}
yield first.value;
for (;;) {
const next = await iterator.next();
if (next.done) {
completed = true;
return;
}
yield next.value;
}
} finally {
clear();
if (!completed) {
void iterator.return?.().catch(() => undefined);
}
}
},
};
}

View File

@@ -0,0 +1,87 @@
/**
* Bounded SSE / NDJSON stream reader guard.
*
* Wraps a `ReadableStreamDefaultReader<Uint8Array>` so the caller's existing
* chunk-by-chunk parsing logic is unchanged, but accumulated bytes are tracked
* against a hard cap. On overflow the underlying reader is cancelled and a
* canonical error is thrown. Mirrors the `readResponseWithLimit` / bounded
* JSON response pattern (see `src/agents/provider-http-errors.ts`).
*
* Internal helper for now. If extensions need it, promote to a plugin-SDK
* subpath in a separate, dedicated PR with full SDK metadata sync.
*/
export type SseStreamOverflow = {
size: number;
maxBytes: number;
};
export type ReadSseStreamWithLimitOptions = {
maxBytes: number;
onOverflow?: (params: SseStreamOverflow) => Error;
};
export type SseByteGuard = {
read(): Promise<ReadableStreamReadResult<Uint8Array>>;
cancel(reason?: unknown): Promise<void>;
totalBytes(): number;
overflowed(): boolean;
cancelled(): boolean;
};
export function createSseByteGuard(
reader: ReadableStreamDefaultReader<Uint8Array>,
opts: ReadSseStreamWithLimitOptions,
): SseByteGuard {
if (!Number.isFinite(opts.maxBytes) || opts.maxBytes < 0) {
throw new RangeError(`maxBytes must be a non-negative finite number: ${opts.maxBytes}`);
}
const onOverflow =
opts.onOverflow ??
((params) =>
new Error(`SSE stream exceeds ${params.maxBytes} bytes (received ${params.size})`));
let total = 0;
let overflowedFlag = false;
let cancelledFlag = false;
return {
read: async () => {
if (overflowedFlag || cancelledFlag) {
return { done: true, value: undefined };
}
const result = await reader.read();
if (result.done) {
return result;
}
const chunkLen = result.value?.byteLength ?? 0;
const next = total + chunkLen;
if (next > opts.maxBytes) {
overflowedFlag = true;
cancelledFlag = true;
const err = onOverflow({ size: next, maxBytes: opts.maxBytes });
try {
await reader.cancel(err);
} catch {
// best-effort cancellation; caller observes the overflow error
}
throw err;
}
total = next;
return result;
},
cancel: async (reason?: unknown) => {
if (overflowedFlag) {
// overflow already set cancelledFlag; do not overwrite
return;
}
cancelledFlag = true;
try {
await reader.cancel(reason);
} catch {
// best-effort cancellation
}
},
totalBytes: () => total,
overflowed: () => overflowedFlag,
cancelled: () => cancelledFlag,
};
}

View File

@@ -0,0 +1,92 @@
// System prompt cache-boundary tests cover the internal marker that separates
// stable prompt text from dynamic per-turn additions.
import { describe, expect, it } from "vitest";
import {
ensureSystemPromptCacheBoundary,
prependSystemPromptAdditionAfterCacheBoundary,
splitSystemPromptCacheBoundary,
stripSystemPromptCacheBoundary,
SYSTEM_PROMPT_CACHE_BOUNDARY,
} from "./system-prompt-cache-boundary.js";
describe("system prompt cache boundary helpers", () => {
it("splits stable and dynamic prompt regions", () => {
expect(
splitSystemPromptCacheBoundary(`Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic suffix`),
).toEqual({
stablePrefix: "Stable prefix",
dynamicSuffix: "Dynamic suffix",
});
});
it("strips the internal marker from prompt text", () => {
expect(
stripSystemPromptCacheBoundary(`Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic suffix`),
).toBe("Stable prefix\nDynamic suffix");
});
it("inserts prompt additions after the cache boundary", () => {
expect(
prependSystemPromptAdditionAfterCacheBoundary({
systemPrompt: `Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic suffix`,
systemPromptAddition: "Per-turn lab context",
}),
).toBe(`Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Per-turn lab context\n\nDynamic suffix`);
});
it("normalizes structured additions and dynamic suffix whitespace", () => {
expect(
prependSystemPromptAdditionAfterCacheBoundary({
systemPrompt: `Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic suffix \r\n\r\nMore detail \t\r\n`,
systemPromptAddition: " Per-turn lab context \r\nSecond line\t\r\n",
}),
).toBe(
`Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Per-turn lab context\nSecond line\n\nDynamic suffix\n\nMore detail`,
);
});
});
describe("ensureSystemPromptCacheBoundary", () => {
it("returns a marker-bearing prompt unchanged", () => {
const prompt = `Stable prefix${SYSTEM_PROMPT_CACHE_BOUNDARY}Dynamic suffix`;
expect(ensureSystemPromptCacheBoundary(prompt)).toBe(prompt);
});
it("appends the boundary to a marker-free prompt", () => {
expect(ensureSystemPromptCacheBoundary("Marker-free override")).toBe(
`Marker-free override${SYSTEM_PROMPT_CACHE_BOUNDARY}`,
);
});
it("does not add a boundary for an empty prompt", () => {
expect(ensureSystemPromptCacheBoundary("")).toBe("");
expect(ensureSystemPromptCacheBoundary(" \n\t ")).toBe(" \n\t ");
});
it("uses a per-turn addition directly when the base prompt is empty", () => {
expect(
prependSystemPromptAdditionAfterCacheBoundary({
systemPrompt: ensureSystemPromptCacheBoundary(""),
systemPromptAddition: "Per-turn media task hint",
}),
).toBe("Per-turn media task hint");
});
it("is idempotent for a marker-free prompt", () => {
const once = ensureSystemPromptCacheBoundary("Marker-free override");
expect(ensureSystemPromptCacheBoundary(once)).toBe(once);
});
it("lets a per-turn addition split into the uncached suffix for a marker-free prompt", () => {
// Marker-free overrides become stable prefixes; additions stay in the
// dynamic suffix so prompt-cache bytes remain deterministic.
const result = prependSystemPromptAdditionAfterCacheBoundary({
systemPrompt: ensureSystemPromptCacheBoundary("Marker-free override"),
systemPromptAddition: "Per-turn media task hint",
});
expect(splitSystemPromptCacheBoundary(result)).toEqual({
stablePrefix: "Marker-free override",
dynamicSuffix: "Per-turn media task hint",
});
});
});

View File

@@ -0,0 +1,66 @@
/**
* System prompt cache-boundary helpers.
*
* Keeps stable prompt prefixes separate from dynamic runtime additions for provider prompt caching.
*/
import { normalizeStructuredPromptSection } from "./prompt-cache-stability.js";
export const SYSTEM_PROMPT_CACHE_BOUNDARY = "\n<!-- OPENCLAW_CACHE_BOUNDARY -->\n";
export function stripSystemPromptCacheBoundary(text: string): string {
return text.replaceAll(SYSTEM_PROMPT_CACHE_BOUNDARY, "\n");
}
// Append the cache boundary when a prompt has none (e.g. a hook systemPrompt override),
// so dynamic additions route into an uncached suffix instead of the cached prefix (#85203).
export function ensureSystemPromptCacheBoundary(systemPrompt: string): string {
if (systemPrompt.trim().length === 0) {
return systemPrompt;
}
return systemPrompt.includes(SYSTEM_PROMPT_CACHE_BOUNDARY)
? systemPrompt
: `${systemPrompt}${SYSTEM_PROMPT_CACHE_BOUNDARY}`;
}
export function splitSystemPromptCacheBoundary(
text: string,
): { stablePrefix: string; dynamicSuffix: string } | undefined {
const boundaryIndex = text.indexOf(SYSTEM_PROMPT_CACHE_BOUNDARY);
if (boundaryIndex === -1) {
return undefined;
}
return {
stablePrefix: text.slice(0, boundaryIndex).trimEnd(),
dynamicSuffix: text.slice(boundaryIndex + SYSTEM_PROMPT_CACHE_BOUNDARY.length).trimStart(),
};
}
export function prependSystemPromptAdditionAfterCacheBoundary(params: {
systemPrompt: string;
systemPromptAddition?: string;
}): string {
const systemPromptAddition =
typeof params.systemPromptAddition === "string"
? normalizeStructuredPromptSection(params.systemPromptAddition)
: "";
if (!systemPromptAddition) {
return params.systemPrompt;
}
if (params.systemPrompt.trim().length === 0) {
return systemPromptAddition;
}
const split = splitSystemPromptCacheBoundary(params.systemPrompt);
if (!split) {
return `${systemPromptAddition}\n\n${params.systemPrompt}`;
}
const dynamicSuffix = split.dynamicSuffix
? normalizeStructuredPromptSection(split.dynamicSuffix)
: "";
if (!dynamicSuffix) {
return `${split.stablePrefix}${SYSTEM_PROMPT_CACHE_BOUNDARY}${systemPromptAddition}`;
}
return `${split.stablePrefix}${SYSTEM_PROMPT_CACHE_BOUNDARY}${systemPromptAddition}\n\n${dynamicSuffix}`;
}

View File

@@ -0,0 +1,2 @@
/** Tool argument validation for TypeBox and JSON Schema declarations. */
export * from "@openclaw/llm-core/validation";

View File

@@ -0,0 +1,8 @@
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"rootDir": "../..",
"outDir": "dist"
},
"include": ["src/**/*", "../../src/agents/**/*", "../../src/shared/**/*"]
}