Vendor OpenClaw source as Adolf fork baseline
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Some checks failed
ClawSweeper Dispatch / dispatch (push) Has been cancelled
CodeQL / Security High (actions) (push) Has been cancelled
CodeQL / Security High (channel-runtime-boundary) (push) Has been cancelled
CodeQL / Security High (core-auth-secrets) (push) Has been cancelled
CodeQL / Security High (mcp-process-tool-boundary) (push) Has been cancelled
CodeQL / Security High (network-ssrf-boundary) (push) Has been cancelled
CodeQL / Security High (plugin-trust-boundary) (push) Has been cancelled
CodeQL / Security High (process-exec-boundary) (push) Has been cancelled
Docs Sync Publish Repo / sync-publish-repo (push) Has been cancelled
Docs / docs (push) Has been cancelled
OpenClaw Stable Main Closeout / Resolve stable release closeout inputs (push) Has been cancelled
OpenClaw Stable Main Closeout / Verify stable main closeout (push) Has been cancelled
Workflow Sanity / no-tabs (push) Has been cancelled
Workflow Sanity / actionlint (push) Has been cancelled
Workflow Sanity / generated-doc-baselines (push) Has been cancelled
CI / runner-admission (push) Has been cancelled
CI / preflight (push) Has been cancelled
CI / security-fast (push) Has been cancelled
CI / pnpm-store-warmup (push) Has been cancelled
CI / build-artifacts (push) Has been cancelled
CI / native-i18n (push) Has been cancelled
CI / ${{ matrix.check_name }} (push) Has been cancelled
CI / ${{ matrix.checkName }} (push) Has been cancelled
CI / checks-node-compat-node22 (push) Has been cancelled
CI / check-bundled-channel-config-metadata (push) Has been cancelled
CI / check-dependencies (push) Has been cancelled
CI / check-guards (push) Has been cancelled
CI / check-lint (push) Has been cancelled
CI / check-prod-types (push) Has been cancelled
CI / check-shrinkwrap (push) Has been cancelled
CI / check-test-types (push) Has been cancelled
CI / check-additional-boundaries-a (push) Has been cancelled
CI / check-additional-boundaries-bcd (push) Has been cancelled
CI / check-additional-extension-bundled (push) Has been cancelled
CI / check-additional-extension-channels (push) Has been cancelled
CI / check-additional-extension-package-boundary (push) Has been cancelled
CI / check-additional-runtime-topology-architecture (push) Has been cancelled
CI / check-session-accessor-boundary (push) Has been cancelled
CI / check-session-transcript-reader-boundary (push) Has been cancelled
CI / check-docs (push) Has been cancelled
CI / skills-python (push) Has been cancelled
CI / macos-swift (push) Has been cancelled
CI / ios-build (push) Has been cancelled
CI / ci-timings-summary (push) Has been cancelled
Native App Locale Refresh / Refresh native fa (push) Has been cancelled
Native App Locale Refresh / Refresh native fr (push) Has been cancelled
Native App Locale Refresh / Refresh native hi (push) Has been cancelled
Native App Locale Refresh / Refresh native id (push) Has been cancelled
Native App Locale Refresh / Refresh native it (push) Has been cancelled
Native App Locale Refresh / Refresh native ja-JP (push) Has been cancelled
Control UI Locale Refresh / plan (push) Has been cancelled
Control UI Locale Refresh / Refresh ${{ matrix.locale }} (push) Has been cancelled
Control UI Locale Refresh / Commit control UI locale refresh (push) Has been cancelled
Live Media Runner Image / Build live media runner image (push) Has been cancelled
Native App Locale Refresh / Refresh native ar (push) Has been cancelled
Native App Locale Refresh / Refresh native de (push) Has been cancelled
Native App Locale Refresh / Refresh native es (push) Has been cancelled
Native App Locale Refresh / Refresh native ko (push) Has been cancelled
Native App Locale Refresh / Refresh native nl (push) Has been cancelled
Native App Locale Refresh / Refresh native pl (push) Has been cancelled
Native App Locale Refresh / Refresh native pt-BR (push) Has been cancelled
Native App Locale Refresh / Refresh native ru (push) Has been cancelled
Native App Locale Refresh / Refresh native sv (push) Has been cancelled
Native App Locale Refresh / Refresh native th (push) Has been cancelled
Native App Locale Refresh / Refresh native tr (push) Has been cancelled
Native App Locale Refresh / Refresh native uk (push) Has been cancelled
Native App Locale Refresh / Refresh native vi (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-CN (push) Has been cancelled
Native App Locale Refresh / Refresh native zh-TW (push) Has been cancelled
Native App Locale Refresh / Commit native locale refresh (push) Has been cancelled
Plugin Init Scaffold Validation / Validate provider scaffold (push) Has been cancelled
Plugin NPM Release / preview_plugins_npm (push) Has been cancelled
Plugin NPM Release / Validate release publish approval (push) Has been cancelled
Plugin NPM Release / preview_plugin_pack (push) Has been cancelled
Plugin NPM Release / publish_plugins_npm (push) Has been cancelled
Sandbox Common Smoke / sandbox-common-smoke (push) Has been cancelled
Website Installer Sync / static (push) Has been cancelled
Website Installer Sync / linux-docker (push) Has been cancelled
Website Installer Sync / macos-installer (push) Has been cancelled
Website Installer Sync / windows-installer (push) Has been cancelled
Website Installer Sync / sync-website (push) Has been cancelled
Adolf is a fork/vendored clone of github.com/openclaw/openclaw (v2026.6.11), free to diverge. Tree copied sans upstream .git; upstream remote added for future syncs. Node pinned to 24 (.nvmrc); engines already require >=22.19. Preserves docs/ARCHITECTURE.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeqyaxJF2nbRXJtae2kNB2
This commit is contained in:
140
scripts/e2e/lib/openai-web-search-minimal/assertions.mjs
Normal file
140
scripts/e2e/lib/openai-web-search-minimal/assertions.mjs
Normal file
@@ -0,0 +1,140 @@
|
||||
// Assertions for minimal OpenAI web-search E2E scenarios.
|
||||
import fs from "node:fs";
|
||||
import { readTextFileTail, tailText } from "../text-file-utils.mjs";
|
||||
|
||||
const command = process.argv[2];
|
||||
|
||||
const ERROR_DETAIL_TAIL_BYTES = 64 * 1024;
|
||||
const REQUEST_LOG_SCAN_CHUNK_BYTES = 64 * 1024;
|
||||
const RESPONSE_PREVIEW_BYTES = 8 * 1024;
|
||||
const RESPONSE_PREVIEW_COUNT = 5;
|
||||
|
||||
function scanTextFileLines(file, onLine) {
|
||||
const fd = fs.openSync(file, "r");
|
||||
try {
|
||||
const buffer = Buffer.alloc(REQUEST_LOG_SCAN_CHUNK_BYTES);
|
||||
let carry = "";
|
||||
let lineNumber = 1;
|
||||
while (true) {
|
||||
const bytesRead = fs.readSync(fd, buffer, 0, buffer.length, null);
|
||||
if (bytesRead <= 0) {
|
||||
break;
|
||||
}
|
||||
const text = carry + buffer.subarray(0, bytesRead).toString("utf8");
|
||||
const lines = text.split(/\r?\n/u);
|
||||
carry = lines.pop() ?? "";
|
||||
for (const line of lines) {
|
||||
onLine(line, lineNumber);
|
||||
lineNumber += 1;
|
||||
}
|
||||
}
|
||||
if (carry.length > 0) {
|
||||
onLine(carry, lineNumber);
|
||||
}
|
||||
} finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
}
|
||||
|
||||
function scanSuccessRequest(logPath) {
|
||||
let responseCount = 0;
|
||||
let success;
|
||||
const recentResponses = [];
|
||||
scanTextFileLines(logPath, (line, lineNumber) => {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) {
|
||||
return;
|
||||
}
|
||||
const entry = JSON.parse(trimmed);
|
||||
if (entry.method !== "POST" || entry.path !== "/v1/responses") {
|
||||
return;
|
||||
}
|
||||
responseCount += 1;
|
||||
const bodyText = JSON.stringify(entry.body);
|
||||
if (recentResponses.length >= RESPONSE_PREVIEW_COUNT) {
|
||||
recentResponses.shift();
|
||||
}
|
||||
recentResponses.push({
|
||||
line: lineNumber,
|
||||
bodyTail: tailText(bodyText, RESPONSE_PREVIEW_BYTES),
|
||||
});
|
||||
if (!success && bodyText.includes("OPENCLAW_SCHEMA_E2E_OK")) {
|
||||
success = entry;
|
||||
}
|
||||
});
|
||||
return { responseCount, success, recentResponses };
|
||||
}
|
||||
|
||||
function assertPatchBehavior() {
|
||||
return import("../../../../dist/extensions/openai/native-web-search.js").then(
|
||||
({ patchOpenAINativeWebSearchPayload }) => {
|
||||
const injectedPayload = {
|
||||
reasoning: { effort: "minimal", summary: "auto" },
|
||||
};
|
||||
const injectedResult = patchOpenAINativeWebSearchPayload(injectedPayload);
|
||||
if (injectedResult !== "injected") {
|
||||
throw new Error(`expected native web_search injection, got ${injectedResult}`);
|
||||
}
|
||||
if (injectedPayload.reasoning.effort !== "low") {
|
||||
throw new Error(
|
||||
`expected injected native web_search to raise minimal reasoning to low, got ${JSON.stringify(injectedPayload.reasoning)}`,
|
||||
);
|
||||
}
|
||||
if (!injectedPayload.tools?.some((tool) => tool?.type === "web_search")) {
|
||||
throw new Error(`native web_search was not injected: ${JSON.stringify(injectedPayload)}`);
|
||||
}
|
||||
|
||||
const existingNativePayload = {
|
||||
tools: [{ type: "web_search" }],
|
||||
reasoning: { effort: "minimal" },
|
||||
};
|
||||
const existingResult = patchOpenAINativeWebSearchPayload(existingNativePayload);
|
||||
if (existingResult !== "native_tool_already_present") {
|
||||
throw new Error(`expected existing native web_search, got ${existingResult}`);
|
||||
}
|
||||
if (existingNativePayload.reasoning.effort !== "low") {
|
||||
throw new Error(
|
||||
`expected existing native web_search to raise minimal reasoning to low, got ${JSON.stringify(existingNativePayload.reasoning)}`,
|
||||
);
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
function assertSuccessRequest() {
|
||||
const logPath = process.argv[3];
|
||||
const { responseCount, success, recentResponses } = scanSuccessRequest(logPath);
|
||||
if (responseCount < 1) {
|
||||
throw new Error(
|
||||
`mock OpenAI /v1/responses was not used. Request log tail: ${readTextFileTail(logPath, ERROR_DETAIL_TAIL_BYTES)}`,
|
||||
);
|
||||
}
|
||||
if (!success) {
|
||||
throw new Error(
|
||||
`missing success request. Recent /v1/responses: ${JSON.stringify(recentResponses)}`,
|
||||
);
|
||||
}
|
||||
const tools = Array.isArray(success.body.tools) ? success.body.tools : [];
|
||||
const hasNativeWebSearch = tools.some((tool) => tool?.type === "web_search");
|
||||
if (!hasNativeWebSearch) {
|
||||
throw new Error(
|
||||
`success request did not include native web_search. Body: ${JSON.stringify(success.body)}`,
|
||||
);
|
||||
}
|
||||
if (success.body.reasoning?.effort === "minimal") {
|
||||
throw new Error(
|
||||
`expected web_search request to avoid minimal reasoning, got ${JSON.stringify(success.body.reasoning)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const commands = {
|
||||
"assert-patch-behavior": assertPatchBehavior,
|
||||
"assert-success-request": assertSuccessRequest,
|
||||
};
|
||||
|
||||
const fn = commands[command];
|
||||
if (!fn) {
|
||||
throw new Error(`unknown OpenAI web-search minimal assertion command: ${command}`);
|
||||
}
|
||||
await fn();
|
||||
201
scripts/e2e/lib/openai-web-search-minimal/client.mjs
Normal file
201
scripts/e2e/lib/openai-web-search-minimal/client.mjs
Normal file
@@ -0,0 +1,201 @@
|
||||
// Client script for minimal OpenAI web-search E2E scenarios.
|
||||
import { readdirSync } from "node:fs";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { readTcpPortEnv } from "../env-limits.mjs";
|
||||
|
||||
async function loadCallGateway() {
|
||||
const candidates = readdirSync("/app/dist")
|
||||
.filter((name) => /^call(?:\.runtime)?-[A-Za-z0-9_-]+\.js$/.test(name))
|
||||
.toSorted();
|
||||
for (const name of candidates) {
|
||||
const mod = await import(pathToFileURL(`/app/dist/${name}`).href);
|
||||
if (typeof mod.callGateway === "function") {
|
||||
return mod.callGateway;
|
||||
}
|
||||
}
|
||||
throw new Error(`unable to find callGateway export in /app/dist (${candidates.join(", ")})`);
|
||||
}
|
||||
|
||||
const DEFAULT_RAW_SCHEMA_ERROR =
|
||||
"400 The following tools cannot be used with reasoning.effort 'minimal': web_search.";
|
||||
const DEFAULT_GATEWAY_SCHEMA_ERROR = "provider rejected the request schema or tool payload";
|
||||
const SUCCESS_MARKER = "OPENCLAW_SCHEMA_E2E_OK";
|
||||
|
||||
function readExpectedRawSchemaError() {
|
||||
return process.env.RAW_SCHEMA_ERROR?.trim() || DEFAULT_RAW_SCHEMA_ERROR;
|
||||
}
|
||||
|
||||
function resolveGatewayPort(env = process.env) {
|
||||
const portText = env.PORT;
|
||||
if (!portText) {
|
||||
throw new Error("missing PORT");
|
||||
}
|
||||
return readTcpPortEnv("PORT", portText, env);
|
||||
}
|
||||
|
||||
async function gatewayAgent(params) {
|
||||
const token = process.env.OPENCLAW_GATEWAY_TOKEN;
|
||||
if (!token) {
|
||||
throw new Error("missing PORT/OPENCLAW_GATEWAY_TOKEN");
|
||||
}
|
||||
const port = resolveGatewayPort();
|
||||
|
||||
try {
|
||||
const callGateway = await loadCallGateway();
|
||||
return {
|
||||
ok: true,
|
||||
value: await callGateway({
|
||||
url: `ws://127.0.0.1:${port}`,
|
||||
token,
|
||||
method: "agent",
|
||||
params,
|
||||
expectFinal: true,
|
||||
timeoutMs: 240_000,
|
||||
clientName: "gateway-client",
|
||||
mode: "backend",
|
||||
scopes: ["operator.write"],
|
||||
deviceIdentity: null,
|
||||
}),
|
||||
};
|
||||
} catch (error) {
|
||||
const combined = String(error);
|
||||
return { ok: false, error: new Error(combined) };
|
||||
}
|
||||
}
|
||||
|
||||
function stringifyError(value) {
|
||||
return value instanceof Error ? value.message || String(value) : String(value);
|
||||
}
|
||||
|
||||
function validateRejectResult(result, expectedRawSchemaError = readExpectedRawSchemaError()) {
|
||||
if (result.ok) {
|
||||
throw new Error(`reject mode unexpectedly completed: ${JSON.stringify(result.value)}`);
|
||||
}
|
||||
const errorText = stringifyError(result.error);
|
||||
if (
|
||||
!errorText.includes(expectedRawSchemaError) &&
|
||||
!errorText.includes(DEFAULT_GATEWAY_SCHEMA_ERROR)
|
||||
) {
|
||||
throw new Error(
|
||||
`reject mode failed for an unexpected reason; expected ${JSON.stringify(
|
||||
expectedRawSchemaError,
|
||||
)} or ${JSON.stringify(DEFAULT_GATEWAY_SCHEMA_ERROR)} in ${JSON.stringify(errorText)}`,
|
||||
);
|
||||
}
|
||||
return errorText;
|
||||
}
|
||||
|
||||
function pushStringText(texts, value) {
|
||||
if (typeof value === "string" && value.trim().length > 0) {
|
||||
texts.push(value);
|
||||
}
|
||||
}
|
||||
|
||||
function pushContentText(texts, content) {
|
||||
if (typeof content === "string") {
|
||||
pushStringText(texts, content);
|
||||
return;
|
||||
}
|
||||
if (!Array.isArray(content)) {
|
||||
return;
|
||||
}
|
||||
for (const item of content) {
|
||||
if (typeof item === "string") {
|
||||
pushStringText(texts, item);
|
||||
} else if (item && typeof item === "object") {
|
||||
pushStringText(texts, item.text);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function extractSuccessReplyTexts(value) {
|
||||
const texts = [];
|
||||
pushSuccessReplyTexts(texts, value);
|
||||
pushSuccessReplyTexts(texts, value?.result);
|
||||
return texts;
|
||||
}
|
||||
|
||||
function pushSuccessReplyTexts(texts, value) {
|
||||
pushStringText(texts, value?.finalAssistantVisibleText);
|
||||
pushStringText(texts, value?.meta?.finalAssistantVisibleText);
|
||||
pushContentText(texts, value?.message?.content);
|
||||
for (const payload of Array.isArray(value?.payloads) ? value.payloads : []) {
|
||||
if (payload?.isError === true) {
|
||||
continue;
|
||||
}
|
||||
pushStringText(texts, payload?.text);
|
||||
pushContentText(texts, payload?.content);
|
||||
}
|
||||
}
|
||||
|
||||
function validateSuccessResult(result, marker = SUCCESS_MARKER) {
|
||||
if (result.value?.status !== "ok") {
|
||||
throw new Error(`agent run did not complete successfully: ${JSON.stringify(result.value)}`);
|
||||
}
|
||||
const replyTexts = extractSuccessReplyTexts(result.value);
|
||||
if (!replyTexts.some((text) => text.includes(marker))) {
|
||||
throw new Error(
|
||||
`agent run completed without success marker ${JSON.stringify(marker)} in final reply: ${JSON.stringify(
|
||||
result.value,
|
||||
)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const mode = process.argv[2];
|
||||
const sessionKey = `agent:main:openai-web-search-minimal:${mode}`;
|
||||
const message = mode === "reject" ? "FORCE_SCHEMA_REJECT" : `Return exactly ${SUCCESS_MARKER}.`;
|
||||
const id = mode === "reject" ? "schema-reject" : "schema-success";
|
||||
|
||||
const result = await gatewayAgent({
|
||||
sessionKey,
|
||||
message,
|
||||
thinking: "minimal",
|
||||
deliver: false,
|
||||
timeout: 180,
|
||||
idempotencyKey: id,
|
||||
});
|
||||
|
||||
if (mode === "reject") {
|
||||
console.error(validateRejectResult(result));
|
||||
return;
|
||||
}
|
||||
if (!result.ok) {
|
||||
throw toLintErrorObject(result.error, "Non-Error thrown");
|
||||
}
|
||||
validateSuccessResult(result);
|
||||
}
|
||||
|
||||
function toLintErrorObject(value, fallbackMessage) {
|
||||
if (value instanceof Error) {
|
||||
return value;
|
||||
}
|
||||
if (typeof value === "string") {
|
||||
return new Error(value);
|
||||
}
|
||||
const error = new Error(fallbackMessage, { cause: value });
|
||||
if ((typeof value === "object" && value !== null) || typeof value === "function") {
|
||||
Object.assign(error, value);
|
||||
}
|
||||
return error;
|
||||
}
|
||||
|
||||
if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
|
||||
try {
|
||||
await main();
|
||||
} catch (error) {
|
||||
console.error(error instanceof Error ? error.stack || error.message : String(error));
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
export const testing = {
|
||||
DEFAULT_GATEWAY_SCHEMA_ERROR,
|
||||
DEFAULT_RAW_SCHEMA_ERROR,
|
||||
SUCCESS_MARKER,
|
||||
extractSuccessReplyTexts,
|
||||
resolveGatewayPort,
|
||||
validateSuccessResult,
|
||||
validateRejectResult,
|
||||
};
|
||||
166
scripts/e2e/lib/openai-web-search-minimal/mock-server.mjs
Normal file
166
scripts/e2e/lib/openai-web-search-minimal/mock-server.mjs
Normal file
@@ -0,0 +1,166 @@
|
||||
// Mock server for minimal OpenAI web-search E2E scenarios.
|
||||
import http from "node:http";
|
||||
import { readTcpPortEnv } from "../env-limits.mjs";
|
||||
import {
|
||||
boundedRequestLogBody,
|
||||
isRequestBodyTooLargeError,
|
||||
readBody,
|
||||
writeRequestLogEntryOrFail,
|
||||
writeJson,
|
||||
writeSse,
|
||||
} from "../mock-openai-http.mjs";
|
||||
|
||||
const port = readTcpPortEnv("MOCK_PORT");
|
||||
const requestLog = process.env.MOCK_REQUEST_LOG;
|
||||
const successMarker = process.env.SUCCESS_MARKER;
|
||||
const rawSchemaError = process.env.RAW_SCHEMA_ERROR;
|
||||
|
||||
function writeOpenAiReject(res) {
|
||||
writeJson(res, 400, {
|
||||
error: {
|
||||
message: rawSchemaError.replace(/^400\s+/, ""),
|
||||
type: "invalid_request_error",
|
||||
code: "invalid_request_error",
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
function hasWebSearchTool(tools) {
|
||||
return (
|
||||
Array.isArray(tools) &&
|
||||
tools.some((tool) => {
|
||||
if (!tool || typeof tool !== "object") {
|
||||
return false;
|
||||
}
|
||||
if (tool.type === "web_search") {
|
||||
return true;
|
||||
}
|
||||
if (tool.type === "function" && tool.name === "web_search") {
|
||||
return true;
|
||||
}
|
||||
if (tool.type === "function" && tool.function?.name === "web_search") {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
function bodyContainsForceReject(body) {
|
||||
return JSON.stringify(body).includes("FORCE_SCHEMA_REJECT");
|
||||
}
|
||||
|
||||
function responseEvents(text) {
|
||||
return [
|
||||
{
|
||||
type: "response.output_item.added",
|
||||
item: {
|
||||
type: "message",
|
||||
id: "msg_schema_e2e_1",
|
||||
role: "assistant",
|
||||
content: [],
|
||||
status: "in_progress",
|
||||
},
|
||||
},
|
||||
{
|
||||
type: "response.output_item.done",
|
||||
item: {
|
||||
type: "message",
|
||||
id: "msg_schema_e2e_1",
|
||||
role: "assistant",
|
||||
status: "completed",
|
||||
content: [{ type: "output_text", text, annotations: [] }],
|
||||
},
|
||||
},
|
||||
{
|
||||
type: "response.completed",
|
||||
response: {
|
||||
id: "resp_schema_e2e_1",
|
||||
status: "completed",
|
||||
usage: {
|
||||
input_tokens: 11,
|
||||
output_tokens: 7,
|
||||
total_tokens: 18,
|
||||
input_tokens_details: { cached_tokens: 0 },
|
||||
},
|
||||
},
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
const server = http.createServer((req, res) => {
|
||||
void (async () => {
|
||||
const url = new URL(req.url ?? "/", "http://127.0.0.1");
|
||||
if (req.method === "GET" && url.pathname === "/health") {
|
||||
writeJson(res, 200, { ok: true });
|
||||
return;
|
||||
}
|
||||
if (req.method === "GET" && url.pathname === "/v1/models") {
|
||||
writeJson(res, 200, {
|
||||
object: "list",
|
||||
data: [{ id: "gpt-5", object: "model", owned_by: "openclaw-e2e" }],
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
let bodyText;
|
||||
try {
|
||||
bodyText = await readBody(req);
|
||||
} catch (error) {
|
||||
if (isRequestBodyTooLargeError(error)) {
|
||||
writeJson(res, 413, { error: { message: error.message } });
|
||||
return;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
let body;
|
||||
try {
|
||||
body = bodyText ? JSON.parse(bodyText) : {};
|
||||
} catch {
|
||||
body = {};
|
||||
}
|
||||
if (
|
||||
writeRequestLogEntryOrFail(res, {
|
||||
requestLog,
|
||||
required: true,
|
||||
label: "mock-openai-web-search",
|
||||
entry: {
|
||||
method: req.method,
|
||||
path: url.pathname,
|
||||
body: boundedRequestLogBody(body, bodyText),
|
||||
},
|
||||
})
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (req.method === "POST" && url.pathname === "/v1/responses") {
|
||||
if (bodyContainsForceReject(body)) {
|
||||
writeOpenAiReject(res);
|
||||
return;
|
||||
}
|
||||
if (body?.reasoning?.effort === "minimal" && hasWebSearchTool(body.tools)) {
|
||||
writeOpenAiReject(res);
|
||||
return;
|
||||
}
|
||||
writeSse(res, responseEvents(successMarker));
|
||||
return;
|
||||
}
|
||||
|
||||
writeJson(res, 404, {
|
||||
error: { message: `unhandled mock route: ${req.method} ${url.pathname}` },
|
||||
});
|
||||
})().catch((/** @type {unknown} */ error) => {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
console.error(`mock-openai-web-search request handler failed: ${message}`);
|
||||
if (!res.headersSent) {
|
||||
writeJson(res, 500, { error: { message: `mock OpenAI handler failed: ${message}` } });
|
||||
return;
|
||||
}
|
||||
res.destroy(error instanceof Error ? error : new Error(message));
|
||||
});
|
||||
});
|
||||
|
||||
server.listen(port, "127.0.0.1", () => {
|
||||
console.log(`mock-openai listening on ${port}`);
|
||||
});
|
||||
91
scripts/e2e/lib/openai-web-search-minimal/scenario.sh
Normal file
91
scripts/e2e/lib/openai-web-search-minimal/scenario.sh
Normal file
@@ -0,0 +1,91 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
source scripts/lib/openclaw-e2e-instance.sh
|
||||
openclaw_e2e_eval_test_state_from_b64 "${OPENCLAW_TEST_STATE_SCRIPT_B64:?missing OPENCLAW_TEST_STATE_SCRIPT_B64}"
|
||||
export OPENCLAW_SKIP_CHANNELS=1
|
||||
export OPENCLAW_SKIP_GMAIL_WATCHER=1
|
||||
export OPENCLAW_SKIP_CRON=1
|
||||
export OPENCLAW_SKIP_CANVAS_HOST=1
|
||||
export OPENCLAW_SKIP_BROWSER_CONTROL_SERVER=1
|
||||
export OPENCLAW_SKIP_ACPX_RUNTIME=1
|
||||
export OPENCLAW_SKIP_ACPX_RUNTIME_PROBE=1
|
||||
|
||||
PORT="${PORT:?missing PORT}"
|
||||
MOCK_PORT="${MOCK_PORT:?missing MOCK_PORT}"
|
||||
TOKEN="${OPENCLAW_GATEWAY_TOKEN:?missing OPENCLAW_GATEWAY_TOKEN}"
|
||||
SUCCESS_MARKER="OPENCLAW_SCHEMA_E2E_OK"
|
||||
RAW_SCHEMA_ERROR="400 The following tools cannot be used with reasoning.effort 'minimal': web_search."
|
||||
scenario_tmp="$(mktemp -d "${TMPDIR:-/tmp}/openclaw-openai-web-search-minimal.XXXXXX")"
|
||||
MOCK_REQUEST_LOG="$scenario_tmp/requests.jsonl"
|
||||
GATEWAY_LOG="$scenario_tmp/gateway.log"
|
||||
MOCK_LOG="$scenario_tmp/mock.log"
|
||||
CLIENT_SUCCESS_LOG="$scenario_tmp/client-success.log"
|
||||
CLIENT_REJECT_LOG="$scenario_tmp/client-reject.log"
|
||||
mock_pid=""
|
||||
gateway_pid=""
|
||||
|
||||
cleanup() {
|
||||
openclaw_e2e_terminate_gateways "${gateway_pid:-}"
|
||||
openclaw_e2e_stop_process "${mock_pid:-}"
|
||||
rm -rf "$scenario_tmp"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
dump_debug_logs() {
|
||||
local status="$1"
|
||||
echo "OpenAI web_search minimal Docker E2E failed with exit code $status" >&2
|
||||
for file in \
|
||||
"$GATEWAY_LOG" \
|
||||
"$MOCK_LOG" \
|
||||
"$CLIENT_SUCCESS_LOG" \
|
||||
"$CLIENT_REJECT_LOG" \
|
||||
"$MOCK_REQUEST_LOG" \
|
||||
"$OPENCLAW_STATE_DIR/openclaw.json"; do
|
||||
if [ -f "$file" ]; then
|
||||
echo "--- $file ---" >&2
|
||||
openclaw_e2e_print_log "$file" >&2
|
||||
fi
|
||||
done
|
||||
}
|
||||
trap 'status=$?; dump_debug_logs "$status"; exit "$status"' ERR
|
||||
|
||||
entry="$(openclaw_e2e_resolve_entrypoint)"
|
||||
mkdir -p "$OPENCLAW_STATE_DIR"
|
||||
|
||||
node scripts/e2e/lib/openai-web-search-minimal/assertions.mjs assert-patch-behavior
|
||||
|
||||
node scripts/e2e/lib/fixture.mjs openai-web-search-minimal-config
|
||||
|
||||
MOCK_PORT="$MOCK_PORT" \
|
||||
MOCK_REQUEST_LOG="$MOCK_REQUEST_LOG" \
|
||||
SUCCESS_MARKER="$SUCCESS_MARKER" \
|
||||
RAW_SCHEMA_ERROR="$RAW_SCHEMA_ERROR" \
|
||||
node scripts/e2e/lib/openai-web-search-minimal/mock-server.mjs >"$MOCK_LOG" 2>&1 &
|
||||
mock_pid="$!"
|
||||
|
||||
openclaw_e2e_wait_mock_openai "$MOCK_PORT"
|
||||
|
||||
gateway_pid="$(openclaw_e2e_start_gateway "$entry" "$PORT" "$GATEWAY_LOG")"
|
||||
openclaw_e2e_wait_gateway_ready "$gateway_pid" "$GATEWAY_LOG" 360 "$PORT"
|
||||
node "$entry" gateway health \
|
||||
--url "ws://127.0.0.1:$PORT" \
|
||||
--token "$TOKEN" \
|
||||
--timeout 120000 \
|
||||
--json >/dev/null
|
||||
|
||||
PORT="$PORT" OPENCLAW_GATEWAY_TOKEN="$TOKEN" node scripts/e2e/lib/openai-web-search-minimal/client.mjs success >"$CLIENT_SUCCESS_LOG" 2>&1
|
||||
|
||||
node scripts/e2e/lib/openai-web-search-minimal/assertions.mjs assert-success-request "$MOCK_REQUEST_LOG"
|
||||
|
||||
PORT="$PORT" OPENCLAW_GATEWAY_TOKEN="$TOKEN" node scripts/e2e/lib/openai-web-search-minimal/client.mjs reject >"$CLIENT_REJECT_LOG" 2>&1
|
||||
|
||||
for _ in $(seq 1 80); do
|
||||
if grep -Fq "$RAW_SCHEMA_ERROR" "$GATEWAY_LOG"; then
|
||||
break
|
||||
fi
|
||||
sleep 0.25
|
||||
done
|
||||
grep -F "$RAW_SCHEMA_ERROR" "$GATEWAY_LOG" >/dev/null
|
||||
|
||||
echo "OpenAI web_search minimal reasoning Docker E2E passed"
|
||||
Reference in New Issue
Block a user